Magento - How to switch Base URLs without modifying the Database

As Magento keeps its base URL in database, there is no easy way of accessing the same database from two different URLs(dev/ staging) or moving the same database to staging environment from development without updating core_config_data table.

After fiddling around with Magento core, I found that getBaseUrl() is responsible of retrieving base URL. So If we could override getBaseUrl() method to get the base URL from dev or staging environments without depending on the database, then we could keep database records with live settings.

Below code will explain how I achieved this task and the code itself is explainable.

1. app/etc/modules/My_Configurator.xml

    
        
            true
            local
        
    

2. app/code/local/My/Configurator/etc/config.xml

    
        
            
                
                    My_Configurator_Model_Store
                
            
        
    

3. app/code/local/My/Configurator/Model/Store.php

Note: Make sure you disable this extension from production server as we want to run getBaseUrl() method from the Core.

Happy Coding..

Why Symfony2?

With the PHP5 there was a big hype in PHP frameworks and MVC pattern. As a result, lots of developers got lost within frameworks while trying to figure out the best.

I was one of them.

When it comes to PHP frameworks, ZEND, CodeIgniter were my favourites.  I used CodeIgniter for simple applications and ZEND framework was the giant who saved my good hours. But I never enjoyed PHP with frameworks. Frameworks are good for just get things done.

After I joined with August I got chance to work with Symfony. Not just Symfony, It's Symfony2 :)

Symfony2 is one of a kind framework, it is not a MVC framework. If you look at the Symfony2 architecture, you could find controllers, views but no models. It has been developed as a HTTP framework.

That is how the web works.

The biggest concern in Symfony2 is decoupling and from the core it has given reusable set of standalone components to use. Simply it has provided a set of tools to develop applications as we want.

At the beginning, I was really frustrated because it seems to be complicated and over engineered. however after few days of work I started liking how it works.

The most impressive part in Symfony2 is the 'Bundle' concept. If you write a Symfony2 bundle, simply you can run it within any Symfony2 project without a hassle.  A bundle contains it's own configurations, routing rules, actions, templates and business logic codes.

Dependency injection is another powerful feature which we can commonly see in JAVA developments. That means Symfony2 has adopted lots of best things from other languages without trying to reinvent the wheel.

According the the Symfony2 documentation, it contains 21 standalone libraries which we can use for day-to-day developments.

1. DependencyInjection
2. EventDispatcher
3. HttpFoundation
4. DomCrawler
5. ClassLoader
6. CssSelector
7. HttpKernel
8. BrowserKit
9. Templating
10. Translation
11. Serializer
12. Validator
13. Security
14. Routing
15. Console
16. Process
17. Config
18. Finder
19. Locale
20. Yaml
21. Form

It is not an easy task to explain the awesomeness of Symfony2 in a single blog post :)

This is going to be the very first of many blog posts about Symfony2. I'll be writing some tutorials specially for beginners who are willing to use Symfony2 in their next projects.

Drupal scaling and performance tuning - Part 3

Running a Developer Portal is not an easy task as I thought. When it comes to WSO2 Oxygentank It is more critical as WSO2 community is heavily depend on the portal.

To be frankly if the site went offline for 1 minute, I used to get more than 20 email /  IM chats or calls. Most of the solutions described in the web really didn't work for us. So we had to find our own way to stabilize the system. After spending so much of time configuring and tuning we ended up with a scalable and stable solution. In part 1 and 2 I have described methods which we used.

During peak hours Our portal started to send 502 Bad gateway message from Nginx. The reason was due to mysql high-load in master 1 server, Nginx didn't receive the response with in defined time. 

Solution was we configured Nginx with backup nodes setting instead of setting up 4 nodes with weight balancing to handle the load.

Sample nginx.conf

upstream wso2.org {

    server node1server:80 weight=5 fail_timeout=20s;

    server node2server.org:80 weight=5 fail_timeout=20s;

    server node3server:80 backup;

    server node4server:80 backup;

  }

with this setup 

1. Nginx always send traffic to node1 and node2 servers
2. Primary data source is master1 mysql instance.
3. Secondary data source is master2 MySQL instance

When node1 or node2 couldn't response to Nginx with in 20 seconds we route the traffic to backup nodes

1. Nginx send traffic to node3 and node4 when node1 and node2 can not response 
2. Primary data source is master2 MySQL Instance
3. Secondary data source is master1 MySQL instance

with that setting we were able to keep the site with 0% downtime.

(Overview of WSO2 Oxygentank)

Nginx always looked after load balancing when http/s traffic get high and with the backup node method we were able to keep our MySQL instance up and running without getting meltdown.   

Drupal scaling and performance tuning - Part 2

Drupal is highly relational.  When it comes to performance, MySQL has a big role to play in Drupal world. As I explained in Drupal scaling and performance tuning - Part 1, we were able tune Apache to handle much load during high traffic hours. But MySQL didn't give a chance to rest.

As the first step, we decided to look at the MySQL slow queries to identify bad queries. Server logged all queries that took more than 2 seconds to process and most of them were node permissions, user sessions, access log, cache, comments, watchdog and node contents.

The server load was gone up and it was almost at the frozen state most of the time. As a solution, I knew that we would have to end up with a modified MySQL configuration file.

But, What are the Parameters I need to change and what are the values for them, like i did for Apache?

Then I came across with a tool called MySQLTuner-perl which is a script written in Perl that allows us to review a MySQL installation quickly and make adjustments to increase performance and stability.  The current configuration variables and status data are retrieved and presented in a brief format along with some basic performance suggestions

Tools used

1. MySQLTuner-perl
2.  Maatkit ( Power tools for open-source databases)

Mysql Optimization - I will not list all configurations here as MySQL Tuner gives you a good guide

1. MySQL Tuner
1. Download the Mysql Tuner (wget https://github.com/rackerhacker/MySQLTuner-perl/blob/master/mysqltuner.pl)
2. Make it executable (chmod +x mysqltuner.pl)
3. Run MySQL Turner - You need your MySQL root password in order to execute this (./mysqltuner.pl) 
4.  You should carefully read the output, especially the recommendations at the end. It shows exactly which variables you should adjust in the [mysqld] section of your my.cnf (on Debian and Ubuntu the full path is /etc/mysql/my.cnf). Whenever you change your my.cnf, make sure that you restart MySQL. You can then run MySQLTuner again to see if it has further recommendations to improve the MySQL performance. This way, you can optimize MySQL step by step.
2. Maatkit
1. mk-duplicate-key-checker - helped me to find duplicate indexes and foreign keys on MySQL tables. 
2. Removed all duplicate indexes and foreign keys - This helped MySQL to process SQLs smoothly.
3. mk-query-digest and mk-query-profiler helped to profile and test new configurations/ Modifications to the database.
4.  mk-variable-advisor to double check changes and recommendations made by MySQL Tuner.
3. Converted comments, node, users tables from MyISAM to InnoDB
4. mysqlcheck -o -A -p command optimized other tables in the Drupal Database

After going through few cycles of above points, we were able to get the MySQL server to a level that it can handle much load without any hiccups. Slow query log didn't report any slow quries. SHOW PROCESSLIST was always below 20 - 30.

Other Optimizations used

1. Disable watchdog, Statistic modules from Drupal to reduce the read/write load to MySQL
2. Uninstalled all unused modules.

Finally I was able to get Our Oxygentank Database to a state where it can breathe freely without running on full power all the time. 

But I was not happy with this setting since the second MySQL master server was always running silently without helping the Primary Master server to handle its load.

I used NGINX to solve this problem.

I will discuss how I used NGINX to share MySQL load between Databases / Drupal nodes with the help of Memcached, in part 3.

Drupal scaling and performance tuning - Part 1

As WSO2Con 2011 was a huge hit in IT sector, I had to face a problem with wso2.org (Oxygentank developer Portal). Which site couldn't handle a large traffic constantly.

The old system we had was NGINX (Load Balancer) fronted 4 Drupal nodes which was running with Apache2. with master - master MySQL replication.

But during a high-traffic Our servers got more than 1 minute (average) time to response  on a request. As  NGINX's fail timeout for 45 seconds, users got 504 Gateway timeout most of the time. Same time Apache server load went above 30 - 70.

First as a quick solution we spinned new 2 servers and routed the load across them. That helped us to cater for few hours but again there was a huge delay when site was loading. Then we found that MySQL server load also gone up and process-list has grown.

Then I found that MySQL Server take time process queries than it was before. Root cause for this new problem was 6 drupal nodes had started stress MySQL servers continually.

Then I started to ask some questioned by my self. 

1. OK, Then why we didn't see this new problem before we plug new nodes?
Answer was simple, Drupal nodes couldn't stress the DB as it got self killed (Frozen) during high traffics.

Before jump in the Apache, Our WSO2 Infrastructure team revisit our monitoring systems (Ganglia) and found that servers were running on swap most of the time

As a solution I stated to tune up the Apache to avoid memory swapping problem. Then I saw that there was some miss configurations which take more memories when Apache process start to handle the traffic. Due to that configuration server can easily excused. 

MaxClients as was one of a major key parameter I had to change during the Apache memory optimization process

Apache Memory Optimization  

1. Find the non-swapped physical memory Apache has used (RES)
1. Run top command and press shift + m to find highest  RES value which is Apache process use
2. Find the AVAILABLE APACHE MEMORY POOL.
1. Run service apache2 stop and type free -m
2. Note the used memory and Subtract it from total (This will give you the FREE MEMORY POOL)
3. Multiply MEMORY POOL by 0.8 to find the average AVAILABLE APACHE POOL (This will allow server  20% memory reserve for burst periods)
3. Calculate MaxClients
1. Divide AVAILABLE APACHE POOL by the highest RES memory used by Apache (Step 1). Now we have MaxClients number
2. Open apache2.conf and change MaxClients value
4. Other tweaks
1. Set Keepalive On (keep it off if you haven't keep 20% memory reserve. )
2. set keepalivetimeout to the lowest value (This will prevent connection hanging, If you experience high latency to your server, set it to 2-5 seconds)
3. Set your Timeout to a reasonable value like 10 - 40 (Keep it low)
4. set MaxKeepAliveRequests with in 70-150 (If you have a good idea about objects in a 1 page set it to match the object count)
5. Set MinSpareServers equal to 10-25% of MaxClients
6. Set MaxSpareServers equal to 25-50% of MaxClients
7. Set StartServers equal to MaxSpareServers
8. Set MaxRequestsPerChild somewhere between 400 - 700 (if you see rapid Apache child process memory use growth) to 10000
5. Apply Changes to Apache
1. Run service apache2 start

Once I restart our servers with above setting, server could handle 150% traffic from Apache end and server load was always below 2 (in a burst server-load was 1-2)

This solved my Apache hanging problem. Then I ran a another load test to verify my setting. I was amazed with the improvement but that haven't solved my problem. 

server is taking bit long to response but I didn't see connection drop.

2. Now, What is wrong with my settings, Why server response is slow ? MySQL!! 

I will discuss in part 2, how I addressed this problem .

Pressflow 6 with mysql Replication

Pressflow6 should support database replication according to the pressflow wiki. but when I configured master and slave, pressflow instance always pick master. If master was down, site couldn't load and site redirected to the site-offline error page.

I did some modifications in database.inc, database.mysqli.inc and settings.php in order to work when master mysql server was down.

Scenario

1. If mysql master is inactive, connect to Slave,
2. if slave is inactive but Master is active, then connect to Master
3. if Master and Slave both are inactive, Redirect user to Offline page

Modifications can be found in red text.

database.inc

1. Below change will modify the db_connect call to Master instance by passing 2nd parameter ("master") and by assigning $db_slave_conns[$name] values to $active_db.
2. Validate before slave connection create. If master is active we are not calling Slave by adding $db_conns[$name]->connect_errno > 0 

function db_set_active($name = 'default') {
  global $db_url, $db_slave_url, $db_type, $active_db, $active_slave_db;
  static $db_conns, $db_slave_conns, $active_name = FALSE;


  if (empty($db_url)) {
    include_once 'includes/install.inc';
    install_goto('install.php');
  }


  if (!isset($db_conns[$name])) {
    // Initiate a new connection, using the named DB URL specified.
    if (is_array($db_url)) {
      $connect_url = array_key_exists($name, $db_url) ? $db_url[$name] : $db_url['default'];
      if (is_array($db_slave_url[$name])) {
        $slave_index = mt_rand(0, count($db_slave_url[$name]) - 1);
        $slave_connect_url = $db_slave_url[$name][$slave_index];
      }
      else {
        $slave_connect_url = $db_slave_url[$name];        
      }
    }
    else {
      $connect_url = $db_url;
      if (is_array($db_slave_url)) {
        $slave_index = mt_rand(0, count($db_slave_url) - 1);
        $slave_connect_url = $db_slave_url[$slave_index];
      }
      else {
        $slave_connect_url = $db_slave_url;        
      }
    }


    $db_type = substr($connect_url, 0, strpos($connect_url, '://'));
    $handler = "./includes/database.$db_type.inc";


    if (is_file($handler)) {
      include_once $handler;
    }
    else {
      _db_error_page("The database type '". $db_type ."' is unsupported. Please use either 'mysql' or 'mysqli' for MySQL, or 'pgsql' for PostgreSQL databases.");
    }


    $db_conns[$name] = db_connect($connect_url,'master');
    if ($db_conns[$name]->connect_errno > 0 && !empty($slave_connect_url)) {
      $db_slave_conns[$name] = db_connect($slave_connect_url); 
    }
  }


  $previous_name = $active_name;
  // Set the active connection.
  $active_name = $name;
  $active_db = $db_conns[$name];
  if (isset($db_slave_conns[$name])) {
    $active_slave_db = $db_slave_conns[$name];
    $active_db = $db_slave_conns[$name];    
  }
  else {
    unset($active_slave_db);
  }


  return $previous_name;
}