We have tested the above module, and at a basic level it works well. There are several enhancements that we have made to the VCL to obtain better performance, compliance and compatibility. But this post aims to help further develop the module by fixing a critical bug.

Using the above extension, it is not possible to use the catalogue listing “add-to-cart” URL encoded link (versus the standard POST form on the product page) when no cookies have been created. It can however be changed to allow for exceptions such as these. These changes are not yet production tested, but give a scalable workaround to certain modules not functioning as they should. It essentially circumvents the cookie detection built into Magento, so it does have its inherent risks (ie. doesn’t alert people to the fact they don’t have cookies enabled!).

The issue only occurs if no cookies at all have been created.

Ie. If you were to go to the my account log in page, then remove all cookies, then attempt to log in, you’ll get the same error.

Ie. If you landed on the homepage (without visiting the site before, ie. having cleared all your cookies), and clicked add to basket, you’ll get the same error.

Further modules, controllers and specific actions can be added to exceptions simply by adding them to the tag in config.xml

In etc/config.xml, add

    
        
            
              add
              cart
              redirect
            
            
              loginPost
              account
              forward
            
        
    

    
        
            ...
            
                
                    
                        singleton
                        varnish/observer
                        preDispatch
                    
                
            

In Model/Observer.php, add

   public function preDispatch(Varien_Event_Observer $observer)
    {
      $helper = Mage::helper('varnish/cacheable'); /* @var $helper Magneto_Varnish_Model_Cacheable */

      if ($helper->getCookie()->get())
        return $this;

      if ($helper->checkExceptions()) {
        $helper->turnOffVarnishCache();
      }

      return $this;
    }

In Helper/Cacheable.php, add

   private $_exceptions = false;

   public function forward($request, $module, $action, $controller, $params = array())
    {

        $request->setParam('nocookie', true);
        $request->initForward();

        if (!is_null($params)) {
            $request->setParams($params);
        }

        if (!is_null($controller)) {
            $request->setControllerName($controller);

            // Module should only be reset if controller has been specified
            if (!is_null($module)) {
                $request->setModuleName($module);
            }
        }

        $request->setActionName($action)
            ->setDispatched(false);

    }

    public function getExceptions()
    {
      if (!$this->_exceptions)
        return $this->_exceptions = (array)Mage::app()->getConfig()->getNode('varnish')->exceptions;

      return $this->_exceptions;
    }

    public function checkExceptions()
    {
        $forwardInfo = array();
        $endParams = '';

        $_request = Mage::app()->getFrontController()->getRequest();

        if ($_request->getActionName() != "noCookies")
          return false;

        /*
         * Magento 1.3 doesn't support getBeforeForwardInfo() or advanced forwarding methods
         * that 1.4 does, so its functionality is going to be somewhat crippled.
         * Solution: Upgrade to 1.4!
         */

        if (version_compare(Mage::getVersion(), '1.4', '<')) {
          #Mage::getSingleton('customer/session')->setNoLoop(1);
          #header('Location: '.$_request->getPathInfo().'?'.htmlspecialchars(SID));
          #exit();

          $requestUri = $_request->getRequestUri();
          $requestUriBits = explode("/", $requestUri);

          $forwardInfo['module_name'] = $requestUriBits[1];
          $forwardInfo['controller_name'] = $requestUriBits[2];
          $forwardInfo['action_name'] = $requestUriBits[3];
          $forwardInfo['params'] = $_request->getParams();

        } else {
          $forwardInfo = $_request->getBeforeForwardInfo();
        }

        $_exceptions = $this->getExceptions();

        if (!array_key_exists('module_name',$forwardInfo))
          return false;

        /*
         * If the session var no_loop is set, it means its tried this once before
         * which means the customer must not have cookies enabled - we need to let
         * Magento display its error if that is the case.
         */ 

        if (Mage::getSingleton('customer/session')->getNoLoop()) {
          return false;
        }

        if (array_key_exists($forwardInfo['module_name'], $_exceptions)) {
            if ((string)$_exceptions[$forwardInfo['module_name']]->action_name == $forwardInfo['action_name'] &&
                (string)$_exceptions[$forwardInfo['module_name']]->controller_name == $forwardInfo['controller_name']) {

              Mage::getSingleton('customer/session')->setNoLoop(1);

              /*
               * We're using a crude header redirect to re-try the session now a cookie
               * has been created. If it fails, the SID is passed - to prevent looping
               * This method won't work with account login, as it doesn't support GET
               * for user credentials.
               */

              if ($_exceptions[$forwardInfo['module_name']]->method == 'redirect') {
                $path = $forwardInfo['module_name'] .'/'. $forwardInfo['controller_name'] .'/'. $forwardInfo['action_name'];

                if (array_key_exists('options', $forwardInfo['params'])) {
                  foreach ($forwardInfo['params']['options'] as $optionId => $value) {
                    $endParams .= '&options['.$optionId.']='.$value;
                  }
                  unset($forwardInfo['params']['options']);
                }                

                $fields = $this->array_flatten($forwardInfo['params']);
                header('Location: '.Mage::getUrl($path, $fields).'?'.htmlspecialchars(SID).$endParams);
                exit();
                #Mage::app()->getResponse()->setRedirect(Mage::getUrl($path, $fields));
              } else {
                $this->forward($_request, $forwardInfo['module_name'], $forwardInfo['action_name'], $forwardInfo['controller_name'], $forwardInfo['params']);
              }
              return true;
            }
        }

        return false;
    }

    public function array_flatten( $array ) {
      $arr = array();
      function flatten( $item, $key, $flattened ) {
        $flattened[$key] = $item;
      };
      array_walk_recursive($array, 'flatten', &$arr);
      return $arr;
    }

Accessing your servers IPMI interface requires you to set up a simple VPN connection to our secure IPMI management network. You can use the guide (right) to help you create the connection required.

VPN Type: PPTP
VPN Host: vpn2.sonassihosting.com

Don’t have an account?

If you do not have an IPMI VPN account yet, please get in touch with the support team at theclientarea.info to request your IPMI VPN access details.

Recently we were challenged by one of our dedicated hosting customers, DirectSight.co.uk, to implement a temporary platform to support up to 1 million unique visitors. Whilst perhaps not a frequent request, with our core-competence lying in dedicated Magento hosting, it was something we felt confident in being able to deliver. As a company, we have offered specialised, dedicated Magento server solutions for just over 2 years now – and this wealth of knowledge would prove worthwhile for the upcoming mammoth Groupon traffic.

1 Million unique visitors, 1 Magento cloud please

So with the challenge set, at quite short notice, we were able to draw up a simple redundant, clustered solution; very similar to that used in our Magento cluster. The cluster consisted of 10 active servers with 3 hot-standby and 2 cold-standby systems; we dubbed it the Magento MicroCloud.

1 day in … some serious traffic

Being a national promotion, the DirectSight Groupon promotion was a large scale, highly featured campaign – even featured on Martins Money Tips. We expected a huge volume of traffic and boy did we, a sustained 60Mbits p/s. To put this into perspective, a typical store with around 3,000 visitors per day usually consumes ~0.1Mbits p/s.

The promotion is still running (at the time of this being written), but performance figures so far really do highlight how good we really are at hosting Magento stores, and more to the point – hosting a high priority, high demand Magento based promotion.

The target

Requirements for the configuration were described as:

1. Support for 250k-500k unique visits per day
2. Average page views of 8 pages
3. Average time on site 6 minutes

Based on this we estimate:

1. 1600GB bandwidth usage per day (500k visitors * 8 avg. pages viewed * 0.4MB avg. page size)
2. 60MBit connection commitment per day
3. 100k peak visitors in a 1 hour period
a. Support for 10,000 concurrent sessions
b. Support for 300 rps

and the outcome …

To give you a taster – the right image shows the output from the loadbalancer’s external network interfaces. Even under this massive influx of traffic, the Magento MicroCloud performed brilliantly and faultlessly.

If anything, the load averages (left) highlight how little stress the campaign placed on the available processing capacity of the MicroCloud.

We estimate that the traffic surge used between 15-20% of the available resource we set up. Our initial specification for the project was to meet twice the required load and provide plenty of overhead.

By the end of the promotion, it had peaked at 55,000 unique visitors on Wednesday 13th July, with a total of 10 page views per customer; resulting in just under 600,000 page views. Given the low load on the platform, it means that with real-world traffic, it should have been suitable for up to around 3,833,000 page views – which is just short of what we believed the solution could deliver, 4,000,000 pageviews.

We have been delighted overall and now are looking to offer this type of high demand hosting for such short-term promotions as a real offering.

Interested?

If you are interested in hosting a Groupon promotion or even just looking for specialised dedicated hosting for your Magento store – then don’t hesitate to get in touch!

By replicating some of the functionality applied in the cluster, we have been able to increase performance by a significant factor, for both static and dynamic content display.

So what does this mean for you?

Better performance without having to change a thing. The upgrade has been performed seamlessly in the background and should provide full compatibility with the previous configuration.

sonassiSDS1.1 the Sonassi Static Delivery Server is a new platform to deliver static content from your site (eg. images, js, css)
sonassiDDS1.0 the Sonassi Dynamic Delivery Server is a new platform to deliver dynamic content from your site (eg. php)

Now the transition is complete, we will be keeping a keen eye on both performance, uptime and service.

We try to make migration painless and ensure downtime is at an absolute minimum, this isn’t an overnight process however and we always reccomend starting this process around 1 week prior to the “big move”.

We are going to move example.com

dig example.com ns

; <<>> DiG 9.3.6-P1-RedHat-9.3.6-4.P1.el5_4.2 <<>> example.com ns
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 7403
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;example.com.                   IN      NS

;; ANSWER SECTION:
example.com.            171994  IN      NS      ns1.sonassihosting.com.
example.com.            171994  IN      NS      ns2.sonassihosting.com.
example.com.            171994  IN      NS      ns3.sonassihosting.com.

;; Query time: 7 msec
;; SERVER: 208.67.220.220#53(208.67.220.220)
;; WHEN: Tue Oct 26 10:42:16 2010
;; MSG SIZE  rcvd: 77

# cd /domains/example.com
# mysqldump -hmysqlserver -uusername -p databasename > http/db_backup.sql
# tar cvfz http.tgz http

# scp http.tgz exampleuser@111.111.111.111:/domains/example.com

# cd /domains/example.com
# tar xvfz http.tgz
# rm http.tgz
# cd http
# mysql -hmysqlserver -uusername -p databasename < db_backup.sql
# rm db_backup.sql
# mv app/etc/local.xml{,.bak}

# which rsync
# crontab -e

RewriteCond %{REQUEST_URI} !/holding.html
RewriteRule .* /holding.html [L,R=302]

# cd /domains/example.com
# mysqldump -hmysqlserver -uusername -p databasename > http/db_backup.sql
# rsync -par /domains/example.com/http/* exampleuser@111.111.111.111:/domains/example.com/http
# rsync -par /domains/example.com/http/.* exampleuser@111.111.111.111:/domains/example.com/http

# cd /domains/example.com/http
# mysql -hmysqlserver -uusername -p databasename < db_backup.sql

Your site migration should have gone without a hitch and you will have lost as few sales as possible during the migration. We would obviously only recommend doing this during your quietest trading period in the week, so that total revenue isn’t affected by any noticeable measure.