Magento Performance – Magento 1.3 Benchmarks
Magento Performance is a hot topic at the moment. We recently installed Magento Commerce version 1.3.0 on a test server with a current production database to do some performance testing and I thought I would publish a few results. They aren’t exhaustive tests, but indicative I think.
Although we haven’t done a huge amount of testing yet there are some interesting results (and some performance improvements over a ‘standard’ non cached install).
First, any figures should be interpreted with respect to our own current configuration for this test. These results were NOT for a standard demo data set.
Our server config is as follows :
Server :
Simple Amazon EC2 Instance , Small Instance (Default) 1.7 GB of memory, 1 EC2 Compute Unit (1 virtual core with 1 EC2 Compute Unit), 160 GB of instance storage, 32-bit platform
Server tweaks :
- MYSQL caching enabled
- PHP caching enabled
- We have also made other changes to MYSQL tuning parameters for max temp table size, query cache etc etc.
Our location :
Melbourne, Australia (server located in US)
Magento Setup :
- 6 websites, each with 1 store
- 10,000 products (3 stores share the same inventory, other stores have 200 items or less)
- Extra page javascript calls include
- Google analytics
- 2 ‘trust’ buttons (US locations)
- Godaddy security seal (US location)
Magento Performance Test Results
We decided to do a little testing on our new 1.3.0 install (same database, products etc as a production server – essentially a mirror image of our 1.1.8 server but upgraded to 1.3.0).
Please keep in mind that the Magento performance stats below are in no part an exhaustive test, nor may represent what you would find on your server. You might find the interesting though.
All stats below were timed using the Firefox plugin “Yslow” in our 10,000 product store. Tests were run with Firefox 3 on a Windows XP machine, 2gb ram, reasonably fast.
Magento Category Page Loads
This chart shows consecutive page loads of a category page in our Magento 1.3.0 store. The page had 30 products, grid view. The one page was loaded once, then repeatedly to record a load time and an average was worked out. Average of the page loads was 9.06 seconds
Category Loads (30 items), Magento cache ON
This graph shows the same page (after all browser caching cleared etc) with Magento system caching turned on (Admin -> System Cache, Cache Control settings). As you can see the initial loads were about the same, but caching improved overall average load time. The lower results were more typical in later testing (perhaps there was a little more load on the box when I ran these). The average load time was 7.8 seconds, about a 15% improvement (but I think in reality it is a little more)
Category Loads (30 items), Magento system cache + flat catalog
Same test again but this time with Magento’s Flat Catalog and Flat Product caching enabled as well. This did result in some improvements, but in our own case not as much as I had hoped for. We couldn’t get category page loads below about 5.8 seconds. We think we hit the ‘server grunt’ limit. Time to buy a bigger box. The overall average for this test was 6.8 seconds.
So for category loads, no caching versus ALL caching meant a drop from 9.06 second average to 6.8 seconds, or a 25% decrease.
We think the performance improvement would have been greater had we been running on faster machines (database and webserver).
Magento Product Page Loads – Interesting!
I then did a series of product page loads. The graph below shows 3 consecutive page loads of different product pages (each coloured line represents a new product page). The page load times are interesting because in each and every case the product page took longer to load each time!
Note this was done after clearing Magento and browser cache before testing.
Product Page Loads - System Caching ON
Strangely the average load time went from 4.23 to 6.61 to 10.36 seconds!
This was done with the system cache enabled, but without flat catalog/product caching enabled.
So what happened with flat catalog/product caching enabled?
Product Page, ALL caching enabled
Well, the performance improved somewhat, but the consecutive loads of each product page still took longer each time.
Average for first page 4.04 seconds, reload1 was 5.99 seconds and reload2 was 6.12 seconds. Big improvements when taken out of context, but why would page reloads take longer each time? Bit of a head scratcher that one. I really can’t explain it, and I don’t think it would be typical, maybe someone could comment?
In Summary
In summary I think in our situation the servers we had were maxed out performance wise. If we wanted faster responses, or better responses from extra caching we would have received it with faster servers.
As it was, there was still a noticable performance increase with Magento system caching and flat catalog enabled.
Even though tthe performance for category loads above was about 25% I think this is on the lower end of performance improvements that can be expected.
I know of a Magento store owner with many thousands of categories and a few thousand products so I might try and organise some quick performance tests of that site (on a faster local host) to see what the gains are with flat catalog.
Related Posts :
If you enjoyed this post, please consider to leave a comment or subscribe to the feed and get future articles delivered to your feed reader.
Comments
Richard,
We did compare load times later on to our 1.1.8 install, and 1.3 was a little faster, but not much. Of course, we would try to keep up to date with recent versions (bugs pending) anyway.
I might look into the elastic block store volumes. I have heard from a few other sources that EC2 IO is a little slow. Next thing to try is a few larger instances.
Ideally a product base of this size would be installed on a dedicated webserver machine (of load balanced servers) with dedicated database backend. Like I said in my post it was by no means a detailed or exhaustive test, but it did show some good gains with caching (and flat catalog cache) enabled.
Our 1.1.8 production server has a dedicated back end database server, so more improvements should be gained in upgrading 1.1.8 in that configuration.
The target of <2 seconds per page load is a good one.
We also have a load balancer setup now, so it is more or less easy to throw more server units from the EC2 service into the pool to boost performance. We may try that shortly.
Of note, we also experienced very slow product save times with a database of products that size. It didn’t seem to matter whether products were shared amongst website and stores or not. As we removed products from the database, the product save time (updating a price for example) decreased proportionately.
[...] Commerce 1.3.0 with over 10000 products in database, and see what flat catalog caching gives you. [...]Internet purchases soon to include SALES TAX | Ron Paul Wins … Myself, along with millions of [...]
In response to Richard, i work at http://ezyelectronics.com.au and our Magento store has 40,000 products. Magento def struggles with some elements, but with the right developers you can work around them.
Interesting, but would you really run your 10,000 product site on this setup?
Having everything on one machine makes it impossible to determine what might cause the pages to return more slowly. clearly your sytem bottlenecks, but at which stage is confused. I have found that amazon IO speed to be a bottleneck. running the app from an elastic block store volume can make a big difference. The size of processor will cause processor bottelnecks durring processing.
Have you considered performance testing a your current 1.1.8 system against the same system using 1.3 to see if it is worth upgrading. Or using the amazon system to build large more scalable systems to determine the real world resources you need to achieve acceptable speed (<2 seconds per page)
Cheers,
Richard