Magento Performance is a hot topic at the moment. We recently installed Magento Commerce version 1.3.0 on a test server with a current production database to do some performance testing and I thought I would publish a few results. They aren’t exhaustive tests, but indicative I think.
Although we haven’t done a huge amount of testing yet there are some interesting results (and some performance improvements over a ‘standard’ non cached install).
First, any figures should be interpreted with respect to our own current configuration for this test. These results were NOT for a standard demo data set.
Our server config is as follows :
Simple Amazon EC2 Instance , Small Instance (Default) 1.7 GB of memory, 1 EC2 Compute Unit (1 virtual core with 1 EC2 Compute Unit), 160 GB of instance storage, 32-bit platform
Server tweaks :
- MYSQL caching enabled
- PHP caching enabled
- We have also made other changes to MYSQL tuning parameters for max temp table size, query cache etc etc.
Our location :
Melbourne, Australia (server located in US)
Magento Setup :
- 6 websites, each with 1 store
- 10,000 products (3 stores share the same inventory, other stores have 200 items or less)
- Google analytics
- 2 ‘trust’ buttons (US locations)
- Godaddy security seal (US location)
Magento Performance Test Results
We decided to do a little testing on our new 1.3.0 install (same database, products etc as a production server – essentially a mirror image of our 1.1.8 server but upgraded to 1.3.0).
Please keep in mind that the Magento performance stats below are in no part an exhaustive test, nor may represent what you would find on your server. You might find the interesting though.
All stats below were timed using the Firefox plugin “Yslow” in our 10,000 product store. Tests were run with Firefox 3 on a Windows XP machine, 2gb ram, reasonably fast.
Magento Category Page Loads
This chart shows consecutive page loads of a category page in our Magento 1.3.0 store. The page had 30 products, grid view. The one page was loaded once, then repeatedly to record a load time and an average was worked out. Average of the page loads was 9.06 seconds
This graph shows the same page (after all browser caching cleared etc) with Magento system caching turned on (Admin -> System Cache, Cache Control settings). As you can see the initial loads were about the same, but caching improved overall average load time. The lower results were more typical in later testing (perhaps there was a little more load on the box when I ran these). The average load time was 7.8 seconds, about a 15% improvement (but I think in reality it is a little more)
Same test again but this time with Magento’s Flat Catalog and Flat Product caching enabled as well. This did result in some improvements, but in our own case not as much as I had hoped for. We couldn’t get category page loads below about 5.8 seconds. We think we hit the ‘server grunt’ limit. Time to buy a bigger box. The overall average for this test was 6.8 seconds.
So for category loads, no caching versus ALL caching meant a drop from 9.06 second average to 6.8 seconds, or a 25% decrease.
We think the performance improvement would have been greater had we been running on faster machines (database and webserver).
Magento Product Page Loads – Interesting!
I then did a series of product page loads. The graph below shows 3 consecutive page loads of different product pages (each coloured line represents a new product page). The page load times are interesting because in each and every case the product page took longer to load each time!
Note this was done after clearing Magento and browser cache before testing.
Strangely the average load time went from 4.23 to 6.61 to 10.36 seconds!
This was done with the system cache enabled, but without flat catalog/product caching enabled.
So what happened with flat catalog/product caching enabled?
Well, the performance improved somewhat, but the consecutive loads of each product page still took longer each time.
Average for first page 4.04 seconds, reload1 was 5.99 seconds and reload2 was 6.12 seconds. Big improvements when taken out of context, but why would page reloads take longer each time? Bit of a head scratcher that one. I really can’t explain it, and I don’t think it would be typical, maybe someone could comment?
In summary I think in our situation the servers we had were maxed out performance wise. If we wanted faster responses, or better responses from extra caching we would have received it with faster servers.
As it was, there was still a noticable performance increase with Magento system caching and flat catalog enabled.
Even though tthe performance for category loads above was about 25% I think this is on the lower end of performance improvements that can be expected.
I know of a Magento store owner with many thousands of categories and a few thousand products so I might try and organise some quick performance tests of that site (on a faster local host) to see what the gains are with flat catalog.