Magento high CPU useage

[gfxpixeldesigns]

Hi everyone,

I'm running a magento install on a dedicated server with the following specs-

Xeon E3-1230 3.20 GHz

8GB RAM

I have Centos installed with cPanel/WHM and i use apachebooster (nginx+varnish) with fcgi as the php handler. The magento install utilises all caching options including a full page caching system (fishpig bolt). This runs okay and i get a fast responsive load times at the magento site so i'm happy with that.

My issue is more with the server performance, magento really seems to hog it down. For instance if i run a top command i can see that magento is using a lot of cpu%

| Top Process | 106 | /usr/bin/php |
| Top Process | 91.5 | /usr/bin/php |
| Top Process | 88.8 | /usr/bin/php |

This is fine at the moment as we dont really have many visitors so its not affecting us but at peak i can only imagine the problems this is going to cause.

At first i thought this was a bad extension causing high load but after stripping everything down so that i was left with plain old magento i was still seeing high loads so i can only assume this is normal for magento?

Perhaps there is something more i can do to reduce loads, would anyone have any recommendations?

Thanks

[Theis Corfixen]

Hi @gfxpixeldesigns

Its always really hard to say exactly where you performance bottlenecks would be but I'll give it a shot anyway.

Seeing that run both Varnish and a FPC extension on top of Magento, means that your customers shouldn't be the ones stealing your CPU.

Meaning that it can come from your administrators, or it can be Magento's Cron running taking up all the CPU. Actions like running the reports or using the indexers in the admin can really CPU consuming and it would not recommend it. If its Magento's cron you can check the table: cron_schedule, to see what events are running. 

Or you can install AOE's Cron scheuduler to maybe get a better overview of this: http://www.magentocommerce.com/magento-connect/aoe-scheduler.html

I hope this helps :-)

[ninteh]

Check your nginx (worker process configs) and SQL configuration (caches, also try memcached or redis combined), it may help offload some of that PHP strain. I'm hosting on a xeon 3.7ghz with 64gb ram, and after a lot of configurating nginx sql etc, we suffered severe lag on the site with 200 concurrent users. This was mainly caused because nginx opens a connection for every image loaded, so product pages could open up a tonne of connections in one go. In the end, we used cloudflare in front of the server and saw a 75% drop in load.. good luck :}

[gfxpixeldesigns]

Thanks for the replies i have been running through everything mentioned but unfortunately i'm still none the wiser.

The magento install only has 1 administrator and we've made certain nobody was in the backend while running the top command. I dont think its coming from the cron either, i've been using AOEscheduler and it doesnt show anything out of the norm just sales reports and currency rates every day at midnight.

Whats odd is i run another magento install on the same server with identical settings but the load is never high from that account. It does receive only 10% of the sessions in comparision to the 1000 a day on the main magento install though so maybe that explains it?

Its almost as if the caching isnt working for the load to be that high but the headers say its all cached. I'm wondering if i should spend some time importing the products to a fresh install.

We are also running memcache and redis combined as of this week so will see how that goes but so far there has been no change in load. I will look into cloudflare also but cant say i'm too familiar with that set up so will have to read up on it... 75% drop in load sounds fantastic though .

[senthilms]

If its cosuming lot of memory then it should have an impact in your load times as well. Check your PHP FPM's slow log config. If its enabled then check the slow log file to find which call is taking to much time to load.

Also check the tcp dump to identiy is there any webservice calls eating lot of your memory.

[MagenX]

i know you are on a default cPanel installation, like many of those who uses cPanel for magento. you need to start from configuring your backend. configure mysql, change your php handler to FPM, needs to recompile everything, then you run profiler to see if anything slow.. etc..

<< Snipped >>

[ninteh]

Cloudflare is a CDN which caches static versions of pages and assets on their own servers, which is what reduced  our server load. 

Typically I see the most CPU load on our server when indexing is being ran via Magento

What's your /var/cache folder looking like within the magento install folder? If the cache is working correctly you should be able to see lots of folders generated within there.

When you run another magento install, is it on the same theme folder? sometimes, high loads can be caused by additional query code within things such as your list.phtml files.

Could you run your website through a page load time tester, such as http://tools.pingdom.com/fpt/ - this will give you an idea of user cache, you want to try and leverage the browser cache where possible so single users aren't constantly hammering your server with the same image request etc.

[Stuart M]

Sometimes high CPU in Magento at PHP level can be caused by the time each process takes to complete thus causing more processes to co-exist at the same time and thus increasing CPU.

Common examples of where this is seen is things like MySQL queries taking longer then a few seconds to respond.

I personally would start with optimizing your database (it is the single easiest thing to optimize and rule out) and then consider enabling the Varien profiler to see on each pageload what PHP functions and queries are being called and how long each one takes.

[davidclarkny2]

Is it standard magento or you customized it? If you customized it, the reason can come from many factors we can't control. But you may try to install and configure some cache solutions to see the changes.

[shahabfk]

Hi,

You can test speed difference by installing fresh downloaded magento and your customized version of the magento.

If the speed of fresh downloaded magento version is good and your customized version is same then check any background process running and utilizing more CPU.

I am using ubuntu 12 with xampp 1.8 with 3 GB RAM ,2.2 GHz . I am getting good speed.