cancel
Showing results for 
Search instead for 
Did you mean: 

Cron jobs pile up and never stop running, thus crashing the server

SOLVED

Cron jobs pile up and never stop running, thus crashing the server

Hi everyone!

 

For some unknown reason Magento 2 never stops running cron jobs. Before one cron job finishes, the next one starts causing a lot of cron jobs to pile up and crash the server. 

 

I've got the following Magento configuration: 2.2.7 CE.

 

Has anyone had similar behavior and how can I solve this?

1 ACCEPTED SOLUTION

Accepted Solutions

Re: Cron jobs pile up and never stop running, thus crashing the server

Hi everyone!

 

I've been able to resolve my issue. The reason why the server wouldn't complete cron jobs in time was Ajax Layered Navigation module which put very high load on the server. This module came with template I purchased via ThemeForest. Turning the module off resolved all issues and made the server happy at a cost of a missing fancy Ajax Layered Navigation feature. Will have to look for an alternative solution to the disabled module. Everything's been working fine for several weeks now.

View solution in original post

26 REPLIES 26

Re: Cron jobs pile up and never stop running, thus crashing the server

Hi @DK_dev 

 

Can you share the cron configuration which you made on server? It can help to identify the issue.

---
Problem Solved Click Accept as Solution!:Magento Community India Forum

Re: Cron jobs pile up and never stop running, thus crashing the server

@DK_devNormally it does not piles up. After certain amount of minutes you may see the status as missed and an error message. There can be bunch of following questions which you may have already looked into..

 

  • Do you know which job is causing the error? Many time indexing takes time due to big catalog and cause that error.
  • While you are running the cron are you running out of memory or CPU etc?
  • Is that appeared suddenly or after an upgrade/patch/code update etc...

You can slow down the cron from the Magento admin. In case running and generating jobs every minute generate for every five minute. This will slow down the processing a bit. Down side is you have to wait 5 minutes for your updates.

 

Problem solved? Please give 'Kudos' and accept 'Answer as Solution'.

- Tarandeep
Problem solved?Please give 'Kudos' and accept 'Answer as Solution'.

Re: Cron jobs pile up and never stop running, thus crashing the server

I've got the following config. As you can see, everything is set to default values (other groups have the default settings as well).

 

Screen Shot 2019-07-02 at 8.29.50 AM.pngScreen Shot 2019-07-02 at 8.30.04 AM.png

 

Since restarting MySQL server yesterday I haven't had any issues (not sure yet if DB server config may be an issue), will check once more today if the issue still exists. Though there are missed cron jobs in cron_schedule database table, which is definitely not a good sign.

 

For some reason I have the following processes which have been running for 20 minutes now:

 

Screen Shot 2019-07-02 at 8.34.46 AM.png

Re: Cron jobs pile up and never stop running, thus crashing the server

And that's the pile of php processes currently running, which makes server basically unresponsive. New cron jobs start while unfinished previous ones haven't finished yet.

Screen Shot 2019-07-02 at 11.10.19 AM.png

I don't have a huge catalog - there are about 4500 products. While these cron jobs are running I have server load average growing infinitely high and server is slowly running out of memory. I can't recall making any code changes other that started using Magento 2 connector for Odoo ERP. Will try to disable these extensions to see if this helps as well as reschedule cron jobs for every five minutes.

Re: Cron jobs pile up and never stop running, thus crashing the server

@DK_dev Your cron jobs are running every minute.

 

Try to run the job every 5 minutes instead  */5 * * * *

 

*/5 * * * * /usr/bin/php -c /etc/php5/apache2/php.ini /var/www/magento2/bin/magento cron:run > /var/www/magento2/var/log/magento.cron.log&

*/5 * * * * /usr/bin/php -c /etc/php5/apache2/php.ini /var/www/magento2/update/cron.php > /var/www/magento2/var/log/update.cron.log&

*/5 * * * * /usr/bin/php -c /etc/php5/apache2/php.ini /var/www/magento2/bin/magento setup:cron:run > /var/www/magento2/var/log/setup.cron.log&

 

 

---
Problem Solved Click Accept as Solution!:Magento Community India Forum

Re: Cron jobs pile up and never stop running, thus crashing the server

Thank you for the reply!

I will try applying your solution and let you know how it works!

Re: Cron jobs pile up and never stop running, thus crashing the server

After updating cron schedule I still get some unknown processes like the following:

Screen Shot 2019-07-03 at 12.29.54 PM.png

Is this normal behavior since I don't user New Relic Reporting?

Re: Cron jobs pile up and never stop running, thus crashing the server

These processes work for up to 5 minutes, often in parallel with cron jobs, causing server load average to rise to 15-20 on a 5-core virtual server.

Re: Cron jobs pile up and never stop running, thus crashing the server

@DK_dev  Can you make sure that New Relic integration is disabled and cron for the same is disabled too. If you have multiple site or store then make sure to switch the store and confirm.

 

Settings for the same can be found in Admin -> Store -> General -> New Relic Reporting.

 

 

Problem solved? Please give 'Kudos' and accept 'Answer as Solution'.

- Tarandeep
Problem solved?Please give 'Kudos' and accept 'Answer as Solution'.