'Whoops our bad' error on main store page every day

[AlexMagSup]

Every day, usually within 1 to 2 hours after midnight, my Magento 2.3 store starts showing 'Whoops our bad' error on the main page. The admin pages work fine though and flushing Magento cache fixes the thing for another 24 hours or so. Usually it happens once per day, shortly after midnight, but sometimes it happens also during the day, once or twice.

The system log shows no errors and there are no changes done to Magento store in between the incidents.

How do I go about identifying what causes it?

[Kavitha Mano]

Please check if any cron is running around the time ? else put some log files to  receive any datas while error is showing?

[Darren Grant]

It will be because cron is not running, check your cron tab, see the following guide for more information. https://devdocs.magento.com/guides/v2.3/config-guide/cli/config-cli-subcommands-cron.html

[AlexMagSup]

Cron is running every minute, usually with no errors or warnings, but today I noticed some warnings. I do not know at what exact time the store crashes so I cannot try to correlate with the log file and I am not finding anything in the log files that would give me a clue.

Here is the current log with some warnings regarding lock acquisition, although the store is still operational even with warnings.

[2019-01-02 22:46:58] main.INFO: Cron Job catalog_product_outdated_price_values_cleanup is run [] []
[2019-01-02 22:46:58] main.INFO: Cron Job catalog_product_outdated_price_values_cleanup is successfully finished. Statistics: {"sum":0.00059890747070312,"count":1,"realmem":0,"emalloc":648,"realmem_start":161742848,"emalloc_start":145815704} [] []
[2019-01-02 22:46:59] main.INFO: Cron Job catalog_product_frontend_actions_flush is run [] []
[2019-01-02 22:46:59] main.INFO: Cron Job catalog_product_frontend_actions_flush is successfully finished. Statistics: {"sum":0.0005650520324707,"count":1,"realmem":0,"emalloc":752,"realmem_start":161742848,"emalloc_start":145822096} [] []
[2019-01-02 22:46:59] main.INFO: Cron Job catalog_product_attribute_value_synchronize is run [] []
[2019-01-02 22:46:59] main.INFO: Cron Job catalog_product_attribute_value_synchronize is successfully finished. Statistics: {"sum":0.0013959407806396,"count":1,"realmem":0,"emalloc":21560,"realmem_start":161742848,"emalloc_start":145868496} [] []
[2019-01-02 22:46:59] main.INFO: Cron Job outdated_authentication_failures_cleanup is run [] []
[2019-01-02 22:46:59] main.INFO: Cron Job outdated_authentication_failures_cleanup is successfully finished. Statistics: {"sum":0.00087785720825195,"count":1,"realmem":0,"emalloc":1128,"realmem_start":161742848,"emalloc_start":145919280} [] []
[2019-01-02 22:47:00] main.INFO: Cron Job sales_grid_order_async_insert is run [] []
[2019-01-02 22:47:00] main.INFO: Cron Job sales_grid_order_async_insert is successfully finished. Statistics: {"sum":0.00055098533630371,"count":1,"realmem":0,"emalloc":752,"realmem_start":161742848,"emalloc_start":146019352} [] []
[2019-01-02 22:47:01] main.INFO: Cron Job sales_grid_order_invoice_async_insert is run [] []
[2019-01-02 22:47:01] main.INFO: Cron Job sales_grid_order_invoice_async_insert is successfully finished. Statistics: {"sum":0.00056982040405273,"count":1,"realmem":0,"emalloc":752,"realmem_start":161742848,"emalloc_start":146028584} [] []
[2019-01-02 22:47:01] main.INFO: Cron Job sales_grid_order_shipment_async_insert is run [] []
[2019-01-02 22:47:01] main.INFO: Cron Job sales_grid_order_shipment_async_insert is successfully finished. Statistics: {"sum":0.0007929801940918,"count":1,"realmem":0,"emalloc":752,"realmem_start":161742848,"emalloc_start":146037032} [] []
[2019-01-02 22:47:02] main.INFO: Cron Job sales_grid_order_creditmemo_async_insert is run [] []
[2019-01-02 22:47:02] main.INFO: Cron Job sales_grid_order_creditmemo_async_insert is successfully finished. Statistics: {"sum":0.00093722343444824,"count":1,"realmem":0,"emalloc":752,"realmem_start":161742848,"emalloc_start":146045576} [] []
[2019-01-02 22:47:04] main.INFO: Cron Job newsletter_send_all is run [] []
[2019-01-02 22:47:04] main.INFO: Cron Job newsletter_send_all is successfully finished. Statistics: {"sum":0.013673067092896,"count":1,"realmem":0,"emalloc":357376,"realmem_start":161742848,"emalloc_start":146053560} [] []
[2019-01-02 22:47:04] main.INFO: Cron Job magento_newrelicreporting_cron is run [] []
[2019-01-02 22:47:04] main.INFO: Cron Job magento_newrelicreporting_cron is successfully finished. Statistics: {"sum":0.0011169910430908,"count":1,"realmem":0,"emalloc":656,"realmem_start":161742848,"emalloc_start":147672368} [] []
[2019-01-02 22:47:04] main.INFO: Cron Job captcha_delete_old_attempts is run [] []
[2019-01-02 22:47:04] main.INFO: Cron Job captcha_delete_old_attempts is successfully finished. Statistics: {"sum":0.0014729499816895,"count":1,"realmem":0,"emalloc":18344,"realmem_start":161742848,"emalloc_start":147678272} [] []
[2019-01-02 22:47:05] main.INFO: Cron Job captcha_delete_expired_images is run [] []
[2019-01-02 22:47:05] main.INFO: Cron Job captcha_delete_expired_images is successfully finished. Statistics: {"sum":0.007537841796875,"count":1,"realmem":0,"emalloc":585056,"realmem_start":161742848,"emalloc_start":147794368} [] []
[2019-01-02 22:47:05] main.INFO: Cron Job get_amazon_capture_updates is run [] []
[2019-01-02 22:47:05] main.INFO: Cron Job get_amazon_capture_updates is successfully finished. Statistics: {"sum":0.0030717849731445,"count":1,"realmem":0,"emalloc":46544,"realmem_start":163840000,"emalloc_start":149051336} [] []
[2019-01-02 22:47:05] main.INFO: Cron Job get_amazon_authorization_updates is run [] []
[2019-01-02 22:47:05] main.INFO: Cron Job get_amazon_authorization_updates is successfully finished. Statistics: {"sum":0.0029430389404297,"count":1,"realmem":0,"emalloc":53472,"realmem_start":163840000,"emalloc_start":149243680} [] []
[2019-01-02 22:47:05] main.INFO: 18 cron jobs were cleaned [] []
[2019-01-02 22:47:05] main.INFO: 0 cron jobs were cleaned [] []
[2019-01-02 22:47:05] main.INFO: Cron Job consumers_runner is run [] []
[2019-01-02 22:47:05] main.INFO: Consumer "async.operations.all" skipped as required connection "amqp" is not configured. Unknown connection name amqp [] []
[2019-01-02 22:47:05] main.INFO: Cron Job consumers_runner is successfully finished. Statistics: {"sum":0.0012240409851074,"count":1,"realmem":0,"emalloc":1544,"realmem_start":136577024,"emalloc_start":123272016} [] []
[2019-01-02 22:47:05] main.INFO: 11 cron jobs were cleaned [] []
[2019-01-02 22:47:05] main.INFO: Cron Job amazon_payments_process_queued_refunds is run [] []
[2019-01-02 22:47:05] main.INFO: Cron Job amazon_payments_process_queued_refunds is successfully finished. Statistics: {"sum":0.0031940937042236,"count":1,"realmem":0,"emalloc":105920,"realmem_start":163840000,"emalloc_start":149309224} [] []
[2019-01-02 22:47:05] main.INFO: Cron Job indexer_reindex_all_invalid is run [] []
[2019-01-02 22:47:05] main.INFO: Cron Job indexer_reindex_all_invalid is successfully finished. Statistics: {"sum":0.0082919597625732,"count":1,"realmem":0,"emalloc":577656,"realmem_start":136577024,"emalloc_start":122947760} [] []
[2019-01-02 22:47:05] main.INFO: Cron Job indexer_update_all_views is run [] []
[2019-01-02 22:47:05] main.INFO: Cron Job indexer_update_all_views is successfully finished. Statistics: {"sum":0.018220901489258,"count":1,"realmem":0,"emalloc":106912,"realmem_start":136577024,"emalloc_start":123527456} [] []
[2019-01-02 22:47:06] main.INFO: Cron Job bulk_cleanup is run [] []
[2019-01-02 22:47:06] main.INFO: Cron Job bulk_cleanup is successfully finished. Statistics: {"sum":0.00089383125305176,"count":1,"realmem":0,"emalloc":2040,"realmem_start":163840000,"emalloc_start":149426408} [] []
[2019-01-02 22:48:04] main.INFO: Cron Job consumers_runner is run [] []
[2019-01-02 22:48:04] main.INFO: Consumer "async.operations.all" skipped as required connection "amqp" is not configured. Unknown connection name amqp [] []
[2019-01-02 22:48:04] main.INFO: Cron Job consumers_runner is successfully finished. Statistics: {"sum":0.0038278102874756,"count":1,"realmem":0,"emalloc":1544,"realmem_start":136577024,"emalloc_start":123270904} [] []
[2019-01-02 22:48:04] main.INFO: Cron Job indexer_reindex_all_invalid is run [] []
[2019-01-02 22:48:04] main.INFO: Cron Job indexer_reindex_all_invalid is successfully finished. Statistics: {"sum":0.0092508792877197,"count":1,"realmem":0,"emalloc":578224,"realmem_start":136577024,"emalloc_start":122944128} [] []
[2019-01-02 22:48:04] main.INFO: Cron Job indexer_update_all_views is run [] []
[2019-01-02 22:48:04] main.INFO: Cron Job indexer_update_all_views is successfully finished. Statistics: {"sum":0.018024921417236,"count":1,"realmem":0,"emalloc":106592,"realmem_start":136577024,"emalloc_start":123524392} [] []
[2019-01-02 22:48:07] main.WARNING: Could not acquire lock for cron group: default, skipping run [] []
[2019-01-02 22:49:07] main.INFO: Cron Job consumers_runner is run [] []
[2019-01-02 22:49:07] main.INFO: Consumer "async.operations.all" skipped as required connection "amqp" is not configured. Unknown connection name amqp [] []
[2019-01-02 22:49:07] main.INFO: Cron Job consumers_runner is successfully finished. Statistics: {"sum":0.0013251304626465,"count":1,"realmem":0,"emalloc":1544,"realmem_start":136577024,"emalloc_start":123268512} [] []
[2019-01-02 22:49:07] main.INFO: Cron Job indexer_reindex_all_invalid is run [] []
[2019-01-02 22:49:07] main.INFO: Cron Job indexer_reindex_all_invalid is successfully finished. Statistics: {"sum":0.0085899829864502,"count":1,"realmem":0,"emalloc":578224,"realmem_start":136577024,"emalloc_start":122947192} [] []
[2019-01-02 22:49:07] main.INFO: Cron Job indexer_update_all_views is run [] []
[2019-01-02 22:49:07] main.INFO: Cron Job indexer_update_all_views is successfully finished. Statistics: {"sum":0.020081996917725,"count":1,"realmem":0,"emalloc":106912,"realmem_start":136577024,"emalloc_start":123527456} [] []
[2019-01-02 22:49:10] main.WARNING: Could not acquire lock for cron group: default, skipping run [] []

[Kavitha Mano]

Try re-indexing data via the CLI, use the following command

bin/magento indexer:reindex

If you want to be sure that NO cache is kept, you can remove the following folders from the vardirectory with:

rm -rf var/cache var/generation var/pagecache var/di

and then re-compile magento using command

bin/magento setup:di:compile 

Thanks!!!

[AlexMagSup]

Reindexing via CLi did not help, still getting the same warning

main.WARNING: Could not acquire lock for cron group: default, skipping run [] []

[AlexMagSup]

I figured out the issue with cron. The cron_schedule table had over 50,000 rows with most of them in pending state. The issue has been reported since Magento 2.2, probably caused by often Magento compilations in the past several days.

However, original "whoops" issue is still there..

[davej]

I too am having this problem, did you find a fix?

It fails just after midnight. Presumabley this is due to one of the cron jobs that runs each day.

Basically a 404 error.

I can resolve the issue by running sudo bin/magento-cli cache:flush and the next day the issue is back.

This is running under a BItnami Magento stack hence running commands as Sudo.

[error0024]

I am currently having the same issue and it's been happening for the past few weeks. However googling around doesn't give us any solution. Did you get the problem fixed?

[JornEyedevelop]

For anyone coming here with the same issue:

This is wat happens

1. The homepage does not accept HEAD requests

2. If a HEAD request to the homepage is made, it returns an error and the 404 page.

3. Sometimes, this 404 page gets cached as the main home page

4. Store home page returns 404 page in browser

To fix:

1. create a basic module with an override (preference) for 

\Magento\Cms\Controller\Index\Index

2. In your overridden class add the following after the class declaration

implements Magento\Framework\App\Action\HttpHeadActionInterface

 So you have something along the lines of: 

class Index extends \Magento\Cms\Controller\Index\Index implements HttpHeadActionInterface

3. run setup:upgrade to install/enable new module and clear generated and cache directories

HOWEVER, this is a temporary fix. Once the underlying bug has been fixed and you've updated to the version implementing that fix, you can remove this quick fix.

Sources

Issue: https://github.com/magento/magento2/issues/21299

Fix: https://github.com/magento/magento2/pull/21331