occasionally getting 404s for valid urls, refresh usually solves it, but why?

[DWJames]

thanks.

It's a quick and dirty bodge, but it seems to get the end results for the customers and work nicely

Good thinking

[devstar]

I just saw this while searching for topics on another problem I'm having... and I too can add my frustrations to yours. I've been having random 404 errors for valid URLs also and it sounds like this is somewhat of a common issue. I'll try the javascript workaround but would really like to get to the bottom of this. We're running on a fast virtual private server so speed isn't a problem.

Hopefully someone can find a solution.

[paul_truscott]

A customer is experiencing this issue in a big way, when using the search feature on Magento 2.2.4, we have been though every possible scenario of testing from reviewing the crons ( quite a few running from Brightpearl inventory management tool), and also now after upgrading the search tool to agolia, this seems to have made the issue worse (more crons and indexing). It is a server 404 not a Magento 404 so the site is not loading at all but once refreshed the page loads no problem. we are running a beefy AWS set up to server resources not an issue.

We have other sites for same customer with same set-up and no problems, only key difference would be database size (qty products) and the sites running without the error had products manually added, the site in question had product data migrated via lite extension.

Any ideas would be fantastic, we may have to apply this auto refresh workaround also as a stop gap ..

give me strength!

[Ricardo_Martins]

Holy moly. Even with Magento 2?!

Well, I'm using Magento 1.9.4.4, and my core_url_rewrite table has only 81 rows, 11 virtual products in the store, running at nexcess with redis and no catalog or category flat enabled.

I faced the 404 issue 3 times myself. And I can see 404 being logged on valid URLs on awstats.

Still have no idea why.

It would be nice if others share more details about the environment and configuration (redis, flat category, etc).

Update 4 months later...

I was able to see the error happening in my local/dev with developer mode activated. It seems to be something in Redis classes, as per my exception.log:

2020-04-22T04:23:05+00:00 ERR (3): 
Mage_Core_Model_Session_Exception in /path/to/magento/app/code/core/Mage/Core/Model/Session/Abstract/Varien.php:431
Stack trace:
#0 /path/to/magento/app/code/core/Mage/Core/Model/Session/Abstract/Varien.php(236): Mage_Core_Model_Session_Abstract_Varien->validate()
#1 /path/to/magento/app/code/core/Mage/Core/Model/Session/Abstract.php(84): Mage_Core_Model_Session_Abstract_Varien->init('reports', NULL)
#2 /path/to/magento/app/code/core/Mage/Reports/Model/Session.php(43): Mage_Core_Model_Session_Abstract->init('reports')
#3 /path/to/magento/app/code/core/Mage/Core/Model/Config.php(1357): Mage_Reports_Model_Session->__construct(Array)
#4 /path/to/magento/app/Mage.php(474): Mage_Core_Model_Config->getModelInstance('reports/session', Array)
#5 /path/to/magento/app/Mage.php(488): Mage::getModel('reports/session', Array)
#6 /path/to/magento/app/code/core/Mage/Reports/Model/Product/Index/Abstract.php(167): Mage::getSingleton('reports/session')
#7 /path/to/magento/app/code/core/Mage/Reports/Model/Product/Index/Abstract.php(185): Mage_Reports_Model_Product_Index_Abstract->_getSession()
#8 /path/to/magento/app/code/core/Mage/Reports/Model/Event/Observer.php(148): Mage_Reports_Model_Product_Index_Abstract->calculate()
#9 /path/to/magento/app/code/core/Mage/Core/Model/App.php(1374): Mage_Reports_Model_Event_Observer->catalogProductView(Object(Varien_Event_Observer))
#10 /path/to/magento/app/code/core/Mage/Core/Model/App.php(1353): Mage_Core_Model_App->_callObserverMethod(Object(Mage_Reports_Model_Event_Observer), 'catalogProductV...', Object(Varien_Event_Observer))
#11 /path/to/magento/app/Mage.php(459): Mage_Core_Model_App->dispatchEvent('catalog_control...', Array)
#12 /path/to/magento/app/code/core/Mage/Catalog/Helper/Product/View.php(135): Mage::dispatchEvent('catalog_control...', Array)
#13 /path/to/magento/app/code/core/Mage/Catalog/controllers/ProductController.php(132): Mage_Catalog_Helper_Product_View->prepareAndRender(10, Object(Mage_Catalog_ProductController), Object(Varien_Object))
#14 /path/to/magento/app/code/core/Mage/Core/Controller/Varien/Action.php(418): Mage_Catalog_ProductController->viewAction()
#15 /path/to/magento/app/code/core/Mage/Core/Controller/Varien/Router/Standard.php(254): Mage_Core_Controller_Varien_Action->dispatch('view')
#16 /path/to/magento/app/code/core/Mage/Core/Controller/Varien/Front.php(172): Mage_Core_Controller_Varien_Router_Standard->match(Object(Mage_Core_Controller_Request_Http))
#17 /path/to/magento/app/code/core/Mage/Core/Model/App.php(381): Mage_Core_Controller_Varien_Front->dispatch()
#18 /path/to/magento/app/Mage.php(694): Mage_Core_Model_App->run(Array)
#19 /path/to/magento/index.php(84): Mage::run('default', 'store')
#20 /Users/myuser/.composer/vendor/weprovide/valet-plus/server.php(131): require('/Users/myuser/...')
#21 {main}

I still need to investigate more, but I'm working on something else...

[medlington2]

Did anyone ever find a solution to this?

Im on 1.9.2.2 and getting this problem, its killing the crawl rate of Google bot and annoying a few customers too

[thomas34]

Bonjour 

J ai egalement le meme probleme , nous avons pas mal de reecritures egalement mais vu la reponse ricardo martin il semblerait qu il faille eliminie l hypothese d une table trop grosse  
Si quelqu'un a une idee je suis preneur ! 

Bonne journée