Products Randomly Disappear from Category View on Storefront and search page

[ronakgwebm215b]

I'm experiencing an issue in my Magento 2 store where products randomly disappear from category view on the storefront. The problem resolves itself temporarily after a hard refresh of the page, but then reappears later. This inconsistency is causing a significant impact on user experience and sales.

Details of the issue:

• Products are correctly assigned to categories in the Magento Admin Panel.
• The issue occurs randomly and affects different categories.
• Products have stock managed (qty and salable qty), and visibility is set to "Catalog, Search."
• Product status is enabled.
• Clearing cache and reindexing temporarily fixes the issue until it happens again.
• No errors related to this issue are found in Magento logs.

Magento Version : 2.4.3
We are using Opensearch (AWS).

After indexing products, they become visible temporarily, but we continue to encounter the same issue afterward.
I'm seeking advice on how to troubleshoot and permanently resolve this issue. Any insights or suggestions would be greatly appreciated.

Thank you for your help!

[Sanjay Jethva]

@ronakgwebm215b 

Hey,

Try to reindex or check the system and exception log if you see any error then that shows to us to make sure that you have configured opensearch properly.

Thank You!

[ronakgwebm215b]

Hello @Sanjay Jethva ,

First of all, thank you for your help.

As per your suggestion, I have checked my exception log file.

Exception log:

main.CRITICAL: ReflectionException: Class "Magento\Framework\App\PageCache\Identifier\Interceptor" does not exist in /var/www/magento/vendor/magento/framework/Code/Reader/ClassReader.php:34"

[frelanhard5a81]

Hey @ronakgwebm215b, have you checked the Opensearch logs for any errors around indexing or product data? They might provide clues alongside Magento logs.

[ronakgwebm215b]

Hello  @frelanhard5a81 

First of all, thank you so much for your help.

As you mentioned, below is my OpenSearch log:

main.CRITICAL: OpenSearch\Common\Exceptions\BadRequest400Exception: {"error":{"root_cause":[{"type":"query_shard_exception","reason":"failed to create query: Cannot search on field [upc] since it is not indexed.","index":"magento2_electro2_product_46_v308","index_uuid":"B_XJ4lWzQc29n9JAPG17lQ"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"magento2_electro2_product_46_v308","node":"sHozfsg0SJiRkYNICUlGZw","reason":{"type":"query_shard_exception","reason":"failed to create query: Cannot search on field [upc] since it is not indexed.","index":"magento2_electro2_product_46_v308","index_uuid":"B_XJ4lWzQc29n9JAPG17lQ","caused_by":{"type":"illegal_argument_exception","reason":"Cannot search on field [upc] since it is not indexed."}}}]},"status":400} in /var/www/Magento/vendor/opensearch-project/opensearch-php/src/OpenSearch/Connections/Connection.php:661
Stack trace:
#0 /var/www/Magento/vendor/opensearch-project/opensearch-php/src/OpenSearch/Connections/Connection.php(328): OpenSearch\Connections\Connection->process4xxError()
#1 /var/www/Magento/vendor/react/promise/src/FulfilledPromise.php(28): OpenSearch\Connections\Connection->OpenSearch\Connections\{closure}()
#2 /var/www/Magento/vendor/ezimuel/ringphp/src/Future/CompletedFutureValue.php(65): React\Promise\FulfilledPromise->then()
#3 /var/www/Magento/vendor/ezimuel/ringphp/src/Core.php(341): GuzzleHttp\Ring\Future\CompletedFutureValue->then()
#4 /var/www/Magento/vendor/opensearch-project/opensearch-php/src/OpenSearch/Connections/Connection.php(340): GuzzleHttp\Ring\Core:roxy()
#5 /var/www/Magento/vendor/opensearch-project/opensearch-php/src/OpenSearch/Connections/Connection.php(238): OpenSearch\Connections\Connection->OpenSearch\Connections\{closure}()
#6 /var/www/Magento/vendor/opensearch-project/opensearch-php/src/OpenSearch/Transport.php(113): OpenSearch\Connections\Connection->performRequest()
#7 /var/www/Magento/vendor/opensearch-project/opensearch-php/src/OpenSearch/Client.php(1389): OpenSearch\Transport->performRequest()
#8 /var/www/Magento/vendor/opensearch-project/opensearch-php/src/OpenSearch/Client.php(1045): OpenSearch\Client->performRequest()
#9 /var/www/Magento/vendor/magento/module-open-search/Model/OpenSearch.php(51): OpenSearch\Client->search()
#10 /var/www/Magento/vendor/magento/module-open-search/SearchAdapter/Adapter.php(111): Magento\OpenSearch\Model\OpenSearch->query()
#11 /var/www/Magento/vendor/magento/module-search/Model/SearchEngine.php(42): Magento\OpenSearch\SearchAdapter\Adapter->query()
#12 /var/www/Magento/vendor/magento/framework/Search/Search.php(85): Magento\Search\Model\SearchEngine->search()
#13 /var/www/Magento/vendor/magento/module-catalog-search/Model/ResourceModel/Fulltext/Collection.php(464): Magento\Framework\Search\Search->search()

[frelanhard5a81 · Accepted solution]

You're welcome, Madlen!

From the log, it looks like the issue might be related to a non-indexed field in your OpenSearch setup. Ensuring all necessary fields are indexed properly could help improve performance. You might want to check your OpenSearch schema and confirm the 'upc' field is indexed correctly. Also, reviewing your Magento and OpenSearch integration settings could be beneficial.

Hope this points you in the right direction! 

On a side note, I recently had to ensure all fields were correctly indexed while setting up an inventory system for Burlington handles, which made a significant difference in performance.

[toniagency82ed · Accepted solution]

You say after reindex everything is fine for a short while? 

Are you maybe running a staging environment on the same server using the same opensearch indexes that are overiding each other as one gets reindexed?

[ronakgwebm215b]

@toniagency82ed 

Absolutely, you're spot on! That's exactly why the issue occurred. We've now set up separate environments for both staging and production for Open Search. Thanks a lot for your suggestion!