Elasticsearch showing error index_not_found_exception during indexer execute by cron

[vikasp3che07bd]

We are getting issues related to index_not_found_exception and resource_already_exists_exception errors related to elasticsearch in the system.log file in our Magento application and due to this products are being disappeared from the category and search listing page.

But after deleting all indexes of elasticsearch alias and again if we run the indexer command php bin/magento indexer:reindex catalogsearch_fulltext manually then it generates only one alias and products started showing properly on frontend.

Below errors are coming when Magento group indexes are run by cron.

CRITICAL: Elasticsearch\Common\Exceptions\Missing404Exception: {"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index [productionmagento2_product_1_v24]","index_uuid":"m6vXUaRxQsKbgHO-FTbuWA","index":"productionmagento2_product_1_v24"}],"type":"index_not_found_exception","reason":"no such index [productionmagento2_product_1_v24]","index_uuid":"m6vXUaRxQsKbgHO-FTbuWA","index":"productionmagento2_product_1_v24"},"status":404} in /var/www/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php:685

We observe that this error comes only when the catalogsearch_fulltext indexer is run by cron.

Our indexer mode has set up Update on Schedule.

Also, at the time of creating indexes on Elasticsearch, it creates the multiple indexes alias on the Elasticsearch server.

We have currently 95K products in our Magento store.

Thanks in advance.

[theMageComp]

Hello @vikasp3che07bd 

1. Make sure your Elasticsearch server is running and accessible by your Magento application.
2. Check your Elasticsearch configuration in Magento. Ensure that the Elasticsearch hostname, port, and index prefix are correctly set in your Magento configuration files (app/etc/env.php).
3. Verify that the Elasticsearch version you are using is compatible with your Magento version. Refer to the official Magento documentation for compatibility information.
4. If you have multiple indexes and aliases created in Elasticsearch, ensure that the correct index alias is configured in Magento.
5. Delete all existing indexes and aliases in Elasticsearch related to your Magento installation. You can use Elasticsearch APIs or command-line tools to perform this task.
6. Run the indexer command manually to reindex the catalogsearch_fulltext index: php bin/magento indexer:reindex catalogsearch_fulltext. Monitor the process for any errors or exceptions.
7. Verify that the catalogsearch_fulltext index is created successfully in Elasticsearch and associated with the correct alias.

It's recommended to check your Magento and Elasticsearch configurations thoroughly to ensure they are properly aligned. Additionally, consider monitoring your Elasticsearch logs for any relevant error messages that could provide further insight into the issue.

If the problem persists, you may need to investigate further or seek assistance from a Magento developer or Elasticsearch expert to diagnose and resolve the Elasticsearch indexing issues specific to your environment.

[vikasp3che07bd]

Hi MageComp,

Thank you for your reply on this.

We have followed all your steps but still this issues is occurring and as per our current finding we noticed that this issue is being fixed when we comment the batch_size indexer configuration value in env.php file and as we uncomment this and after invalidating catalogsearch_fulltext indexer then it started showing index_not_found_exception error again.

<?php
return [
    'indexer' => [
        'batch_size' => [
            'cataloginventory_stock' => [
                'simple' => 200
            ],
            'catalog_category_product' => 666,
            'catalogsearch_fulltext' => [
                'partial_reindex' => 100,
                'mysql_get' => 500,
                'elastic_save' => 500
            ],
            'catalog_product_price' => [
                'simple' => 200,
                'default' => 500,
                'configurable' => 666
            ],
            'catalogpermissions_category' => 999,
            'inventory' => [
                'simple' => 210,
                'default' => 510,
                'configurable' => 616
            ]
        ]
    ]
];

We added this configuration value for indexer optimization as per magento devdocs suggestions https://developer.adobe.com/commerce/php/development/components/indexing/optimization/.

so, what we can do in this case?

Thanks in advance.