Hi
Magento v 2.4.5
Elasticsearch v 7.17.9
This has been an ongoing intermittent issue for several months.
All products (15000+) suddenly do not show in categories on the frontend. Products can be seen through their direct URL.
This appears to happen when staff are editing products, or switching them between categories in the backend, although this is not reproduceable so it may be coincidental.
We have made the following changes over the past few weeks:
- Upgraded Elasticsearch from 6.8.23 to 7.17.9
- Corrected an issue where our POS system was using the same Elasticsearch indices for LIVE and STAGING sites.
Before these changes we used to get Elasticsearch crashing at the same time as the products all disappearing. To bring back all products we would run:
service elasticsearch restart
php bin/magento indexer:reindex
Since upgrading Elasticsearch it is no longer crashing.
To solve the problem now we just reindex.
System.log shows errors like this at or around the time the products disappear:
[2023-03-06T20:55:11.703339+00:00] main.CRITICAL: Elasticsearch\Common\Exceptions\BadRequest400Exception: {"error":{"root_cause":[{"type":"query_shard_exception","reason":"No mapping found for [position_category_9] in order to sort on","index_uuid":"gaRlC2HOSy2-X4XnI0kzQQ","index":"magento2_webpos_productsearch_fulltext_1_v84"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"magento2_webpos_productsearch_fulltext_1_v84","node":"Em9qFtSWSpafRrRYnfSzvg","reason":{"type":"query_shard_exception","reason":"No mapping found for [position_category_9] in order to sort on","index_uuid":"gaRlC2HOSy2-X4XnI0kzQQ","index":"magento2_webpos_productsearch_fulltext_1_v84"}}]},"status":400} in /home/*********/public_html/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php:693
magento2_webpos_productsearch_fulltext_1_v84 is an index used by our POS system. I have queried this with them as well.
Any suggestions of what might be happening, or where to look next would be really useful!
Thanks
Ben
Some details about our Elasticsearch below:
curl http://localhost:9200/_cluster/health?pretty
{
"cluster_name" : "elasticsearch",
"status" : "yellow",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
"active_primary_shards" : 51,
"active_shards" : 51,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 48,
"delayed_unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0,
"task_max_waiting_in_queue_millis" : 0,
"active_shards_percent_as_number" : 51.515151515151516
}
service elasticsearch status
● elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2023-03-06 20:26:37 GMT; 12h ago
Docs: https://www.elastic.co
Main PID: 26680 (java)
CGroup: /system.slice/elasticsearch.service
├─26680 /usr/share/elasticsearch/jdk/bin/java -Xshare:auto -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+Alwa...
└─26936 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller
Mar 06 20:26:25 ssd356r.simpleservers.co.uk systemd[1]: Starting Elasticsearch...
Mar 06 20:26:37 ssd356r.simpleservers.co.uk systemd[1]: Started Elasticsearch.
curl -XGET 'http://localhost:9200'
{
"name" : "***********",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "ZOn1c039RZCAGehzZ3D8tw",
"version" : {
"number" : "7.17.9",
"build_flavor" : "default",
"build_type" : "rpm",
"build_hash" : "ef48222227ee6b9e70e502f0f0daa52435ee634d",
"build_date" : "2023-01-31T05:34:43.305517834Z",
"build_snapshot" : false,
"lucene_version" : "8.11.1",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
curl http://localhost:9200/_aliases?pretty=true
{
"magento2staging_product_3_v3" : {
"aliases" : {
"magento2staging_product_3" : { }
}
},
"magento2_product_1_v2" : {
"aliases" : {
"magento2_product_1" : { }
}
},
"magento2staging_product_5_v3" : {
"aliases" : {
"magento2staging_product_5" : { }
}
},
"magento2_product_5_v2" : {
"aliases" : {
"magento2_product_5" : { }
}
},
"magento2staging_webpos_productsearch_fulltext_1_v3" : {
"aliases" : {
"magento2staging_webpos_productsearch_fulltext_1" : { }
}
},
"magento2_webpos_productsearch_fulltext_3_v2" : {
"aliases" : {
"magento2_webpos_productsearch_fulltext_3" : { }
}
},
"magento2_webpos_productsearch_fulltext_1_v2" : {
"aliases" : {
"magento2_webpos_productsearch_fulltext_1" : { }
}
},
"magento2staging_webpos_productsearch_fulltext_3_v3" : {
"aliases" : {
"magento2staging_webpos_productsearch_fulltext_3" : { }
}
},
"magento2staging_product_2_v3" : {
"aliases" : {
"magento2staging_product_2" : { }
}
},
"magento2_webpos_productsearch_fulltext_2_v2" : {
"aliases" : {
"magento2_webpos_productsearch_fulltext_2" : { }
}
},
"magento2_webpos_productsearch_fulltext_5_v2" : {
"aliases" : {
"magento2_webpos_productsearch_fulltext_5" : { }
}
},
"magento2staging_webpos_productsearch_fulltext_2_v3" : {
"aliases" : {
"magento2staging_webpos_productsearch_fulltext_2" : { }
}
},
"magento2_product_2_v2" : {
"aliases" : {
"magento2_product_2" : { }
}
},
"magento2_product_3_v2" : {
"aliases" : {
"magento2_product_3" : { }
}
},
"magento2staging_product_1_v3" : {
"aliases" : {
"magento2staging_product_1" : { }
}
},
"magento2staging_webpos_productsearch_fulltext_5_v3" : {
"aliases" : {
"magento2staging_webpos_productsearch_fulltext_5" : { }
}
}
}
