ElasticSearch and Magento 2.4.1

[alexander_frost]

I upgraded from magento 2.3.1 to magento 2.4.1 -- I have ElasticSearch installed and Test Successful from the Magento Admin Panel.

If I start the ElasticSearch service ... the site works and you can start searching .. but as searches come in the php74.cgi starts to grow and grow and grow ... and eventually uses all the resources on the server.

%Cpu(s): 9.7 us, 4.7 sy, 0.0 ni, 6.9 id, 76.3 wa, 0.0 hi, 2.3 si, 0.0 st
KiB Mem : 4006920 total, 53148 free, 3749040 used, 204732 buff/cache
KiB Swap: 4194300 total, 2198488 free, 1995812 used. 81636 avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
32482 cosemp 20 0 2973804 2.400g 1280 R 84.3 62.8 0:25.06 php74.cgi

We were running the Magento 2.3.1 version with the MySQL search and the site was blazing fast... after the upgrade and using ElasticSearch the site now crawls and ultimately stops responding 

If I stop the ElasticSearch Service, everything calms down but obviously all the Categories are tied to the ELasticSearch so the products dont show up ... a reindex shows there are no nodes in the cluster... 

SO my question is ... why does ElasticSearch make the php74.cgi grow and grow to use up all the resources??? How can I solve this issue??? 

Or Can I downgrade the site back to 2.3.1??? 2.4.1 is not working well...

Thank you!

Lex

[alexander_frost]

After much more debugging, I found out it was the MSG MegaMenu that was causing the issue. When a user clicked on a Menu item... say Jewelry ... it was doing something that was making the php74.cgi peg the CPU at 100% and eat up 3GB of ram. 

If I disabled the MSG MegaMenu plugin the site ran much better, I could go to the category manually domain.com/jewelry and the products would display but it did not spike the CPU or Ram .. so it is definitely something to do with the MSG MegaMenu .. but I dug through all the logs and could not find any error reference to the issue.

We are using the Supro Theme that comes with a bunch of MSG Plugins so I am a bit concerned that not all the plugins are 2.4 compatible.

[janejenkin5a96]

Thanks for the information. keep sharing your experiance NavyArmyCCU