HI
We are a set of custom scripts running on a set of Magento 2.2.7 installations.
As Magento 2.3 is now out, we wanted to make sure that our code was running perfectly on the new version.
To make the cleanest possible test, we made a completely new Magento 2.3.0 installation on a similar host as the other Magento 2.2 installations.
As our custom scripts pulls 110.000 products and 6.000 product categories from files on an external server, it does take some time to process.
This means that we disable the cronjobs and indexing is forced to update by schedule and since the cronjobs are also disabled, nothing should create deadlocks on MySQL when updating the database with this huge amount of data.
Loading this on a fresh 2.3.0 database processed the 6.000 categories in around 3 minutes, which is around the same as the 2.2.7 databases do.
We initially run the 2.3.0 installation and our custom scripts, using PHP 7.2.14
As we got some strange performance issues when we started to process the products, we aborted and switched the scripts to PHP 7.1.26 instead.
This gave the same result.
To explain what we are experiencing, the categories process in about the same time as Magento 2.2.7. But when we start processing products, the story changes quite a bit. As more and more products are processed, it takes longer and longer to create each new product in Magento 2.3.0. In Magento 2.2.7, the scripts runs at a quite steady speed and the entire script is finished and database is reindex in just under 6 hours.
The Magento 2.3.0 version of the same code has now been running for 27 hours and has processed 6.000 categories and 13.200 products.
To compare this to Magento 2.2.7, the script has now been running for 15 minutes and has processed 6.000 categories and 11.000 products.
Through some debugging of the scripts, we have found that when we call save() on the created product object, then this is where the problem is seen.
For server performance we see 100% usage by PHP on one CPU core, but only 2-5% usage from MySQL. All servers are running highspeed nVMe storage, so diskspeed should not be a problem at all
Does anyone know why we see such a huge performance degradation for Magento 2.3?
My guess would be some kind of bug for the indexing, so although we have turned off the cronjobs and left the indexers to update by schedule, something is still updating indexes upon product creation/update.
