Removing 85.000 products from backend takes 28 hours!

[apiuser]

Feature request from rklerck, posted on GitHub Sep 09, 2015

We've added 85000 products with rest, this takes 14 hours. This seems fair because of a request for each product. But when we remove the products all at once in the backend, it takes 28 hours!!

[apiuser]

Comment from kandy, posted on GitHub Sep 09, 2015

First of all, better to use indexation on schedule for this case. Second, in your case better to use import functionality that was optimized for mass update.

[apiuser]

Comment from rklerck, posted on GitHub Sep 09, 2015

Use import functionality for mass delete? We don't really care import takes this long, it's the initial load..

[apiuser]

Comment from vrann, posted on GitHub Sep 10, 2015

@rklerck hi, can you please confirm that you use "reindex on schedule" for all the indexers related to catalog?

[apiuser]

Comment from rklerck, posted on GitHub Sep 10, 2015

Tried that, ended up with https://github.com/magento/magento2/issues/1833

[apiuser]

Comment from pboisvert, posted on GitHub May 04, 2016

@choukalos Chuck--i see your reference to a closed issue. Can you take a look at this one and reassign as needed?

[apiuser]

Comment from choukalos, posted on GitHub Jul 26, 2016

Hi @rklerck

What version of Magento are you using? Did you use REST API to delete the entire catalog ( i.e. all 85k products )?
Is your use case the need to delete the entire catalog quickly - for what reason? Is this a merchant use case, why? Do you have merchant use cases where you'd only want to delete portions of the catalog? What time to do so would you expect?

Thanks, Chuck

[apiuser]

Comment from acidreian666, posted on GitHub Jul 26, 2016

@choukalos Having similar issues with my large product database also. Magento 2.1 is unable to import more then 95000 products at a time if it don't fail already because it says the "URL already exists" and it ends with importing 66k of the 95k list of products then breaks the indexing ability.

Then like @rklerck says Mass Delete function for me i have let run for 3 days to delete over 85k products at once because they imported and had missing data causing an Segmentation Core error while indexing keywords.

Magmi for 1.92 was amazing at importing/updating products but this isn't an option for magento 2.x imagine having a perfect Database CSV file made with 180k products then using magento 2.x OEM import tool and it fails everytime.

to compare to what ever @rklerck server is mine is. Magento 2.1 ubuntu 16.04 php7.08 Nginx mysql5.7 4 core cpu 24gb ram mysql and php both tuned to use high amounts of memory execution times = umlimited

result = magento 2.x needs Magmi style import/update/delete system.