- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-25-2020
07:38 AM
09-25-2020
07:38 AM
Large collection processing
Hello I am currently upgrading our magento store from 1.4 to 2.4 so I am just exporting / importing data via scripts. I am currently running into issues with large order / credit memo collections around 180,000. I have tried looping through the collection by pages by eventually the page takes longer to load than the previous one and slows to a crawl. I have also tried the iterator which was very fast at first but then also slows down eventually and crashes. I have used collection->clear() etc . Any ideas on exporting these large data sets? Maybe I should look at the raw db and export from there? Im kind of stuck at the moment. Perhaps I should just export 40k chunks or something instead of all in one go. Thanks in advance!
Josh
Labels: