Optimizing Magento 1 to 2 Migrations scripts

[frank95a]

I am running the migration script for a Magento 1.9.3.7 to a Magento 2.3 install.

The store has less than 700 products and maybe 20K customers.

The script is stuck on log step and when I run Mytop, this is the SQL that I see for the past 3 days:

SELECT lv.visitor_id, lc.customer_id, lv.session_id, lv.last_visit_at FROM log_visitor AS lv LEFT JOIN log_customer AS lc ON lv.visitor_id = lc.visitor_id GROUP BY lv.visitor_id ORDER BY lv.visitor_id ASC LIMIT 10000 OFFSET 5440000

My question is, if I wanted to create an index on visitor_id in both tables before this step and drop the index after this step, what do I need to do? What do I have to change and where? Obviously having an index on vistor_id in both tables will help this query run a lot faster

Any help is appreciated.

Thank you

[Manish Mittal]

@frank95a 

Why you are not trying with manual creating index in both database table and then try.

One more thing just try migration command with -vvv you will know exact issue in which table you are getting.

[frank95a · Accepted solution]

Thanks... an index is valuable when you are doing the query on the table, but when you are inserting data, it is a little costly since you are not only doing the insert but also building the index. So I wanted to create the index on the fly and then drop it after I was done.

Regardless, it turned out not to be the problem. The problem was the 5 million records in the log table which is useless anyways. So I truncated and reran the process. It ended in under 5 minutes!

But thanks again

[Manish Mittal · Accepted solution]

Good to hear! Cheers

[molybrown38daf]

Thanks for sharing the solution and help to fix the issue. 

Netgear router login