Import products and categories from external database
I am creating a webshop on which car parts can be ordered. I need to show a list of appropriate car parts to customers based on their filled in license plate.
My solution was to create a subcategory for each car type and connect products to those car type categories. Once a license plate is filled in, I am using a special controller action to check with an external database what the corresponding car type is. I then redirect users to the category page corresponding to that particular car type.
Since I am new to Magento, does this approach sound like a reasonable way to create this functionality?
I have the above mentioned functionality working, however I am now investigating the possibilities for importing large numbers of products and categories. The car types and products are stored in an external database, which is occasionally (daily to weekly) updated. I therefore need to automatically synchronise the Magento webshop with this data source.
I have about 26000 products and 32000 car type categories. Each product will be linked to roughly 20 car types. My initial plan was to just fill the database using custom queries. This works fast, however when I manually input a category with the right values to the different catalog_category_entity tables, it doesn't become visible. Am I missing updating some tables or maybe other operations?
I am also thinking about automatically creating a csv file of the external data and importing it into Magento. However, I am afraid that a csv import will be very slow (on 6GB RAM and 8 cores).
I am really looking forward to any advise you guys can give me.