Config and config cache with 1K to 10k websites/ stores
I am trying to find a way to improve Magento performance when the amount of websites/ stores exceeds 1k, and my goal is around 10k. Here are some questions; any tips/ helps are extremely welcome!
1. Adding new websites/ stores is slow;
I comment out $this->cleanModelCache() in _afterSave() in Mage_Core_Model_Abstract, and the situation seems better but is getting slower with increasing number of websites/ stores. And I don't know what would this affect the whole system in the future.
2. Api calls become slow.
One of the main processes is to place order; my customized model deals with it by processing some data, and essentially using sales/quote model and sales/service_quote models. The process begins with Oauth. Both Oauth and placing order take longer when the number of website/ store grows, and the memory consumption seems larger. Does this have something to do with Mage loading the config xml, and the fact that config data gets bigger with increasing number of websites?
3. Opening up n98-magerun dev:console is taking longer; don't know the cause of it.
4. Saving configuration from admin panel takes longer; don't know how to improve it.
Is it possible to reconstruct the way Magento generate and load config data to lower its memory consumption? Is this one of the factors that cause performance issue for my situation?
Current Magento instance:
Version = Magento EE 18.104.22.168
Config cache on; other cache off.
Using Mysql 5.6 and MongoDB( for catalog_category_entity, catalog_product_entity, core_website)
number of websites = number of stores = number of views = 1024;