Big amount of websites and stores

[Jack_Wo]

I am facing a situation where more than one thousand of websites/ stores are needed in one Magento instance.

After the number of website/ store exceeds 150, things become extremely slow including the admin panel and some of my customized extensions. 

I am wondering what makes things slow due to increasing number of websites/ stores. For example, even getting Oauth response becomes slow; and one of the main functionalities, placing order, gets slower too.

Any helps/ tips are extremely welcome. Thank you all in advance!

[sherrie]

Hi @Jack_Wo, which version of Magento are you on? Performance tips sometimes vary based on version.

[Jack_Wo]

Hi @sherrie, thank you for your reply!

I am using Magento Enterprise 1.14.2.4

[mhauri]

Besides the Magento version, what's the reason to have them all in one Magento instance?

[Jack_Wo]

@mhauri I am helping develop a platform for customers; it is not my decision but my boss's..

By doing so, we are able to control everything, while each customer will be able to access his/ her part.

[IvanChepurnyi]

Hi Jack,

First of all, you need to ask yourself what is the purpose of creating such an amount of store views. Is there any other way to implement the required functionality without increasing the size of your database?

The main performance problem you are facing coming from multiple places:

1. Configuration size growth

Magento loads all store views and all its related configuration for every page, it means if you have 100 store views for each of the store view there is going be done the resolution of all possible configuration values. In general, on average webshop a single store configuration might occupy around 32kb of data, if you have 100 of them, it might grow into 3.2mb of only text data itself, without considering memory consumed by PHP to parse XML structure and time to load it into the memory for processing. 

That is why you have slow performance on every page, not only on the ones that are related to the catalog.

2. Database size growth

Magento uses store views to pre-generate various indexes like layered navigation, price (websites) and flat (new table per each store view). Adding a new store view means adding records with the whole catalog just to pre-cache those values. 

As you know with the huge size of the table, MySQL performance might become a nightmare.

Solution

There is no simple solution that will fit any requirement easily, but in some cases, it is possible to achieve complex setups without the multiplying number of store view. Let's say if you sell the same products on multiple domains but with same prices, translations, payment options and shipping methods, you don't need to create a store view. You can create a custom request matcher for to replace at runtime base_url of the store view for a matched domain, instead of creating a store view for each domain. 

The same applies to other situations. Let's say if you need to change some configuration options, like home page or store name based on domain, you can create a separate entity that will at the real time replace Magento configuration values, without multiplying its size. 

Hope that will help you to properly diagnose your problem and find the right solution. 

[Jack_Wo]

Hi @IvanChepurnyi,

Thank you so much for your reply; the information is valuable and helpful.

Unfortunately, I will have to use multiple stores to achieve company's goal, because each individual business needs one store, and there is no easy way to merge them into fewer stores.

However, since we are mostly using api calls, is it possible that there are some shortcuts to avoid configuration generation?

Again, thank you for your precious time.

[IvanChepurnyi]

@Jack_Wo I was describing an easier solution for you, if you want to stay with stores then you will need the following:

1. Write your implementation of configuration loading process that will do a lazy load of configuration data on request for a particular store view. That is a tremendous amount of work that will help you to solve memory usage in configuration problem. 

2. You will have to rewrite all indexers in Magento where website/store view is used as a primary key part, so your database will not get into a wild growth with increased number of store views. That is a big peace of work as well

3. Final, but not the least important, your EAV structure will be out of control when you'll have to manage translations for different languages multiplied by the number of websites you have. Changing this structure might bring more problems than benefits, as a lot of modules depend on its structure.

[DavidStephan]

I have got a blog for you. 

http://magebase.com/magento-tutorials/magento-session-storage-which-to-choose-and-why/

Please check this and you will get the solution for Magento . Revert me, if my help comes out with good result.

[NirvanaServices]

You can't, Magento loads all data in to memory, at 100s of stores it becomes unusable both on the frontend and backend. You need to adjust your business model, having pseudo stores, using categories, price lists, or having multiple installs by group. There's no one solution, it is based on many tangible and intangible factors that are different per store That's all there is to it, a partner of ours tested it on multi-cluster hosting that enterprises use and Magento just ground to a halt! They had to adjust the business model so they could bring it down to 10-20 stores but provide the same functionality.