A multi-tenant auto-scaling low-cost infrastructure for hosting Magento 2 stores: Can it be done?

[Eric Seastrand]

Currently the company I work for has about a hundred Magento 1 stores on an infrastructure we built on AWS. Here's a diagram of how it currently works:

[Not pictured: Redis server for caching and sessions]

Obviously this isn't as fast as having a single dedicated server with SSD storage, but it has worked well for us for many years. It allows us to easily add new stores, and unless they get lots of traffic, they don't really cost us anything to keep around. In the event of a high-traffic event, we can add more "app" servers, and keep everything stable. It also makes our annual PCI compliance audit simpler, because we don't have to audit a bunch of separate dedicated servers.

We would now like to start using Magento 2, but what we are finding is that when we deploy Magento 2 on this sort of infrastructure, even with all caches enabled and in production mode, the stores are just too slow. I mean, unusably slow: 20 second load times on admin pages slow.

In trying to diagnose the speed issues (using xdebug profiler and kCacheGrind), and I'm finding that in almost every case, Magento is spending a TON of time on filesystem-related actions like file_exists(). Obviously those calls take a lot of time because we're hosting the code on a network-attached disk. Hosting separate copies of the Magento code on each application server would add cost, maintenance overhead, and complexity, so that isn't really an option (not to mention: I'm just a developer -- our sysadmins make those decisions).

Is what we're trying to do even possible? Has anyone else had any success doing it?

[Tom Robertshaw]

Having all the code sitting on NFS is a setup I'm only familiar in development environments and it is slow, even with caches enabled. I know you/the sysadmins said otherwise but there's not much you can do to Magento on NFS up so if you want this setup then I'd recommend deploying code to each app server and just using NFS for shared writable directories like media. 

[kgerasimov]

Hello,

I don't think M2 will show good performance without SSD. It is just they way it was designed, it does lots of reads/writes to disk. 

Is frontend slow as well?

[Eric Seastrand]

Hi Konstantin,

I am inclined to think that you are correct. Our sysadmins are now looking into using some sort of filesystem caching in the NFS layer. I'll definitely report back if we're successful.

To answer your question, yes, the frontend is also unusably slow, even with all caches enabled.

[kgerasimov]

Is that a fresh M2 install?

Are you sure it is your infrastructure and not the Magento 2 installation itself.

Here is a simple trick I use very often to justify to my clients their hosting is OK:

If you run a custom M2 try to install a fresh M2 on the same hosting and then compare results. If the fresh M2 is a lot faster then your custom M2 than your infrastructure is OK and you need to look into M2. If the fresh M2 is as slow as your custom M2 then your hosting is the problem.

[Eric Seastrand]

It is indeed a fresh install on v2.2, created using the composer installation method.

The weird thing is that Magento 1 runs perfectly fine on the same hosting. We also run some Yii2 sites on it, and they are pretty fast.

Maybe I'll try putting our slow Magento 2 on a virtual machine to compare.