Hey there,
I am helping out with one site to debunk the over 10s+ loading time and generation of 3-7 seconds per page. Pages showcasing 7 times generate in around 5 second, pages with 14 generate in 7 seconds.. So the loading times are pretty bad. We switched the server to a much faster one as a test, but we still only have got around 20% of an improvement.
Here's a couple of things that I would love to get some info with:
1) While stracing it seems that Magento, on every page load - goes through an insane amount of folders looking for the same php files.
For example:
23610 lstat("/home/admin/web/domain.name/public_html/app/code/community/Mage/Core/functions.php", 0x7ffed022cd70) = -1 ENOENT (No such file or directory)
23610 gettimeofday({1460032891, 644903}, NULL) = 0
23610 lstat("/home/admin/web/domain.name/public_html/app/code/core/Mage/Core/functions.php", {st_mode=S_IFREG|0646, st_size=10797, ...}) = 0
23610 gettimeofday({1460032891, 682191}, NULL) = 0
23610 lstat("/home/admin/web/domain.name/public_html/app/code/local/Zend/Db/Adapter/Pdo/Mysql.php", 0x7ffed022b4b0) = -1 ENOENT (No such file or directory)
23610 gettimeofday({1460032891, 682233}, NULL) = 0
23610 lstat("/home/admin/web/domain.name/public_html/app/code/community/Zend/Db/Adapter/Pdo/Mysql.php", 0x7ffed022b4b0) = -1 ENOENT (No such file or directory)
23610 gettimeofday({1460032891, 682288}, NULL) = 0
23610 lstat("/home/admin/web/domain.name/public_html/app/code/core/Zend/Db/Adapter/Pdo/Mysql.php", 0x7ffed022b4b0) = -1 ENOENT (No such file or directory)
23610 gettimeofday({1460032891, 682350}, NULL) = 0
23610 lstat("/home/admin/web/domain.name/public_html/lib/Zend/Db/Adapter/Pdo/Mysql.php", {st_mode=S_IFREG|0646, st_size=9326, ...}) = 0
23610 gettimeofday({1460032891, 645632}, NULL) = 0
23610 lstat("/home/admin/web/domain.name/public_html/app/code/local/Varien/Autoload.php", 0x7ffed022cd70) = -1 ENOENT (No such file or directory)
23610 gettimeofday({1460032891, 645673}, NULL) = 0
23610 lstat("/home/admin/web/domain.name/public_html/app/code/community/Varien/Autoload.php", 0x7ffed022cd70) = -1 ENOENT (No such file or directory)
23610 gettimeofday({1460032891, 645729}, NULL) = 0
23610 lstat("/home/admin/web/domain.name/public_html/app/code/core/Varien/Autoload.php", 0x7ffed022cd70) = -1 ENOENT (No such file or directory)
23610 gettimeofday({1460032891, 645768}, NULL) = 0
23610 lstat("/home/admin/web/domain.name/public_html/lib/Varien/Autoload.php", {st_mode=S_IFREG|0646, st_size=4885, ...}) = 0
1.1)
So does it need to include all of these on every page generation? I suppose so - since it's PHP. But seriously, around 4000 lsstat requests for one page to open? Wow..
1.2)
I am sure Though that it wouldn't need to do so many IOPS to randomly search in various dirs? CPU & IOPS go crazy. Is this something normal? Or is there a problem with our installation? For example with Mysql.php, it searches for that in 3 different folders until it finds it in /lib/Zend/ - Why??
In the end, right now for a single page load there's about 3974 lstat calls and 799 errors.
2) gettimeofday? Constantly?
For a single page load there's about 3927 requests for get time of day (with every lstat), is this normal in this case?
3) Additional info:
Full strace is available here: http://tinyurl.com/hr54ybs
Full strace -fc is available here: http://tinyurl.com/god5pcu
These are made on a single load of a category page with 36 items.
Any help is greatly appreciated, the strace is from a server running with memcached.
Before offering to upgrade to an insanely fast server, I wanted to go around and see if there are any issues, then I will probably go to look at caching. Hopefully somebody can give me some pointers and also I hope that this will help somebody else as well!
TYVM.
Yukinarija
---
Update 1:
Disabling memcached gave an increase to both lstat & gettimeofday, and did not make the multiple file searches go away. I logically it shouldn't make it better in any case - though I would just like to be clear, that it's not involved here.
