cancel
Showing results for 
Search instead for 
Did you mean: 

Magento 2.1.6 Performace Problems

SOLVED

Magento 2.1.6 Performace Problems

Hi,

 

My store used to be very fast, but now I am getting >10 second load times.  Recent system changes are, upgraded to 2.1.6, installed system patches, upped PHP FPM servers/children.   Below are my configuration files and system specs.   Any idea why I'm slow?  I'm at my wits end.   When I use curl I do see no-cache directives I don't think I should see that though, should I?  I also have varnish logs for a main site load, not included here, its 900+ lines.   Let me know if its useful and I will include.

 

Thanks for any help you can give.

 

Edit: Time to first byte is really high
ttfb.png

curl fetch

curl -I -v --location-trusted https://www.mysite.com/
*   Trying xxx.xxx.xxx.xxx...
* Connected to www.mysite.com (xxx.xxx.xxx.xxx) port 443 (#0)
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs/
* TLSv1.2, TLS handshake, Client hello (1):
* TLSv1.2, TLS handshake, Server hello (2):
* NPN, negotiated HTTP1.1
* TLSv1.2, TLS handshake, CERT (11):
* TLSv1.2, TLS handshake, Server key exchange (12):
* TLSv1.2, TLS handshake, Server finished (14):
* TLSv1.2, TLS handshake, Client key exchange (16):
* TLSv1.2, TLS change cipher, Client hello (1):
* TLSv1.2, TLS handshake, Unknown (67):
* TLSv1.2, TLS handshake, Finished (20):
* TLSv1.2, TLS change cipher, Client hello (1):
* TLSv1.2, TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* Server certificate:
*        subject: C=US; postalCode=***; ST=***; L=***; street=***; street=***; O=***; OU=InstantSSL Pro; CN=www.mysite.com
*        start date: 2015-04-29 00:00:00 GMT
*        expire date: 2018-05-06 23:59:59 GMT
*        subjectAltName: www.mysite.com matched
*        issuer: C=GB; ST=Greater Manchester; L=Salford; O=COMODO CA Limited; CN=COMODO RSA Organization Validation Secure Server CA
*        SSL certificate verify ok.
> HEAD / HTTP/1.1
> Host: www.mysite.User
> com-Agent: curl/7.42.1
> Accept: */*
>
HTTP/1.1 200 OK
Server: nginx
Date: Fri, 14 Apr 2017 14:45:58 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
Vary: Accept-Encoding
Set-Cookie: PHPSESSID=99999999999999999999999999999999; expires=Fri, 14-Apr-2017 15:45:56 GMT; Max-Age=3600; path=/; domain=www.mysite.com; secure; HttpOnly
Set-Cookie: mage-messages=%5B%5D; expires=Sat, 14-Apr-2018 14:45:58 GMT; Max-Age=31536000; path=/
Pragma: no-cache
Cache-Control: max-age=0, must-revalidate, no-cache, no-store
Expires: Thu, 14 Apr 2016 14:45:56 GMT
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Accept-Ranges: bytes
Strict-Transport-Security: max-age=63072000; includeSubdomains;

 

System

  • openSUSE 13.2 (Harlequin) (x86_64)
  • Redis 3.2.3
  • nginx/1.11.4
  • varnish-4.1.3 revision 5e3b6d2
  • mysql Ver 15.1 Distrib 10.0.27-MariaDB, for Linux (x86_64) using readline 5.2

Magento Config

  • Merge JavaScript Files: Yes
  • Enable JavaScript Bundling: Yes
  • Minify JavaScript Files: Yes
  • Merge CSS Files: Yes
  • Minify CSS Files: Yes
  • Production mode

memory

 

# free -m
             total       used       free     shared    buffers     cached
Mem:          3958       3816        141         32          0       2931
-/+ buffers/cache:        885       3073
Swap:         2055          0       2055

 

 

CPU

 

DAEMON_OPTS="-a :80 \
             -T 127.0.0.1:6082 \
             -f /usr/local/etc/varnish/default.vcl \
             -p cli_buffer=16384 \
             -P /var/run/varnishd.pid \
             -p http_resp_hdr_len=64000 \
             -S /usr/local/etc/varnish/secret \
             -s malloc,256m \
             -F"

 

# lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                4
On-line CPU(s) list:   0-3
Thread(s) per core:    2
Core(s) per socket:    1
Socket(s):             2
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            15
Model:                 4
Model name:            Intel(R) Xeon(TM) CPU 3.00GHz
Stepping:              1
CPU MHz:               3000.191
BogoMIPS:              5999.17
L1d cache:             16K
L2 cache:              1024K
NUMA node0 CPU(s):     0-3

 

 

Varnish

 

DAEMON_OPTS="-a :80 \
             -T 127.0.0.1:6082 \
             -f /usr/local/etc/varnish/default.vcl \
             -p cli_buffer=16384 \
             -P /var/run/varnishd.pid \
             -p http_resp_hdr_len=64000 \
             -S /usr/local/etc/varnish/secret \
             -s malloc,256m \
             -F"

 

 

vcl 4.0;

import std;

backend default { .host = "localhost"; .port = "8080"; .first_byte_timeout = 600s; } backend mysite { .host = "xxx.xxx.xxx.xxx"; .port = "8080"; .first_byte_timeout = 600s; } acl purge { "localhost"; "192.168.0.0"/24; } sub vcl_recv { if (req.http.host ~ "mysite.com") { set req.backend_hint = mysite; } else { set req.backend_hint = default; } if (req.method == "PURGE") { if (client.ip !~ purge) { return (synth(405, "Method not allowed")); } if (!req.http.X-Magento-Tags-Pattern) { return (synth(400, "X-Magento-Tags-Pattern header required")); } ban("obj.http.X-Magento-Tags ~ " + req.http.X-Magento-Tags-Pattern); return (synth(200, "Purged")); } if (req.method != "GET" && req.method != "HEAD" && req.method != "PUT" && req.method != "POST" && req.method != "TRACE" && req.method != "OPTIONS" && req.method != "DELETE") { /* Non-RFC2616 or CONNECT which is weird. */ return (pipe); } # We only deal with GET and HEAD by default if (req.method != "GET" && req.method != "HEAD") { return (pass); } # Bypass shopping cart, checkout and search requests if (req.url ~ "/checkout" || req.url ~ "/catalogsearch") { return (pass); } # normalize url in case of leading HTTP scheme and domain set req.url = regsub(req.url, "^http[s]?://", ""); # collect all cookies std.collect(req.http.Cookie); # Compression filter. See https://www.varnish-cache.org/trac/wiki/FAQ/Compression if (req.http.Accept-Encoding) { if (req.url ~ "\.(jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf|flv)$") { # No point in compressing these unset req.http.Accept-Encoding; } elsif (req.http.Accept-Encoding ~ "gzip") { set req.http.Accept-Encoding = "gzip"; } elsif (req.http.Accept-Encoding ~ "deflate" && req.http.user-agent !~ "MSIE") { set req.http.Accept-Encoding = "deflate"; } else { # unkown algorithm unset req.http.Accept-Encoding; } } # Remove Google gclid parameters to minimize the cache objects set req.url = regsuball(req.url,"\?gclid=[^&]+$",""); # strips when QS = "?gclid=AAA" set req.url = regsuball(req.url,"\?gclid=[^&]+&","?"); # strips when QS = "?gclid=AAA&foo=bar" set req.url = regsuball(req.url,"&gclid=[^&]+",""); # strips when QS = "?foo=bar&gclid=AAA" or QS = "?foo=bar&gclid=AAA&bar=baz" # static files are always cacheable. remove SSL flag and cookie if (req.url ~ "^/(pub/)?(media|static)/.*\.(ico|css|js|jpg|jpeg|png|gif|tiff|bmp|mp3|ogg|svg|swf|woff|woff2|eot|ttf|otf)$") { unset req.http.Https; unset req.http.X-Forwarded-Proto; unset req.http.Cookie; } return (hash); } sub vcl_hash { if (req.http.cookie ~ "X-Magento-Vary=") { hash_data(regsub(req.http.cookie, "^.*?X-Magento-Vary=([^;]+);*.*$", "\1")); } # For multi site configurations to not cache each other's content if (req.http.host) { hash_data(req.http.host); } else { hash_data(server.ip); } # To make sure http users don't see ssl warning if (req.http.X-Forwarded-Proto) { hash_data(req.http.X-Forwarded-Proto); } } sub vcl_backend_response { if (beresp.http.content-type ~ "text") { set beresp.do_esi = true; } if (bereq.url ~ "\.js$" || beresp.http.content-type ~ "text") { set beresp.do_gzip = true; } # cache only successfully responses and 404s if (beresp.status != 200 && beresp.status != 404) { set beresp.ttl = 0s; set beresp.uncacheable = true; return (deliver); } elsif (beresp.http.Cache-Control ~ "private") { set beresp.uncacheable = true; set beresp.ttl = 86400s; return (deliver); } if (beresp.http.X-Magento-Debug) { set beresp.http.X-Magento-Cache-Control = beresp.http.Cache-Control; } # validate if we need to cache it and prevent from setting cookie # images, css and js are cacheable by default so we have to remove cookie also if (beresp.ttl > 0s && (bereq.method == "GET" || bereq.method == "HEAD")) { unset beresp.http.set-cookie; if (bereq.url !~ "\.(ico|css|js|jpg|jpeg|png|gif|tiff|bmp|gz|tgz|bz2|tbz|mp3|ogg|svg|swf|woff|woff2|eot|ttf|otf)(\?|$)") { set beresp.http.Pragma = "no-cache"; set beresp.http.Expires = "-1"; set beresp.http.Cache-Control = "no-store, no-cache, must-revalidate, max-age=0"; set beresp.grace = 1m; } } # If page is not cacheable then bypass varnish for 2 minutes as Hit-For-Pass if (beresp.ttl <= 0s || beresp.http.Surrogate-control ~ "no-store" || (!beresp.http.Surrogate-Control && beresp.http.Vary == "*")) { # Mark as Hit-For-Pass for the next 2 minutes set beresp.ttl = 120s; set beresp.uncacheable = true; } return (deliver); } sub vcl_deliver { if (resp.http.X-Magento-Debug) { if (resp.http.x-varnish ~ " ") { set resp.http.X-Magento-Cache-Debug = "HIT"; } else { set resp.http.X-Magento-Cache-Debug = "MISS"; } } else { unset resp.http.Age; } unset resp.http.X-Magento-Debug; unset resp.http.X-Magento-Tags; unset resp.http.X-Powered-By; unset resp.http.Server; unset resp.http.X-Varnish; unset resp.http.Via; unset resp.http.Link; }

 

 

nginx.conf

 

user wwwrun www;
worker_processes  auto;

events {
    worker_connections  1024;
    use epoll;
}

http {
    include mime.types;
    include /etc/nginx/fastcgi.conf;
    default_type  application/octet-stream;
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    server_tokens off;

    upstream fastcgi_backend {
        server unix:/var/run/php7-fpm.sock;
    }

    #mysite.com NON-SSL
    server {
        listen xxx.xxx.xxx.xxx:8080 default_server;
        server_name mysite.com www.mysite.com;
        set $MAGE_ROOT /srv/www/vhosts/mysite.com/htdocs;
        include magento.nginx_2.conf;
    }

    #mysite.com SSL
    server {
        listen xxx.xxx.xxx.xxx:443 ssl;
        server_name mysite.com  www.mysite.com;
        ssl_certificate      /etc/nginx/ssl.crt/www_mysite_com_2015-04-23-nginx-bundle.crt;
        ssl_certificate_key  /etc/nginx/ssl.key/www.mysite.com_2015-04-23.key;
        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
        ssl_prefer_server_ciphers on;
        ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:ECDHE-RSA-AES128-GCM-SHA256:AES256+EECDH:DHE-RSA-AES128-GCM-SHA256:AES256+EDH:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AE
        ssl_stapling on;
        ssl_stapling_verify on;
        add_header Strict-Transport-Security "max-age=63072000; includeSubdomains; ";
        location / {
            proxy_pass http://127.0.0.1:80;
            proxy_read_timeout 600;
            proxy_set_header X-Real-IP  $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto https;
            proxy_set_header X-Forwarded-Port 443;
            proxy_set_header Host $host;
        }
        set $MAGE_ROOT /srv/www/vhosts/mysite.com/htdocs;
        root $MAGE_ROOT/pub;
    }
}

 

 

1 REPLY

Re: Magento 2.1.6 Performace Problems

Fixed it.   I looked in Cache Management and all caches were disabled.  I don't think that happened any other time up upgraded.  Lesson learned, check that the caches are enabled. lol