Benchmarking Best Practices

[StevenAntonucci]

Based upon a lot of really bad industry attempts to provide meaningful data around Magento performance, I wrote this blog post which covers what is wrong with how web hosting and Magento hosts get benchmarking wrong.

http://blog.litespeedtech.com/2016/09/19/democracy-and-the-benchmark/

Open to discussing best practices so the community can get a good idea of how Magento will perform in the real world.

[AHLeslie]

Read the blog post all the way through and it's definitely given me some stuff I now feel obligated to go tinker with.

Proper testing is important. Proper representation of the results, more so. Which you sort of hit on with the "13x performance" bit. A good number of companies these days like making up strange performance "results" that are slightly based on x test they ran to use as buzzwords, and it's not exclusive to the web hosting / software industries.

It's a good read for sure. Give me a few hours and I'll likely be back to discuss something with you.

May seem a silly thing to say, but thanks!

[StevenAntonucci]

It's important to note that it isn't about who is fastest or best, but instead, using best practices to actually develop a meaningful way to evaluate technology effectively.

The hosting industry is rife with people producing "benchmarks" that have no value whatsoever.  Just like Magento community ranks and stacks developers by skills, the performance community does the same...and we've been doing it much, much longer.  I've just started scratching the surface...hopefully, to catch people's attention and explain why the current solutions aren't useful.

Next up... we're going to open source a practical test harness for Magento customers.