Web Performance Stats are Overrated

Note: A more polite version of this post originally appeared on Instart Logic’s blog

Who cares about performance?  Your customers, that’s who!  Speeding up your site by several seconds, or even by hundreds of milliseconds, will make your customers happy. Well, at least they’ll be less irritated.  That’s pure logic. I am not citing statistics. I am using common sense.

For example, a 50% decrease in file size — cutting bloat in half — has a greater impact on someone 2G mobile than on a T1 line. Common sense dictates the impact is greater on a 5Mb site than a 5Kb one, no matter the network speed as performance gains achieved by halving a 5Kb download may not even have a noticeable effect. Similarly, improving a 12s page load to 11s doesn’t have the same impact as improving a site from 3s down to 2s. The impact of a one second improvement depends on the original experience. This is obvious. It’s common sense.

Conclusions based on common sense are often accurate, but aren’t considered factual like conclusions based on statistics.  Statistics are often used to demonstrate something as fact, but a statistic is really only a fact about a piece of data. Mark Twain is credited with saying “Facts are stubborn things, but statistics are pliable.”

We discuss web performance statistics as if the web were a monolith, as if all web applications were uniform, like performance improvement effects are linear. They may not be represented by a simple exponential equation, but performance isn’t a simple science, and metric interpretation shouldn’t be assumed to be linear, or even accurate.

The statistic many web speeders quote “a one second delay in web page responsiveness leads to a 7% decrease in conversions, an 11% drop in pageviews, and a 16% decrease in customer satisfaction” is kind of bullshit today. This quote comes from a 2008 study: a study that came out a few months after the iPhone SDK was released and Android Market came into being, before either proliferated. When that study was conducted, the iPad and Android tablet were a few years off. As web speeders, we can’t quote a performance study that basically predates mobile. Yet, that’s what we’re doing.

While the customer satisfaction statistics might be BS, the conclusions are not. The longer the site takes to load, the less happy your customers will be, the more users will abandon your site, the lower your conversion rates, and the less money you’ll make. It’s not even that your customers will be unhappy — it’s that they may not be your customers anymore. That’s common sense, no matter what the actual percentages are.

Definitely give yourself a performance budget and test your site. Test your application throughout the development process. Optimize and right size your images, setting dimensions. Reduce DNS lookups. Reduce bloat, minimizing request size, GZipping all requests. Make fewer HTTP requests or serve content over HTTP/2, caching what you can. Basically, follow as many performance recommendations as you can.

Improve the performance of all of your content, not just your home page. Focus your energies on the critical path of your site. It’s obviously less important to optimize some pages, say the “libraries we use” page linked to from an “about us” page linked to from a political candidate’s website. But do realize users often enter your site thru a side door: for example, a product page, with the shopping cart experience being a necessity for all sales for an online store. In the case of eCommerce, optimizing your homepage might create the most “savings”, but will do little to improve user experience for your actual potential customers.

And never forget: improving the download speed is not enough: improving your site by one second delay won’t improve customer satisfaction if, once loaded, they’re met with an unresponsive UI.

Or maybe it will.

Let me go back to 2008 to test that out.

Published by

Estelle Weyl

My name is Estelle Weyl. I an open web evangelist and community engineer. I'm a consulting web developer, writing technical books with O'Reilly, running frontend workshops, and speaking about web development, performance, and other fun stuff all over the world. If you have any recommendations on topics for me to hit, please let me know via @estellevw.