The awesome TPRC Conference starts on Friday this week, at the Scalia Law School at George Mason U. in Arlington. This conference brings academics and policy scholars from around the world together with policy makers who want to make informed decisions. I’ve been going for seven or eight years and have written four papers for it.

This year, I’m presenting a paper on performance measurement and policy making that was inspired by Noam Chomsky, the MIT philosopher. In one of Chomsky’s interviews on the American imperialist system, he observed that the Internet is slowing down.

This shocked me because I know broadband is speeding up: the average download has increased about 35% a year for the last eight years. Combining that figure with Tom Wheeler’s “virtuous cycle hypothesis” would lead one to believe that the web must be getting faster too.

Virtue Theory

The virtuous cycle hypothesis says apps make broadband faster and better and then broadband makes apps faster and better. As with any hypothesis, we have to test it before it graduates to the status of theory.

Fortunately, the hypothesis is already stated in the form of a prediction so all we have to do is take some measurements over time to see if the idea holds up. The Wheeler FCC was quick to proclaim the 2010 Open Internet Order a success for protecting the virtuous cycle:

Four years ago, the Commission adopted open Internet rules to protect and promote the “virtuous cycle” that drives innovation and investment on the Internet—both at the “edges” of the network, as well as in the network itself. In the years that those rules were in place, significant investment and groundbreaking innovation continued to define the broadband marketplace. For example, according to US Telecom, broadband providers invested $212 billion in the three years following adoption of the rules—from 2011 to 2013—more than in any three year period since 2002. 3. Likewise, innovation at the edge moves forward unabated. For example, 2010 was the first year that the majority of Netflix customers received their video content via online streaming rather than via DVDs in red envelopes.

Today, Netflix sends the most peak downstream traffic in North America of any company. Other innovative service providers have experienced extraordinary growth— Etsy reports that it has grown from $314 million in merchandise sales in 2010 to $1.35 billion in merchandise sales in 2013. And, just as importantly, new kinds of innovative businesses are busy being born. In the video space alone, in just the last sixth months, CBS and HBO have announced new plans for streaming their content free of cable subscriptions; DISH has launched a new package of channels that includes ESPN, and Sony is not far behind; and Discovery Communications founder John Hendricks has announced a new over-the-top service providing bandwidth-intensive programming. This year, Amazon took home two Golden Globes for its new series “Transparent.”[¶ 2-3]

The offered proof is heavy on streaming and almost completely devoid of measurement, however. While the claims can be measured, the order doesn’t bother to say how many startups were born or what the connection is between the success of “Transparent” and broadband quality. And the remark on Etsy was premature.

What the Numbers Say

Startup formation and exits are easy to count, but it’s hard to distinguish the effects of policy from general economic factors. So I’ll leave that for others to analyze. The FCC notes investment figures for a three year period, but again this is something for economists to analyze (as they have, many times over.)

I want to do something simpler and harder to fudge: compare the rate of improvement in broadband speed with the rate of improvement in web page load time. It seems that these two numbers should be roughly parallel if each influences the other.

Depending on whose numbers you like, broadband has been increasing somewhere between 29 – 43% per year since 2010.

Rate of Improvement in US Broadband Speed, 2010-2016

Akamai’s average is 29%, Ookla’s is 40%, and the FCC’s is 43%. So the average of the averages is 37%.

The Pokey Web

According to Akamai, the web is basically flatline.

Web Page Load Time

On fixed networks, web page load time is about 7% faster every year, and basically flat since 2014 at 3% a year. Web pages actually took a tenth of a second more to load in 2016 than they did in 2015. So this particular comparison doesn’t support the virtuous cycle theory.

It’s peculiar that the web has made no progress since speedy web pages translate into revenue for site operators. Akamai is placing a lot of emphasis on web page load time in the e-commerce market because there’s a direct line between fast page loads and success.

Preaching to the Converted

When you visit a retail site, the operator wants you to sign up and make a purchase; these activities are called conversions. While both are ideal, the retailer is satisfied with either because conversions lead to revenue either now or in the future.

You can get a good feel for the relationship of speed to conversions from this chart that correlates them with device type. The faster the device, the more conversions.

Akamai Conversion Rates by Device Type

The lesson here is that Quality of Experience – the experience of satisfaction – is determined by a whole constellation of factors, only one of which is network speed. Device speed, page speed, and site design also play into the consumer’s sense of satisfaction.

When all of these factors are equal except for device speed, end-user devices make a huge impact, much greater than variations in broadband speed. And the FCC’s data says pages don’t load faster at broadband speeds greater than 15 Mbps.

Conclusions and Takeaways

It appears that the strong emphasis regulators in the US and other developed nations have placed on arbitrary increases in broadband speed doesn’t correlate with innovation, broadband adoption, or broadband satisfaction. The data say it’s important to have 15 Mbps (or slightly more) download speed, but once you do speed means less and less.

FCC Data Show No Improvement in Web Page Load Time at Broadband Speeds Above 15 Mbps

Bringing unconnected people to the Internet probably depends on providing them with satisfactory experiences. That means a broad range of engaging content and interactions, and it also means fast web sites, fast computers, and well-organized information.

The emphasis on raw download speed shows we’re missing out on some of outcomes that might come about if other dimensions of the Internet game were more healthy. Pages are slow to load because it takes so long to populate them with relevant ads.

Sites can’t do away with advertising even if they have alternate sources of income such as subscription fees and retail purchases. The share of ad revenue that goes to site operators is low, which means it takes more ads for a web company to be successful.

If speeding up the web means selling fewer ads, we’re never going to have a speedy web.