The latest addition of the New America Foundation’s “Cost of Connectivity” report is successfully gaining attention from bloggers and journalists who accept its claims at face value. Some of the stories about the report have appeared in major publications, and nearly all the sources that generally deal with Internet policy have covered it:

New York Times: Why the U.S. Has Fallen Behind in Internet Speed and Affordability

Los Angeles Times: The American way of broadband: slow

Daily Dot: America has terrible, horrible, no good, very bad Internet service

Vox Media: The basic truth about internet access that cable & phone companies don’t want you to know

The Hill: US Internet speed, prices lag behind other nations’

Huffington Post: America Pays More For Internet, Gets Slower Speeds, Than Other Countries

Telecom TV: US broadband: Overpriced, Underpowered, and (for once) Over There

Roll Call: Fastest Broadband For Your Buck? Try Seoul, Hong Kong, Tokyo or Paris

247 Wall St: South Korea Has World’s Fastest Internet

CNN: Americans pay more for slower Internet

So here we have ten stories that accept the NAF’s claims without the slightest bit of skepticism or critical examination. Each of them repeats the tale that American broadband is slower than broadband in the rest of the world without any need to verify or validate it by examining additional data or asking hard questions. The closest any of these stories gets to asking questions is the piece in the New York Times by Claire Cain Miller who at least asked one critic (me) for an alternate point of view, even though she used the smallest possible part of my interview.

Journalists used to require two sources to confirm a story, and stories like this one would be more informative if that were still the case. Two of the writers did consult a second source on the NAF speed claims, but didn’t do so in a highly responsible way. CNN’s James O’Toole compared the advertised speed claims in the NAF report to the “average connection speed” measured by Akamai in the US a year ago:

For comparison’s sake, the average U.S. connection speed stood at 9.8 megabits per second as of late last year, according to Akamai Technologies.

The Hill’s Mario Trujillo made the same comparison:

Another study earlier this year found that the United States had an average connection speed of 10.5 Mbps.

I’ve noted that reporters don’t understand what the Akamai State of the Internet report measures, and here we see that claim in action. New America surveyed some ads and ranked nations according to the speeds offered in a very select group of ads in 24 cities and towns. Akamai’s “average connection speed” measures download speeds seen within complex interactions between web browsers and servers that aren’t representative of network speeds; “average peak connection speed” is more appropriate for network measurement. This is explained in Akamai’s reports:

In addition to providing insight into high broadband and broadband adoption levels, the report also includes data on average and average peak connection speeds — the latter provides insight into the peak speeds that users can likely expect from their Internet connections. (See the blog post at https://blogs.akamai.com/2013/04/clarifying-stateof- the-internet-report-metrics.html for more information on how these metrics are calculated.)  (Akamai State of the Internet, 2Q 2014, page 19)

So you wouldn’t compare Akamai’s average connection speed to the advertised rate of a broadband connection if you had read the Akamai report; that’s apples and oranges. You also wouldn’t compare the highest speed offered in an advertising survey to the performance of the connections that people actually buy; very few people buy the highest speed available. But what would you find if you compared the Akamai average connection speeds of all the countries New America surveyed?

It turns out you can’t do that because NAF doesn’t survey whole countries, only carefully selected international cities and US cities and towns. Despite the narrow basis of their survey, they’re quite happy making sweeping generalization about whole nations, however; the comparisons between San Francisco, Bristol, VA and Chattanooga, TN with Seoul and Hong Kong are offered up as indictments of American broadband.

It’s difficult to compare Chattanooga with Hong Kong, and it’s even more difficult to compare the entire US to specific nations in Europe and Asia that are scarcely larger than America’s San Bernardino County, California. So to make an apples-to-apples comparison between American broadband speeds and those in the rest of the world using Akamai’s data, let’s compare US states with the overseas nations and administrative regions (Hong Kong is part of China, not a nation) and see where the chips fall. When you do that, you get this:

10 fastest nations and 10 fastest states. Source: Akamai State of the Internet, Q2 2014; www.akamai.com/stateoftheinternet.

This isn’t very shocking, is it? Five of the 10 fastest regions are American states, six of the 11 fastest regions are states, and 10 of the 17 fastest regions are American states. I’m not inventing data here, I’m simply extracting data from a report cited by two the analysts who accept the New America’s work as perfectly valid.

I’m happy for analysts to read and cite Akamai, I just wish they’d do it in a more consistent and thorough way. And yes, it is true that Akamai’s measurement of average connection speed for the entire US is only 11.4 Mbps in the most recent report. That puts us in 14th place, between Norway and Belgium, on a global basis. Given that Akamai measures 200 nations and regions, most of which are smaller than the US and some of which are richer, 14th place is not too shabby; we’ve been as low as 22nd in the past.

The point I’m making is simple: you can’t judge a nation’s Internet service on the basis of an advertising survey, and both New America and the journalists who uncritically accept their claims are being less than forthright when they claim otherwise.

The second claim that New America makes concerns prices, and I’ll deal with that in a subsequent post. Hint: they’re misleading on this point as well, but in a more subtle way.

While you’re waiting, take a look at my paper on broadband quality in the G7, “G7 Broadband Dynamics: How policy affects broadband quality in powerhouse nations“. The interactions between demographics, geography, policy, and quality are fascinating.