Public policy discourse takes place on two tracks: the research track and the advocacy track. The research track consists of expert analysis of the technical, economic, and legal issues while the advocacy track boils down the research into sound bites and echo chamber reverberations.

Much of the advocacy is profoundly disconnected from the research for understandable reasons. The average citizen is not familiar with key net neutrality concepts such as network quality of experience metrics or difference-in-differences models estimated as two-way fixed effects regressions.

So we see advocacy claims about the evils of “fast lanes” and “paid prioritization” applied to markets where all major players already purchase or construct content delivery network services. Advocates also make claims about the effects of regulation on investment based on dubious data and improper analysis.

Christopher Hooton’s Glaring Error

The Internet Association’s chief economist, Christopher Hooton, is at the center of the net neutrality investment effects controversy once again as a result of a paper he submitted to the journal Telecommunications Policy titled “Testing the economics of the net neutrality debate.” Hooton mines a new SEC dataset for information on an expense category called “Capital Expenditures Incurred But Not Yet Paid” that doesn’t mean what he thinks it means.

According to George Ford’s analysis, CEIBNYP measures liabilities rather than investments:

Hooton measured investment using the account Capital Expenditures Incurred But Not Yet Paid, which is not capital expenditures (investment) but an optionally reported credit account that holds the unpaid balance for purchases of fixed assets that occurred in the past. While Hooton offers a detailed and glowing portrayal of his measure investment, his description is entirely fabricated, despite the fact the true nature of the account can be ascertained online or from an accounting handbook.

Ford explained this expense category in detail in a previous note:

The definition of Capital Expenditures Incurred But Not Yet Paid is exactly what you would expect: a “[f]uture cash outflow to pay for purchases of fixed assets that have occurred.” This account is an accrued expense or “an expense incurred but not yet paid; recorded in the accounts by debiting an expense account and crediting a liability account.” It is a supplemental account rarely reported in 10Q or 10K forms.

While Hooton tries to make CEIBNYP do the work of measuring capital investment as it is made, in reality it measures current spending on expenses bought on credit in the past. Thus, it is entirely unsuitable for measuring firms’ reactions to newly-enacted regulations. Yet Hooton appears to believe that CEIBNYP is exactly the opposite of what it is:

Within the data, the author tracks figures for Capital Expenditures Incurred But Not Yet Paid for each company as the measure for telecommunications investment, its dependent variable. The author chose this particular metric since it measures new investment obligations assumed in the current period rather than actualized, previous obligations captured by capital expenditures paid. This paper’s metric offers a previously unexamined and more accurate method for tracking reactionary investment decisions to NN rule changes than current capital expenditures. There are 8913 unique observations across all firms and time periods for capital expenditures incurred with 270 specifically for telecommunications firms. [Hooton, p. 6]

This is amazing. Hooton’s description is entirely contrived and patently incorrect. A simple Google search on “Capital Expenditures Incurred But Not Yet Paid” and a couple of minutes would have spared him the embarrassment.

Hooton’s Secondary Errors are Fatal as Well

Compounding the error of using a retrospective accounting category to measure prospective capital investment, Hooton’s choice of an optional reporting category omits investment by two of the largest Internet Service Providers, AT&T and Verizon. These firms are the two largest investors in the entire US economy.

Hooton also uses data collected at the firm level, conflating broadband spending with spending on completely different lines of business such as cable TV, content, and services such as telephony. This is a common error in investment analysis based on reported data, but one that he doesn’t acknowledge or attempt to correct.

The debate over net neutrality’s impact on investment in Internet access needs to capture decisions by firms to shift investment from highly regulated lines of business to less regulated ones, both in the US and in foreign markets. Hooton simply white-washes this effect by treating firms with some broadband business as if they were exclusively spending on broadband.

In an effort to see what a Hooton-like analysis using real investment data rather than the metric Hooton uses, Ford uses Hooton’s dataset with data from the correct account for capital spending. He finds that investment by telecommunications sector firms did indeed crash after the Title II order was passed. The statistics show a 55% decline in investment after 2015.  In the figure below (borrowed from Ford’s paper), the collapse in investment by telecommunications firms relative to other sectors is clearly illustrated.  I note, however, that Ford admits that he did “not [address] all of Hooton’s many errors,” and that he offered “these results as illustrative.”

Telecom Firm Investment Crashed after Title II Order

Advocates Tout Hooton Without Reading His Paper

The Hooton paper is so horrible that it’s inconceivable that anyone would find it authoritative after bothering to read it.  The data don’t measure what Hooton claims they measure, the data are contaminated by other effects, and the sample of firms isn’t representative of the relevant sector.

To make matters worse, the paper spends most of its word budget meandering around. It offers multiple definitions of net neutrality, commentary on irrelevant papers, charts and graphs on alleged investments that don’t match up with any known empirical patterns, and other trivia.

Ultimately, Hooton takes his inability to find evidence of regulatory investment effects in a dataset that doesn’t measure investment as proof that no such effects exist. The absence of evidence becomes evidence of absence.

Broadcasting Hooton

Nonetheless, the usual suspects were rhapsodic: the Wheeler FCC, the left-wing public interest groups, and the left-leaning tech policy blogs took the paper at face value. Tom Wheeler himself touted Hooton in a New York Times op-ed:

The court decision comes on the heels of a new study from George Washington University demonstrating that the argument used by the F.C.C. to repeal net neutrality — that the regulation hurt investment in broadband infrastructure — was false. The agency acted in true Trumpian style: Truth be damned, just keep telling a colorful story often enough.

I doubt that Wheeler could have said this if he had read Hooton’s study, so the basis for his denouncing the Restoring Internet Freedom order is probably more personal rather than factual. While Hooton does contradict Pai, Hooton’s analysis is ridiculous and Wheeler is on the “truth be damned” side of the debate.

Muted Reaction from Tech Policy Blogs

The reaction to Hooton’s flat-eartherish paper wasn’t as strong as I expected. While it was mentioned in marginal outlets The Daily Dot and Quartz, normally reliable net neutrality advocate Ars Technica didn’t cover it. I suspect the lack of coverage may have something to do with the fact that Dr. Ford decimated Hooton’s study only four days (a long weekend) after it was made public. It was also overshadowed by the release of the DC Circuit’s opinion on challenges to the RIF order.

The Daily Dot simply repeated reactions from Free Press and a blog, along with the study’s conclusion. Quartz ran a longer and much broader story on the investment effects debate generally.

While the Quartz story has some merit – it references facts on both sides of the debate – it uncritically accepted Hooton’s thesis that loan payments represent current investments. Quartz also includes a graphic from the Hooton paper than can’t possibly be correct.This figure alone is a dead giveaway that something is wrong with Hooton’s data. It is unbelievable that a researcher supposedly knowledgeable about telecommunications could produce this graph and not detect a problem with the spikes.

This spiky Hooton graph is not what investment trends ever look like.

Karl Bode’s Torrent

Free-lance blogger Karl Bode embraced the Hooton paper with gusto, touting it in stories on Motherboard and Techdirt days apart. In large part, it was Bode that made the other blogs aware of the study.  Both posts quote the same text from the paper’s introduction (probably as far as Bode went in his reading):

The results of the paper are clear and should be both unsurprising and uncontroversial. The key finding is there were no impacts on telecommunication industry investment from the net neutrality policy changes. Neither the 2010 or 2015 US net neutrality rule changes had any causal impact on telecommunications investment.

There you have it, the assertion that the paper is valid because it says so. Bode falsely claims the Hooton study is “massive” and that no evidence exists on the other side of the debate despite the existence of studies by Wright and Hazlett, Hal Singer, George Ford, Free State Foundation and US Telecom and the analysis by the FCC that over-regulation does suppress investment. The DC Circuit found this body of evidence sufficient to warrant the FCC’s repeal of the Title II order:

Who shall we believe, a free-lance blogger with an ax to grind and no evidence or the second highest court in the land with a bibliography?

Conclusion

In general, arguments like the one Hooton makes are pointless.  It is easy to construct an empirical model that finds nothing.  All you need is bad data or a bad model to do that.  Hooton uses both—a habit of his.  Absence of evidence is not evidence of absence, especially when plenty of evidence of presence is available.  We see the evidence of investment effects in the work cited by the DC Circuit and the FCC.

To the extent that advocates have praised the Hooton study, such praise may be entirely attributed to confirmation bias.  Hooton’s study is “best” and “largest” only because he got the “right answer.” In fact, he got no answer at all (a statistically insignificant result), but none of his supporters are much worried about the details.  Cheerleading such a dubious study undermines credibility—at least for those of us on the round planet.