Netflix and Comcast Declare Peace
Netflix and Comcast announced today that they’ve come to terms on an interconnection agreement that they’ve been working out since their CEOs and technical teams met in Las Vegas during the Consumer Electronics Show. The details of the deal aren’t yet known, but in general it means that Comcast and Netflix will connect their networks as peers, which cuts middlemen Cogent, Level 3, and Tata out of the path between their networks. Many are speculating that Netflix is paying Comcast for direct connection, but most believe the fee is less than Comcast has been paying the transit providers. Officially, Netflix and Comcast aren’t discussing terms.
I use both Netflix and Comcast myself, and I’ve seen a more direct path to the Netflix servers for the last couple of weeks. Here’s what I see at the boundary between Comcast and Netflix when I run the “traceroute” command on my Macbook at home, disregarding private network hops and omitting the time stamps:
This says the Comcast residential network (ibone.comcast.net) connects to the Comcast Business Services network (18.104.22.168), which connects to the Netflix CDN (ix.nflxvideo.net) at the Equinix SV1 data center at 11 Great Oaks Blvd. in San Jose. The two firms most likely buy cross-connects from Equinix over which their traffic runs. This is not the only point of interconnection between Comcast and Netflix, just the one that happens to be most relevant to me since I live in the Silicon Valley area.
The path from my house to the Equinix data center goes through Pleasanton and San Francisco before it gets to San Jose, so the packets that I get from Netflix travel 100 miles across cables owned and maintained by Comcast. They travel a few feet from a Netflix server inside the Equinix SV1 building before connecting to the Comcast network through an Ethernet switch. So that’s the whole path from end-to-end. Most people are considerably farther away from Netflix than I am.
Both firms pay Equinix for ports on the Ethernet switch they share, which almost nobody finds controversial, but there are some raised eyebrows over the speculation that Netflix has to pay Comcast for carrying the packets as far as they do.
While very few people find it controversial that Netflix has to pay Cogent, Level 3, or Tata to carry their packets to the hundreds of ISPs in the US with who serve limited areas, there is a fairly widespread belief that Comcast should provide packet carriage for free to anyone who can directly connect to their network. A corollary to the belief about free carriage is a belief that this deal changes the Internet or makes net neutrality irrelevant.
These kinds of arrangements – known as “paid peering” – have been going on since the days of AOL, but have become more prominent since the rise of video streaming. I wrote a post on GigaOm about them more that four years ago, and Akamai has been connecting its Content Delivery Network to ISPs directly for as long as it has been in business (since 1998.)
The bottom line to American consumers, investors, and entrepreneurs is whether paid peering is harmful to their long-term interests, not whether it violates any obscure and poorly reasoned principles such as net neutrality. Does the existence of paid peering agreements between large and successful firms such as Netflix (with its 40 million customers) and Comcast (with 20 million going on 30 million) harm the chances of start-up service providers to get into the video service game or related businesses?
The usual suspects are certain that paid peering is harmful: Public Knowledge VP Harold Feld characterizes it as a “back end connection” that provides Netflix with “higher last mile speeds”, and his Free Press counterpart accuses Comcast of “abusing its gatekeeper status.” TechCrunch’s Greg Ferenstein supposes the deal “may be legally outside of the traditional net neutrality rules” despite the fact that it’s clearly outside the scope of said rules. Dan Rayburn, one of the small number of bloggers who actually understands peering, streaming, and CDNs, fumes at Ferenstein:
May be? Are they serious? Commercial interconnect relationships, also referred to as paid peering agreements, have been around since the Internet started, and it’s how the Internet works. Commercial interconnect deals have NOTHING TO DO WITH NET NEUTRALITY. Implying otherwise shows a complete lack of regard in understanding how traffic is and has been exchanged across networks for the past twenty years.
Traffic delivery across the Internet – from service providers such as Netflix and Google to ISPs such as Comcast and AT&T – follows a tiered model. When you start a new business, you buy services from an ISP pretty much the same way you do for a home connection, only you get higher capacity by paying for a commercial grade of service. You also rent space in one or more hosting centers for your servers (or pay for an equivalent cloud service, such as Amazon AWS.)
When you get more customers and need to send more traffic than you can handle as a startup, you typically outsource to a CDN like Akamai, as Apple and Major League Baseball do. When you get more even more customers and need to send even more traffic, you build your own CDN as Google and Netflix have done. Netflix is simply moving up the ladder from outsourcing to Akamai and Limelight for CDN services to running their own CDN, which means they have to make the same business deals that Akamai and Limelight have had to make. This isn’t changing the Internet as much as it’s following a well-worn path from startup to established Internet hyper-giant. Apple is building its own CDN these days as well.
The best clue that the Netflix/Comcast agreement isn’t a bad deal is the fact that neither party is complaining. If you knew the people who work for Netflix as well as I do you, you’d realize how remarkable this is. Comcast is also the kind of company that has long been willing to sign long-term deals at terms that are favorable to its business partners in order to achieve peace and stability, which is only possible if nobody is getting hurt. It seems that both firms regard this deal as beneficial, which is a pretty strong clue that it’s good for ordinary people as well.
Running companies as large as Netflix and Comcast requires a high degree of collaboration at the technical level. Comcast probably interconnects to the rest of the Internet at hundreds, if not thousands of places, and Netflix will, when its CDN is fully deployed, connect at dozens if not hundreds. As Sam Gustin explains:
Prior to Sunday’s agreement, Netflix was connecting with Comcast’s network through intermediaries at five or fewer locations around the country, according to a person familiar with the deal. Now, Netflix will be able to connect directly to Comcast’s network at dozens of locations, the person says, which should improve performance for consumers.
So they needed to interconnect where their pipes are fattest and most numerous, close to both server resources and eyeballs. There’s no better way to ensure this happens than by making a formal deal and paying some money. And make no mistake, no matter how much Netflix is paying Comcast to deliver its TV shows, it will still cost Comcast much, much more to deliver them from their points of interconnection to its end users. And I doubt Comcast’s return on assets, share price growth, and annual growth rate will match those of Netflix any time soon.
So take it easy, it’s not the end of the Internet or even a new chapter: this really is business as usual, the kind of arrangement that Internet firms have to make when traffic becomes concentrated in a few hands. Given that Netflix sends more than 30 percent of the Internet’s prime time traffic load in North America, it’s actually long overdue.