Communications

The Dumb Pipe Paradox ― How open access networks build shareholder value

On more than one occasion, "natural monopoly" arguments have been used to obtain, and/or hold onto, a government granted monopoly,  This is definitely true of access networks where the physical right-of-way to any specific real estate parcel is a limited resource, but the economics of what you put in it have changed over time.  At one time it made sense to consider telephone service a natural monopoly.  And for most of the twentieth century, telephone service was operated that way (think Bell System in the US and government run PTTs in most of the rest of the world).  In the latter 20th century, cable TV became a second such monopoly.  Today, it's increasingly clear that telephony and television are higher layer services, not inherently tied to the access network.  Yet our laws and regulation have barely evolved ― each access network is still regulated as a different vertically integrated monopoly.  And, managers in each business focus on preserving their historic monopoly even as market forces or government regulations force them to also offer Internet access.

So it's interesting to see solid economic analysis showing that access network shareholders would make more money if management was willing to open up their access networks, i.e. become "dumb pipes."

The first such argument I encountered was by Craig Moffett and Amelia Wong of Bernstein Research who wrote an interesting paper The Dumb Pipe Paradox, early in 2006.  The original paper is not on line but I have some quotes here and there are some other comments here.  Craig and Amelia were looking at Cable TV's hybrid fiber-coax networks and concluded that cable companies could make more return on investment if they were in the pure dumb pipes business.

More recently, I reported on a speech given by Benoît Felten of Yankee Group and Fiberevolution in which he argued that new fiber to the home (FTTH) investments could be paid back more rapidly if the FTTH network were open, i.e. offered to competitors at attractive wholesale rates.

Now Benoît has written a detailed report describing his findings.  Although the report is for Yankee Group subscribers only, Benoît is also giving a webinar on the subject next Tuesday and that's open to all.  Register here.

I particularly liked the polite suggestion near the end of Benoît's report:

Recommendations for Incumbents
• Re-examine your economic fundamentals in light of the FTTH business model. It’s irrational to cling to antiquated business practices if new approaches, no matter how disruptive, deliver better shareholder value.

Google's Peering and Caching Strategy

This is a followup on yesterday's post about how little Google/YouTube pays for bandwidth. Google wants to promote peering with ISPs, so they give presentations at ISP meetings.  After reading yesterday's post, Alex Benik of Battery Ventures sent me a link to this presentation given by Google at a 2008 meeting of the Latin America and Carribean Internet Addresses Registry (LACNIC).

As expected, Google peers with as many relevant ISPs as possible.  For the ISP, peering with Google eliminates their upstream costs for traffic to Google.  Since Google represents a substantial volume of traffic for most ISPs, this is a big saving.   As of May 2008, Google was present in 33 public Internet exchanges around the globe, so major ISPs already have connections in places where they can peer with Google.  The minimum qualifications are 5 Mbps of Google traffic and the ability to interconnect using Gigabit Ethernet at one of these 33 major Internet exchange points.

Google Global Cache

What's interesting is Google's caching strategy.  Just as Akamai puts servers in ISP's local facilities, Google is providing a distributed cache for their content.  This available to larger ISPs and allows them to serve Google content directly at the edge of their networks, thus reducing traffic on the ISP's backbone network.  Here's a representative rack that Google provides to the ISP.

YouTube's fine - Analysts don't understand Internet peering

As widely reported, Credit Suisse analysts have estimated Google's YouTube may lose $470M in 2009 and more in the future.  However, their estimates say Google will pay $360M for bandwidth in 2009.  I don't know how Google figures their cost of bandwidth, but anyone who understands anything about Internet transit/peering knows Credit is way off base.

Google does not pay for Internet Transit the way most tier 2/3 ISPs or most content providers must.  The economics are simple.  If you are a Tier 2 ISP, you have to purchase Internet Transit services from a Tier 1 network to handle that customer traffic which goes off your network and for which you cannot make other arrangements.  The most notable 'other arrangement' is peering.  If you have significant traffic to/from another specific network, you and the other network can both save Internet Transit costs by exchanging traffic locally, i.e. peering.  Of course an enormous amount of your traffic is directed to Google.  If you have a presence in any data center where Google has a presence, you would love to peer with Google, as that saves an enormous amount on your payments for upstream Internet Transit.

A similar effect plays out among Tier 1 providers.  If one tier 1 network cuts a special deal with Google, Google routes all their traffic through this provider and suddenly the other tier 1 networks have large asymmetries in their tier 1 peering arrangements.  Either they also cut deals with Google or they have to renegotiate their tier 1 peering arrangements to pay for the traffic asymmetry (something that's highly unlikely!).  Google is the one with leverage here!

I don't know what, if anything, Google pays for bandwidth, but it's not paying $360M for Internet transit.  Sorry Credit Suisse, you better go back to analyzing derivatives, credit swaps and other purely financial plays.

Google does have costs.  They have data centers in many parts of the world and they have a private fiber backbone that interconnects their sites and connects their private network to many, many potential peering points.  Operating their private backbone is a real cost to them and I haven't examined their financial reports to see if there is any way (from public data) of estimating their costs for this private network.

But until someone does this analysis, forget what you've read from Credit Suisse.

Shareholders should demand phone companies open their fiber networks

A few weeks ago, Benoît Felten of Yankee Group and Fiberevolution gave a speech at a New Zealand Commerce Commission event which included a fascinating argument.

He points out that receiving a return on the substantial capital investments that a fiber to the home (FTTH) project requires is much more dependent on takeup rates than on the average revenue per user.  

Single player large scale deployments usually achive only 20%-25% initial adoption after which growth is exceedly slow.  On the other hand, systems which are open to competitors, i.e. with viable wholesale services, attract many more players who market, sell and deliver new services thus dramatically increasing adoption and accelerating the wholesaler's return on investment.

In short, obtaining a regulatory holiday so you can run your new fiber network as a monopoly is actually bad for your shareholders!

Benoît's speech was filmed.  For the full presentation (in four segments of < 10 minutes each on YouTube) see Benoît's post.

If you want just the essence, listen to the first three minutes of this:

Here is an enlarged version of the chart that Benoît is using while he speaks:

Why the Danes text while the US talks

Recently I can by some interesting data on monthly voice and SMS usage in Denmark.  But first let me set the stage.  It's well known that people are price sensitive ― as per minute prices fall, telephony usage goes up.  It's also true that people prefer flat rate pricing as it reduces their mental transaction costs.  This graph (from Andrew Odlyzko) shows what happened to mobile minutes of use when flat rates were introduced in the US.

AT&T launched their Digital One-Rate plan in 1998 offering a block of minutes for one monthly fee with no long distance or roaming complications.  This plan was so popular that all operators were forced to respond, with the results visible in the graph above.

Today US mobile voice usage is approaching 800 minutes per month and that's average.  Leap Wireless and MetroPCS subscribers use 1500-2000 minutes per month.  One wonders how much time people can spend on the phone...

So here's the new (to me) data from Denmark.

The Danish regulator has a wonderful set of statistics available in half-year increments.  Those above are from 2H02 thru 2H07.  Voice minutes are out-going traffic in millions of minutes per period (6 months).  The population of Denmark is 5.5 million (82% are over the age of 14) so this represents ~330 voice minutes and ~225 messages per person per month.

Wireless minutes of use continue to rise, but fixed line minutes are falling faster so total voice minutes are falling.  But total person-to-person communications is still increasing because SMS and E-Mail usage has soared, growing to roughly 40% of all communications.

This is very interesting as flat rate pricing for monthly bundles of SMS messages was introduced in Denmark in 2002.  Meanwhile, Danish mobile voice calls are mostly charged per minute (very typical in Europe) and are expensive compared to the US.

So price matters, but flat rate monthly bundles (rather than per minute or per message charges) is even more important in driving usage.

Monopolists, caught in a time warp

This afternoon, I was in a local shopping area and noticed a new storefront advertising Verizon FiOS service.  I've been a FiOS subscriber for two years now, but I'd never seen a FiOS storefront.  Unfortunately, the store wasn't open.

There was a T-Mobile store 50 feet away and a Verizon Wireless store two blocks away.  They were both open, as was an AT&T store about a half mile away.  The Verizon Wireless store hours were typical of all the mobile phone stores, i.e.
  M-F    10am-8pm
  Sat      9am-8pm
  Sun   10am-6pm

Indeed, these hours are typical of retail stores in the neighborhood.  Weekday openings are at 9:30am or 10am and closing times are 8pm or 9pm.  And all retail stores are open at least a half day on Sunday.

The Verizon FiOS store hours:
  M-F   9am-6pm
  Sat   9am-1pm
  Sun    closed

I remember when retail stores were closed on Sundays.  It was a long time ago.  I even remember, as a child, when some neighborhood stores were only open a half day on Saturday.  That was a very long time ago.

Apparently the FiOS store was opened because Verizon is now competing with Comcast for TV subscribers in our neighborhood.  Comcast is another long term monopolist, so it was interesting to drive (about a mile) to ther retail storefront.  It was also closed on Sunday.  In fact their only store hours were M-F 9-5.  Wow!

I suppose two sources of wired TV services is better than one, but this is another case where it's clear a duopoly provides very little competition (for example, compared to our mobile market with 6+ players).

US Mobile Internet access prospects looking up

This is a very good week for the mobile Internet in the US.  Our best prospect for open mobile Internet access is not legislation or regulation, but having four or more competing networks that are technically able to offer mobile broadband access.

We have three such networks today — Verizon, AT&T and Sprint — but three is not enough to break the walled garden mentality.  What's changed?

1.  T-Mobile USA has launched their first 3G service using the spectrum they won in the 2006 AWS auctions.  For now, it's only New York City, but Reuters reports that T-Mobile plans to launch in 20 to 25 new markets by the end of the year and T-Mobile's stated intention is a full national HSPA network. In 2009, this will be our fourth national 3G network fully capable of multi-Mbps down and multi-hundreds Kbps up.

2.  Clearwire has cut a deal to take over Sprints WiMAX network.  As the Wall Street reports (subscription required) today:

Sprint Nextel and Clearwire are close to announcing a $12 billion joint venture that plans to roll out ultra-fast wireless Internet access for cellphones and laptops in coming years, with the backing of an unlikely alliance of technology and cable companies. Sprint has agreed to merge its wireless broadband unit with Clearwire, a Kirkland, Wash., firm founded by cellphone pioneer Craig McCaw. The new company has raised a total of $3.2 billion in outside financing from several heavyweights -- $1.05 billion from cable provider Comcast, $1 billion from Intel, $550 million from Time Warner Cable and $500 million from Internet giant Google. Smaller cable provider Bright House contributed $100 million. The investments value the new company at more than $12 billion.

This also reduces Sprint's financial exposure and hopefully reduces the likelihood they will be taken over or their network consolidated, at least in the short term.  I've been negative on the prospects for WiMAX in the past, but if anyone can make this go, Craig McCaw is good bet.  So Clearwire represents our fifth national network capable of delivering mobile broadband Internet access.

Assuming all this holds together, we will see affordable flat rate open mobile Internet access in the US by 2010.

Models for Muni WiFi completely neglect technology evolution

Modern travel means interminable waits, but it's a good time for reading. I finally read Wireless Pittsburgh: Sustainability of Possible Models for a Wireless Metropolitan-Area Network by Jon M. Peha, published in February as a working paper of the New America Foundation.

The good news — it’s full of interesting cost estimates and projected subscriber take rates based on specific demographics in Pittsburgh, Minneapolis and Philadelphia.  The paper also examines a range of business models, in detail, from complete monopoly to structural separation (wholesale–retail) to let-the-market-take-care-of-it.

The bad news — all of the models turn out to be extremely sensitive to revenue assumptions, i.e. to the estimates of subscriber adoption and willingness to pay.

The flaws in this study

All of the models compute a net present value based on five years of stable operations, but there is no mention of technology evolution or adoption rates of competing broadband services, i.e. cable and telco (DSL or FiOS) services since this is a US study.  Technology is evolving at a great rate.  You can’t bet on stability.

During the past five years we saw WiFi go from 11 Mbps to widely deployed 54 Mbps systems and bleeding edge (pre-standard 802.11n gear) systems doing well over 100 Mbps.  The last five years also saw costs decline to the point where we see widespread deployment of WiFi by individual consumers, a significant percentage of which are running open WiFi hotspots. 

On a recent drive through three different neighborhoods in Portland Maine, I was interested in looking up real estate information on the web.  On each of a half dozen occasions, I was able to find a open WiFi hotspot within one city block of deciding I wanted to connect.  In January, I was in north New Hampshire and spent two nights at small motel (not part of any chain) in Littleton, NH.  They had no Internet on offer, but a quick check for WiFi signals revealed two within range of our room.  If you don’t like my anecdotal information, look at the WiFi hotspots that have been mapped by Navizon.  It’s very different than five years ago.

No matter how much it simplifies the analysis, you can’t bet on stability.

What might happen with WiFi in the next five years?  The latest WiFi specifications add multiple input, multiple output (MIMO) support, additional modulations and other goodies.  As low cost WiFi routers incorporate these advances we’ll see speeds go over 300 Mbps, but more importantly MIMO technology increases both range and directionality.  This means overlapping systems work better (despite their overlap) and the signals from isolated systems reach further.

If you’re worried today’s open systems will be locked down, then spend your time thinking about schemes like FON which offer more secure ways for consumers to share WiFi bandwidth.

If you’re worried consumer solutions won’t reach the inner city, perhaps someone needs to relook at where WiFi has already been deployed, and then forecast what might happen over the next five years, given the cost of a basic PC is approaching that of a television and the Cable and Telco duopolists both push triple play bundles. 

Don’t short change technology evolution. 

NY Times grossly misreads WEF report

Today's New York Times includes an article by John Markoff entitled "Study Gives High Marks to US Internet."  But either John Markoff is fuzzy about exactly what the Internet is or he didn't actually read the report.  His title is way off base.  He did interview a few people who are quoted in the latter part of the article, so there is some information in the article.  But he's done a major disservice for the many who read only the title or perhaps first paragraph.

The study in question is the Global Information Technology Report recently published by the World Economic Forum.  While the printed report costs money, the summary is on-line and an interactive version gives access to all their data.

This is a report on information technology not specifically on the Internet (just look at the title).  The researchers measured 68 different attributes of information technology, only a few of which directly pertain to the Internet, e.g.

Attribute	US Rank
Internet users (per 100 pop.)	7
Internet access in schools	12
Broadband Internet subscribers	17
Internet bandwidth	19

To his credit, the first section of the report is about "network readiness" and the US is ranked #4, however the study's definition of network readiness includes all sorts of features of the broad business environment, regulatory aspects and computing infrastructure.

Prior to this, I remember John Markoff only for the book Takedown, which I enjoyed.reading even though my sympathies were with Kevin Mitnick.  :-)  So this article is a major disappointment.

Here are the 68 attributes the World Economic Forum study examines:

WiMAX in the US, a limited window of opportunity?

Their first service launch has been delayed, but Sprint Nextel CEO Dan Hesse repeated his vow to blanket the US with a WiMAX network in his talk at CTIA this week.

I sincerely hope he succeeds.  We need as many competing data networks as possible, if we're to see any measure of open mobile Internet access. However, I'm worried.

WiMAX has been quite successful in emerging markets, providing fixed wireless Internet access in countries as diverse as Pakistan, Bulgaria, Nigeria, Georgia, Ethiopia and Georgia (the country!).

The US is another story.  The only significant US deployment is Clearwire's with roughly 400K subscribers, but on a mostly “pre-WiMax” network.  Again, the application is fixed wireless Internet access. 

It’s one thing to use fixed WiMAX to provide Internet access in Pakistan.  It’s another thing to compete for fixed access in the US.  Yes, our telephone & cable duopoly is moving slowly, but they are going after all the more profitable neighborhoods with broadband offerings substantially faster than what fixed WiMAX provides.

What about mobile WiMAX?  Mobile WiMAX is in trials today, using technology and providing performance that the 3GSM community will only see 2–3 years from now.  Sprint clearly hopes to use WiMAX as a springboard past its competitors and past concerns about its declining user base.  Presumably, in the longer term, they hope to converge their dissimilar networks (Sprint EVDO and Nextel iDEN) on mobile WiMAX.  But can mobile WiMAX build a large enough market soon enough? 

Volume drives down cost and cost advantages win in the end — witness Verizon’s announcement that they are jumping ship on Qualcomm’s CDMA evolution in favor of the 3GSM community’s long term evolution (LTE).  Today, GSM networks (GSM/ EDGE/ W-CDMA/ HSPA) have 80% of the world mobile phone market with billions of subscribers and a billion or so handsets manufactured each year.  Right now the entire Sprint-Nextel customer base is ~54 million subscribers.  Perhaps emerging markets will also adopt mobile WiMAX, thus driving up volumes?  Unfortunately emerging markets are even more price sensitive with the high volume application being basic voice telephony plus SMS.  GSM is the lowest cost solution by a wide margin. 

I hope I'm wrong.  I hope mobile WiMAX's performance lead (over LTE) is enough to carry the day.  And, in particular, as I’ve written before, I would very much like to see Sprint succeed, with or without Clearwire, because their WiMAX network would represent yet another source of wireless Internet access.  With four of more networks capable of mobile broadband Internet access, competitive pressures alone should give us what the FCC is currently ignoring, i.e. mobile data plans that are both open and reasonably priced.