Communications

Google's Peering and Caching Strategy

This is a followup on yesterday's post about how little Google/YouTube pays for bandwidth. Google wants to promote peering with ISPs, so they give presentations at ISP meetings.  After reading yesterday's post, Alex Benik of Battery Ventures sent me a link to this presentation given by Google at a 2008 meeting of the Latin America and Carribean Internet Addresses Registry (LACNIC).

As expected, Google peers with as many relevant ISPs as possible.  For the ISP, peering with Google eliminates their upstream costs for traffic to Google.  Since Google represents a substantial volume of traffic for most ISPs, this is a big saving.   As of May 2008, Google was present in 33 public Internet exchanges around the globe, so major ISPs already have connections in places where they can peer with Google.  The minimum qualifications are 5 Mbps of Google traffic and the ability to interconnect using Gigabit Ethernet at one of these 33 major Internet exchange points.

Google Global Cache

What's interesting is Google's caching strategy.  Just as Akamai puts servers in ISP's local facilities, Google is providing a distributed cache for their content.  This available to larger ISPs and allows them to serve Google content directly at the edge of their networks, thus reducing traffic on the ISP's backbone network.  Here's a representative rack that Google provides to the ISP.

Can You See Me Now? Video-enabled call centers...

Some weeks ago, CRM magazine asked for an article on video-enabled call centers.  This idea is a bit futuristic for the US market, but such call centers are actually showing up in Asia and the EU, at least experimentally.

Think of calling a help line and being asked to point your video telephony handset at the control panel of the appliance that's causing problems.  That's becoming possible in some Asian and EU markets where most 3G handsets support video telephony and 3G penetration is well over 50% (and much higher in Japan).

In any event the article,

Can You See Me Now? 
Video-enabled call centers will change the company-customer relationship

is up on the DestinationCRM website.

African telecom: corruption mostly limited to publicly run operators

The latest issue (#421) of Balancing Act News Update has an interesting story based on the prosecution (in the US) of former sales agents for ITXC found guilty of bribery in arranging VoIP connectivity contracts.

Six of the seven telcos involved were state owned businesses and in the one with private ownership (Sonatel, the PTT in Senegal that's partially owned by France Telecom), it was France Telecom that got suspicious.  Balancing Act News goes on to point out the problems with bribery:

It will be clear from the summary of the available public documents that this approach to business has a number of pitfalls. The bribed employees in two instances subsequently had to sort out “cost disputes” which were the pretext for asking for more bribes. In one instance, the bribed employee failed to share his gains with the managing director. In the only company with a private shareholder (which also sells wholesale minutes), the bribed employee became isolated under commercial questioning: this provides a strong if not always decisive argument for privatisation.

Based on their conclusion, it appears there is no viable way to do business with the state controlled Telcos in Africa:

From the telco side, what has been known privately for years is now revealed clearly and publicly by the Rwandatel and Sotelma examples. Corruption is systemic and goes right to the top and bribes are expected to be shared with the boss of the organisation and woe betide anyone who tries to keep the money to themselves.

Luckily, the mobile telecom industry in Africa is mostly private and increasingly competitive, with the result that mobile phone adoption is soaring in Africa.  It would be nice to think the PTTs will be cleaned up, but in any event, the benefits of telecom are reaching African citizens via mobile phone services.

One patent justifies Microsoft's interest in Yahoo

I've watched coverage of Microsoft's bid for Yahoo! and the related maneuvering between Google and Yahoo!.  The explanations are not very convincing.  Microsoft doesn't need Yahoo's search technology or their morale-impacted work force.  Yahoo's search market share continues to decline and there's little of strategic relevance in the rest of their business.  What's the attraction?

Usman Latif has the first interesting explanation I've seen, which he explores in depth in a ten page article at Techuser.  The issue he uncovers is US patent 6,269,361 on the bid-for-position paid-for search mechanism that funds Google and indeed all of the Web 2.0 phenomena.  Patent '361 was issued to GoTo.com, later called Overture, in 2001.  Yahoo! acquired Overture in 2003 and used the '361 patent in litigation with Google shortly thereafter.

Usman is uniquely positioned to recognize the importance of this patent as he has followed paid-for search patents for years and has previously written (in 2005) about the strange settlement between Google and Yahoo, reached on the eve of Google's IPO.  Google negotiated a perpetual, royalty-free license to this patent, but it is not irrevocable!  Usman's surmise is that Google's license would be revoked if they sue Yahoo over any other intellectual property.  That's a commonly requested clause in such agreements and, on the eve of their IPO, one that Google likely had to submit to.

If you are at all interested in patents or in the current Microsoft-Yahoo-Google machinations, take a look at Usman's paper.  It makes more sense that all the fluff that's been written in the financial press.

Another analyst under estimates mobile subscriber growth

From a recent press release

Worldwide mobile subscriptions will rise from 3.9 billion in 2008 to 5.6 billion in 2013, according to a new Strategy Analytics report.

Sorry guys, you're forecast has fallen into the same trap as almost all the five year forecasts I've seen since 1995.  About the only credible analyst's statement in this area is one by Mark Newman, head of research at Informa, who said (in 2007):

The mobile industry has constantly outperformed even the most optimistic forecasts for subscriber growth.

The problem is analysts' forecasts are consistently pessimistic.  Consider Strategy Analytics' numbers.  They project 1.7 billion new subscribers in 5 years (60 months).  By every measurement I've seen, mobile subscription growth has been over 50M per month at least since 2004-2005.  For example, 2 billion subscriptions in September 2005 and 3.3 billion subscriptions in November 2007 comes out at 52 million new subscriptions per month.  If you believe Strategy Analytics' forecast of 3.9 billion by the end of 2008, that's 54 million more per month for the balance of 2008.  If we merely sustain current rates, we will hit Strategy Analytics' December 2013 forecast in just 31 months, i.e. in July 2011.  So they expect current growth rates to decline rather substantially in coming years. 

Why would they think this?  I haven't spoken with anyone at Strategy Analytics, but in discussions with other analysts over the past ten+ years, a frequent answer is "current growth rates will have to slow as we're running out of people who can afford mobile phone service."  Wrong!  This is the view of someone who doesn't understand Moore's law.  Whether it's transistor density or wireless performance, innovation drives exponential improvements in price-performance.  Per-capita GDP is improving slowly, but the cost of mobile phone infrastructure and mobile handsets is dropping substantially every year.  Today, those at the bottom of the pyramid are in a position to at least use a phone and increasingly to acquire their own mobile phone.

In addition to innovation ― at work since the introduction of mobile services ― political obstacles are falling in many countries.  Since 2000 it's become clear, mobile phone adoption brings significant economic advantages, and the best way to get mobile phone adoption is to attract multiple competitive mobile phone operators to your country.  Increasingly, even the most backward dictators are realizing there's more money to be made taxing multiple competitive mobile phone services than in keeping all the profits from one government phone company.  So country after country are encouraging competitive mobile phone operators and allowing substantial foreign investment.  This political trend serves to increase mobile phone adoption rates.

Of course, adoption rates will saturate at some point.  When might that be?  The best evidence we have is for 2G and 2.5G services in developed countries.  It appears things slow down a bit around 120 subscriptions per 100 people.  Of course this may change with 3G.  It's too early to tell.

Technology adoption generally follow an S-curve:

If saturation is at 120 or above, and we're currently around 50, we have at least another five years of rapid growth.  Perhaps a five year forecast made in 2013 will be justified in projecting a decrease in adoption rates.  But by then we'll also understand the impact of 3G.  Who knows? 3G may drive things even faster. 

Or perhaps by 2013, I will have a single mobile data subscription for a gateway device on my body, that in turn provides connectivity for all my other devices.

In any event, I expect 5.6 billion mobile phone subscriptions by the summer of 2011 and more than 6 billion by mid 2012.

Mobile video-on-demand Yes! Mobile broadcast TV not so hot

I have an article, Going Mobile (TV), that's recently been published by MobileIN, a wireless and mobile information site.  In it I basically argue that major investments in mobile TV broadcast capability are less likely to pay off than investments mobile video-on-demand.

The biggest trend in commercial television viewing is personal video recorders like TiVo.  People want to watch TV content when they want, not when broadcasters schedule it.  The only exception is major sports events (the Superbowl or World Cup matches).  Even the evening news is frequently rescheduled for later in the evening.

The second relevant trend is growth in YouTube and similar web-based video content.  Broadcast TV went from 2-3 channels in the 1950s to hundreds of channels on a typical cable system today.  But consumers are also interested in the long tail of millions of videos that can only be served over the Internet today and, potentially, over the mobile Internet in the future.

Finally, survey's of early adopters of mobile video show music videos, movie trailers, weather, sports action clips, comedy videos, cartoons and amateur video shorts – typically a few minutes long at most – are the most popular content. In addition, it appears 85 percent of mobile video viewers watched viral videos (content sent or pointed out by others) rather than content they found themselves.

All and all, mobile consumers are looking for video -on-demand, not pre-scheduled broadcast TV.

So what's the logic for massive investments in spectrum, followed by even more money in new wireless infrastructure, followed by the need to sell everyone new handsets that can receive the new broadcast mobile TV channels?

The end of Yahoo, and the end of Microsoft

If you are not familiar with Umair Haque &/or don't regularly read his Bubble Generation blog, read Umair's views on this potential merger. 

He begins with "Yahoo + Microsoft isn't just a mistake - it's a double suicide."

It gets better. 

And his argument is independent of the culture clash between the these two companies (a subject eloquently raised by his first commenter).

How tiered Internet pricing could actually facilitate P2P

Time Warner Cable's planned experiment with tiered charging for Internet access has generated a flurry of coverage in the blogsphere, but no new insights (at least that I've seen).

The primary problem ISP's complain about is that 5% of their customers use 90% of the available bandwidth and when they examine this traffic, it's mostly peer-to-peer file sharing.  A reasonable question is how to allow as much of this traffic as possible without increasing an ISP's variable costs or slowing down their other users.

This may not be as difficult as it appears.   Indeed if Internet access was as competitive as mobile telephony, we might already have seen what I'm about to propose — a combination of bundled pricing equivalent to mobile's "free nights and weekends" and "free on-net calls" with a way to facilitate P2P traffic that leverages exactly these "free" periods.

An ISP's costs

ISPs have some costs which are relatively fixed and others that are tied to usage.  A network is a relatively fixed cost and when it's not full, the incremental cost of adding traffic is zero!  This is the reason mobile operators give away free nights and weekend.  They've built their mobile network for the peak daytime traffic, so it costs them nothing to run promotions that add incremental traffic at off hours.  Peak hours and off hours may be different for an ISP, but the concept is the same. When a data pipe is lightly loaded the ISP's cost of adding incremental traffic is zero.

On the other hand, some ISP costs are usage based, for example "IP Transit" or more properly, Internet Transit.  This is the ISP's upstream cost to send and receive traffic to/from the rest of the Internet.  However, even here, usage-based costs occur at heavy usage.  Light usage periods don't save money.  To understand what's happening, it's worth a digression on Internet Transit.

Internet Transit

Internet access is monopoly or duopoly or a heavily regulated industry.  The middle mile connections from the local network to the Internet backbone may or may not be competitive depending on where you are.  But the Internet backbone itself is extremely competitive.  If you can get to a major Internet Exchange Point in the US or Europe, there are many providers offering extremely competitive rates for Internet Transit.  Typically these services are priced on a megabit per second per month basis (Mbit/s/Month) with lower rates for higher volume commitments.  The other key idea is that charges are based on the 95th percentile of all the five minute data rate samples taken during the month.  So an ISP can have a few bursts above their typical rate, as long as they represent less than 5% of the sampled intervals.

But this also means there is no extra cost to run at or near the typical rate at all times.

Local traffic

Even more important, if file sharing is done with other computers on the same ISP's network, then there is no need to pay for Internet Transit at all.  The question is how to figure out which potential peers are "on-net" and which are "off-net."

Sending signals to P2P software

Most P2P file sharing software has relatively little knowledge of locality.  Some P2P software practices "prefix awareness," for example, Joost gives preference to peers in the same /24 IP address block when they are available.  But if a major operator provided an automatic way for P2P client software to determine whether a prospective peer's IP address was currently reachable "for free", it seems likely the file sharing community would leap on it, and if there's money to be saved, active file sharers would download the new clients immediately.

A standard way to present such information might be via an extension to the XML-based response codes in one of the whois information exchange proposals, e.g. from ICANN or from APNIC.  Also, while what I'm proposing might start as a pricing plan rather like a mobile operator's "free nights and weekends" and "free on-net calling," it's not hard to see extensions where an ISP could offer dynamic access to underused capacity to those programs that were prepared regularly interrogate an ISP's server and use just the advertised off-hours capacity.

In closing

People liked fixed price deals.  Unlimited is great, but there's plenty of experience with bundles of minutes and the idea of data bundles has already showed up in 3G mobile data plans.  The combination of several tiered data bundle prices with the availability of "free" connectivity for "on-net" peers and during off peak intervals is likely to appeal to file sharers and produce better results for both the sponsoring ISPs and file sharers alike.

Global mobile network operators subscriber data

There's an interesting free source of market data that I've been using for the past two years.  Until two hours ago, I hadn't given this much thought, but when I mentioned it to some colleagues, their reaction was:  Wow!  This is really cool. 

So I thought I'd share it more widely. It's Wikipedia's List of Mobile Network Operators.

 

There's quite a bit of information, by country and by operator, for example:

I've been tracking mobile subscriber growth for more than ten years, in part by purchasing ITU data every other year or so; in part by getting copies of any data than any NMS business unit purchases from traditional market analysts and in part by capturing anything that's published in press releases or other teasers from analyst firms.

Wikipedia doesn't have all data for all operators, but they have almost everything and it's very credible.  They also cover countries that I have never seen in traditional analyst reports, for example:

Somolia is famous for having no functioning central government, no spectrum regulation and no telecom regulations, but none-the-less having the lowest international calling rates in Africa.

Ringback Tones Adoption Rates — Culture, Biz Model or Marketing Savvy?

Ringback tone services have been wildly successful in many parts of Asia and they are emerging as a major money maker, not to mention a major music distribution channel, in many other countries, but not in the US or the EU.

Nearly 55% of Korean subscribers have ringback tone service.  China Mobile has close to 50% adoption.  Yet, as of September, adoption rates in most of Europe were below 10% (the UK was at 2%) and US adoption was only 5%.  Since our LiveWire Mobile division provides white label managed ringback tone services and sells ringback tone service platforms, I'm obviously interested in understanding more.  And, with 30+ operators on six continents using LiveWire Mobile platforms or services, I have some access to comparative data.

A late start is only part of the story.  Korea went from 0 to 30% penetration in just nine months (April to December, 2002).  China took a bit longer as the service only became available in stages across China Mobile's 31 separate, relatively autonomous, provincial units.  That roll out took from mid-2003 to early 2004.  But by the end of 2004, China Mobile had 30 million ringback tone subscribers (then 69M by 12/05 and 144M by 8/06).  And yet after ~2 years of service, the US has only 12M subscribers or 5% penetration.

One clear difference in Korea and China, versus the US and EU, is the number of independent marketing organizations directly or indirectly promoting the new service.  I remember visiting Seoul Korea in January 2003 and having a billboard pointed out to me where a Korean rock star was advertising his latest hit and how you could make it your color ring tone. I was told there were more than 20 independent organizations providing content for SKT's Color Ring service at that time.  More significantly, revenue splits were such that these independent organizations found it worth while directly promoting their content to mobile subscribers.  That means there were 20+ marketing teams with independent activities that all served to increase subscriber awareness.

Compare this with the typical operator in the US or EU where the only source of information or content is the operator's portal and the only service promotion activities are those of the operator.  That means there is one product manager focused on ringback tones, they have a limited budget and they must compete with their associates for position on the operator's WAP deck.  Is it any wonder awareness is low?

Of course other factors also contribute to service success.  Our LiveWire Mobile team has accumulated a set of best practices based on discussions and consulting work with operators around the world.  They have a formal service offer of course, but I'll endeavor to summarize some of their learning in a future blog post.