The Internet Trap

How the Digital Economy Builds Monopolies and Undermines Democracy

PRINCETON UNIVERSITY PRESS

Contents

Rethinking the Attention Economy

The American Beauty rose can be produced in its splendor and fragrance only by sacrificing the early buds which grow up around it.

— John D. Rockefeller, Jr. on trusts, quoted in Ida Tarbell's

The History of the Standard Oil Company

In early 2000, Google conducted one of its first online experiments. The result was a disaster.

Google's mistake started with a simple question: How many results should it return per search query? The young company had always given users ten results, because that was what previous leader AltaVista had done. Competing search engines like Yahoo! returned twenty, though, and focus groups suggested that users preferred more. Google's researchers decided to try a real world test, splitting off groups of users to receive twenty, twenty-five, or thirty results instead of ten. But when they checked a month later, they found — to their shock — that more results had produced huge drops in traffic. Searches in the thirty-result group had fallen by more than 20 percent, and tens of thousands of users had abandoned Google altogether.

The researchers rushed to figure out what had gone wrong. Were users overwhelmed by more results? The data showed no evidence of this. Were users just clicking the "next" button less? Few users clicked on the next button to begin with, so this effect was tiny.

Google eventually traced the traffic drop to a surprising source: It took a fraction of a second longer to return more results. The control group waited 0.4 seconds on average, while those in the twenty-five-result group waited 0.9 seconds. Over a day or two this added lag meant little. But as the weeks wore on, the effects of that extra half-second multiplied. People visited Google less often, and performed fewer searches when they did visit. Even when the experiment ended, slowed-down users did not come back immediately. Their usage started to increase again, but from the new, lower baseline. There are several morals to the story of Google's early experiments, which this book will unpack at length. But the most important lesson is about how to understand online advantage.

Digital survival depends on stickiness — firms' ability to attract users, to get them to stay longer, and to make them return again and again. Stickiness is like a constantly compounding internet interest rate, in which a small early edge in growth creates a huge long-term gap. Differences in stickiness do not add up, they multiply together.

Google's ascent from upstart to the world's most valuable company came from learning this lesson. Google spent billions to make its site faster — but it also did much, much more. It reinvented itself not as a search engine but as a bundle of the stickiest online activities: email, video, maps, mobile, even office software. And in its pursuit of an ever-larger slice of users' attention Google rebuilt the fundamental infrastructure of the internet: new data centers larger than any before, new fiber optic cables, new ways of running rich applications over the web and a speedy new browser to match, new forms of artificial intelligence running on new types of computer chips, even a new mobile operating system that now runs on two billion active smartphones.

How did online audiences and digital revenue get so concentrated? What does this concentration mean for business, politics, news, even national security? Is online oligopoly inevitable, or is there a way out of the internet trap? These are the questions this book seeks to answer. For us today, just as for Google two decades ago, the first lesson is this: small effects that compound quickly are not small effects.

A Scarcity of Attention

The World Wide Web is the most astonishingly successful technology of modern times, a technology that now underpins our social, economic, and political lives. You likely ordered this book online — or even had it wirelessly delivered to your phone or tablet or reader. The web is so popular, powerful, and omnipresent that we forget just how badly it has failed at its original goal.

The World Wide Web was built in an explicit attempt to eliminate hierarchy in communications. It tried to do this by combining other technologies developed with that same goal in mind. The first such technology was hypertext, originated in the 1960s by sociologist Ted Nelson. Hypertext documents were designed to be consumed nonsequentially. Links within the hypertext could point to other relevant passages, definitions, graphics, tables, or even other documents on the same computer. The second technology was the internet, which by the 1980s had become ubiquitous in universities and research labs. The internet had been created as a peer-to-peer network, in which there were no central hubs: each computer could send and receive data with any other computer.

Tim Berners-Lee, the creator of the web, saw that hypertext could piggyback on top of the internet. Instead of being sandboxed within a single computer, hypertext could link documents on computers continents apart. Berners-Lee called the project the World Wide Web to emphasize that "any node can be linked to any other," and to reflect "the distributed nature of the people and computers that the system could link." In the opening sentence of the WWW's project overview, one of the very first pages online, Berners-Lee declared, "There is no top to the Web." All pages, and all sites, were supposed to be created equal.

If there were no "top" sites on the World Wide Web in 1991, there certainly are now. The fact that some sites and apps are far more popular than others is the most important fact of online life. We network on Facebook, while competing sites get less than 1 percent of Facebook's traffic. We search the web using Google or Bing, and competing search engines have minuscule market share. We read and write our email using Google or Yahoo! or Microsoft. We use eBay for auctions, and we buy our books (and increasingly everything else) from Amazon. Among hundreds of millions of sites on the web, the four largest internet firms — Google, Facebook, Microsoft, and Yahoo! — capture a third of all web visits.

Concentration in online revenue is even more dramatic. The ten largest digital firms have always dominated digital advertising, taking three-quarters of ad dollars since at least the mid-1990s. But the shift to mobile and video has intensified concentration at the very top. As of mid-2016, Google and Facebook together combined for more than 73 percent of digital advertising in the United States, a remarkable duopoly over a $60 billion-a-year industry.

At its core, this is a book about the attention economy. It focuses on the interplay between money and attention, and how each is exchanged for the other. It shows how digital firms spend money — often mountains of money — to attract and keep audiences. The book shows why investments in things like server farms and fiber and backend software matter so much, and how sites with more content and better designs are able to tilt online competition in their favor.

The book also details how audiences are traded for revenue through ads and subscriptions. This exchange is highly unequal: for several reasons, larger outlets earn far more per audience member than smaller outlets. This reciprocal relationship between money and attention creates feedback loops, in which sites and apps that have early success can invest to become more successful still.

In itself, the idea of attention economics is not new. The notion is often traced back to Herbert Simon, political scientist and Nobel prize–winning economist. Writing about the problems of information overload in the 1960s, Simon wrote that

in an information-rich world, the wealth of information means a dearth of something else: a scarcity of whatever it is that information consumes. What information consumes is rather obvious: it consumes the attention of its recipients. Hence a wealth of information creates a poverty of attention and a need to allocate that attention efficiently among the overabundance of information sources that might consume it.

Simon's passage has been repeated so often that it verges on cliché, with Google claiming to find the quote on more than 4.5 million web pages.

Many scholars have claimed to pick up where Simon left off. In the past two decades many books and articles have been written about "attention scarcity," though this phrase has animated radically different ideas about digital media. Some discussions have been cloaked in business jargon or cyber-utopian rhetoric. One early treatment even declared that "attention transactions" would replace traditional money. More recent works, such as James Webster's excellent The Marketplace of Attention, have relied not on an idealized vision of the web but on data on how digital media is actually consumed.

This book differs from previous work on attention economics in numerous ways. Too often attention economics has been invoked to argue that "old" economics does not apply, that the digital world is throwing off the shackles of the analog one. This book uses new data sources, together with models borrowed from economic domains such as international trade and mathematical finance, to show that the digital world is not so different after all. Large media firms still dominate for reasons economists will find both novel and familiar. Diverse preferences can serve to concentrate audiences rather than spread them out.

Even more fundamentally, this book challenges claims that "money cannot reliably buy attention." This belief that digital attention cannot be bought is woven throughout internet scholarship and public debates — but it is flatly, undeniably wrong. Showing exactly how audiences are bought and sold is a central theme of this book.

There is one key point, though, in which this book is in agreement with previous work: understanding attention economics is crucial for understanding digital media's social impact. As James Webster writes, "Media need an audience before they can achieve a purpose." Much is at stake beyond just commercial interests. Thinking about attention economics leads us to consider the political economy of attention — the ways in which attention economics impacts politics, and vice versa. On one hand, political choices and public policies help determine who wins and loses online. On the other, the digital attention economy increasingly shapes public life, including what content is produced, where audiences go, and ultimately which news and democratic information citizens see.

Forces of Concentration

Building a better version of attention economics starts with a key problem: our understanding of the internet has been lopsided. The forces that disperse digital attention are widely understood, while the forces of concentration are not.

This sort of asymmetric understanding in internet scholarship looks a lot like the historically uneven advance of knowledge in other fields. Consider the history of geology. When geology was organized as a scientific field in the late 1700s, it was mostly the study of erosion. While geologists quickly learned how wind and water wore mountains down, figuring out what built them up in the first place was harder. It was not until the surprise discovery of plate tectonics in the 1960s that geology had good models of geologic uplift. In a sense, geology went for 150 years as only half of a discipline.

This sort of uneven understanding has occurred in the social sciences, too. Writing in 1997, Paul Krugman argued that economic geography — the study of where production and consumption happen in space — was like geology before the discovery of plate tectonics: "economists understood why economic activity spreads out, not why it becomes concentrated." Economists had long understood how high rents push economic activity away from an urban center, a process modeled well by Econ 101 supply-and-demand models. But such models had a critical problem: they could not explain why cities would ever form in the first place. Good models of city formation emerged only once economists understood how imperfect competition can create urban concentration — for example, how stockbrokers in lower Manhattan in New York City can outperform those in Manhattan, Kansas.

Those who write about digital media today face the same sort of problem that previously a?icted geology and spatial economics. The distribution of online audiences, like the distribution of people in the real world, is the result of a tug-of-war between forces of concentration and forces of dispersion. But while the forces of dispersion have been widely understood and celebrated, the forces that concentrate audiences have been systematically ignored.

Most writing to date has assumed that digital media will produce "centrifugal diffusion" in audiences. Again and again, observers have said that the web is a "narrowcasting" or "pointcasting" medium, that it tilts the playing field toward small content producers, that it will, to paraphrase the U.S. Supreme Court in Reno v. ACLU (1997), make everyone a pamphleteer. We have been told that "the monolithic empires of mass media are dissolving into an array of cottage industries," that the internet has empowered an "army of Davids," that the internet will mean "the end of big." Argument has focused not on whether audiences were diffusing, but rather how far and how fast they were spreading out.

Traffic data, though, show that audiences keep stubbornly refusing to decentralize. This book is, in part, an outgrowth of my previous book The Myth of Digital Democracy, which showed that online audiences follow concentrated power law patterns. Some commentators dismissed this evidence of concentration as premature, saying that it ignored the rapidly evolving nature of the internet. Micah Sifry argued, for example, that "it is dangerous to make conclusive statements about such a young and dynamic space." The book's evidence of audience concentration online, Matt Bai suggested, "only reflects a particular moment in time."

This wait-and-see argument is a natural consequence of our lopsided understanding of online audiences. If the only forces that shape internet traffic are dispersive, then, indeed, we should just wait, and all will be leveled. Like Vladimir and Estragon in Waiting for Godot, many still hold out hope that the internet they've been waiting for will arrive. Eventually.

All of these commentators are right that some old forces of concentration do not apply in digital media. There is no scarcity of spectrum on the web. There is no need for digital-only media firms to maintain printing presses or a fleet of delivery vans. Motivated citizens can share video without a broadcast license. Single journalists can start their own digital publications with only a laptop and a Starbucks latte.

But the unrelenting focus on new, small-scale sites ignores the elephants in the room — the large digital platforms where users spend the most time, and that soak up nearly all online profits. In celebrating the profusion of citizen blogs, we must also understand why the vast majority are abandoned and unread. The centripetal forces deserve just as much attention as the centrifugal ones do.

So the body of this book begins by documenting the other half of the story. Chapter 2 details how large internet firms can take advantage of a host of economies of scale, even beyond simple network effects. As a group, large sites load faster. They are prettier and more usable. They have more content updated more frequently. They rank higher in search results. They have established brands, and visitors are more practiced in navigating them. They are more efficient platforms for advertising. There is substantial evidence that each of these factors, individually, serves to concentrate traffic.

Chapter 3 goes further, showing that large firms and websites are far better at personalizing their content for users. Digital media defined by "The Daily Me" does not favor small-scale content producers, as most have assumed. Instead, it favors those with resources: money, staff, data, computing horsepower, intellectual property, an established audience.

Better Models

Detailing the forces that concentrate internet audiences is a start. But what has been especially needed are not new facts but new models, theories that explain the broad patterns of digital traffic and revenue from Facebook and Google all the way down to tiny personal blogs. What we need are simplified stories that can explain both the enormous concentration at the top of the web and the (very) long tail of smaller sites.