Beautiful Data: The Stories Behind Elegant Data Solutions [Paperback]

Toby Segaran (Author), Jeff Hammerbacher (Author)

Book Description

Publication Date: July 28, 2009 | ISBN-10: 0596157118 | ISBN-13: 978-0596157111 | Edition: Original

In this insightful book, you'll learn from the best data practitioners in the field just how wide-ranging -- and beautiful -- working with data can be. Join 39 contributors as they explain how they developed simple and elegant solutions on projects ranging from the Mars lander to a Radiohead video.

With Beautiful Data, you will:

• Explore the opportunities and challenges involved in working with the vast number of datasets made available by the Web
• Learn how to visualize trends in urban crime, using maps and data mashups
• Discover the challenges of designing a data processing system that works within the constraints of space travel
• Learn how crowdsourcing and transparency have combined to advance the state of drug research
• Understand how new data can automatically trigger alerts when it matches or overlaps pre-existing data
• Learn about the massive infrastructure required to create, capture, and process DNA data

That's only small sample of what you'll find in Beautiful Data. For anyone who handles data, this is a truly fascinating book. Contributors include:

Nathan Yau Jonathan Follett and Matt Holm J.M. Hughes Raghu Ramakrishnan, Brian Cooper, and Utkarsh Srivastava Jeff Hammerbacher Jason Dykes and Jo Wood Jeff Jonas and Lisa Sokol Jud Valeski Alon Halevy and Jayant Madhavan Aaron Koblin with Valdean Klump Michal Migurski Jeff Heer Coco Krumme Peter Norvig Matt Wood and Ben Blackburne Jean-Claude Bradley, Rajarshi Guha, Andrew Lang, Pierre Lindenbaum, Cameron Neylon, Antony Williams, and Egon Willighagen Lukas Biewald and Brendan O'Connor Hadley Wickham, Deborah Swayne, and David Poole Andrew Gelman, Jonathan P. Kastellec, and Yair Ghitza Toby Segaran

Editorial Reviews

About the Author

Toby Segaran is the author of Programming Collective Intelligence, a very popular O'Reilly title. He was the founder of Incellico, a biotech software company later acquired by Genstruct. He currently holds the title of Data Magnate at Metaweb Technologies and is a frequent speaker at technology conferences.

Jeff Hammerbacher is the Vice President of Products and Chief Scientist at Cloudera. Jeff was an Entrepreneur in Residence at Accel Partners immediately prior to joining Cloudera. Before Accel, he conceived, built, and led the Data team at Facebook. The Data team was responsible for driving many of the statistics and machine learning applications at Facebook, as well as building out the infrastructure to support these tasks for massive data sets. The team produced several academic papers and two open source projects: Hive, a system for offline analysis built above Hadoop, and Cassandra, a structured storage system on a P2P network. Before joining Facebook, Jeff was a quantitative analyst on Wall Street. Jeff earned his Bachelor's Degree in Mathematics from Harvard University.

Product Details

• Paperback: 386 pages
• Publisher: O'Reilly Media; Original edition (July 28, 2009)
• Language: English
• ISBN-10: 0596157118
• ISBN-13: 978-0596157111
• Product Dimensions: 7 x 1 x 9.1 inches
• Shipping Weight: 1.8 pounds (View shipping rates and policies)
• Average Customer Review: 3.9 out of 5 stars  See all reviews (19 customer reviews)
• Amazon Best Sellers Rank: #57,897 in Books (See Top 100 in Books)
  • #6 in Books > Computers & Technology > Programming > Software Design, Testing & Engineering > Structured Design
  • #27 in Books > Computers & Technology > Databases > Database Design
  • #42 in Books > Computers & Technology > Computer Science > Information Theory

Most Helpful Customer Reviews

50 of 50 people found the following review helpful

4.0 out of 5 stars Occasionally brilliant discussions on data and what data can and cannot do October 11, 2009

By Allison M. Perkel

Format:Paperback|Amazon Verified Purchase

"Beautiful Data" is a collection of essays on data; how people have transformed it, worked within its confines, and offers a glimpse of where we might go. Many of the essays are wonderful snippets into how some people perceive data while others fall flat. Overall its a mostly enjoyable read that helps open up your mind to new potentials.

First a disclaimer; I am not a data person. However I've been involved, fairly heavily, in the data field. In the parlance of the world, I'm a back end person. However I'm always trying to think about the front end; how will things be used and what information can we gleen from the system (or systems). With that in mind, this is a book that speaks to me - its all about the front end.

Some of the best essays in the book would be:

The first essay by Nathan Yau he talks very much about user created data and personal databases (knowledge bases). What's exciting here is how he takes data already out there, data you have provided, and creates something useful and yes, beautiful, out of it.

The Second essay by Follett and Holm really gets down to how if you want the data, you need to present it in a way that brings people into the process. As someone who has a slight crush on the statistics and practices in polling (and designing poll questions) this essay really was a fascinating read.

The third essay by Hughes detailed how he handled images on the Mars mission. There wasn't anything here that wasn't done in embedded systems 15 years ago; still it was a great walk down memory lane since I used to program embedded imaging systems.

Chapter 4 really hit home PNUTShell is cloud storage and data processing in real time. This really is the stuff of the future....

Chapter 5 by Jeff Hammerbacher really didn't offer too many insights but his writing style is fluid and fun plus he offered a glimpse into how Facebook grew.

We then have the slow section of the book - Chapter 8 on distributed social data had promise but it read more like a company white page than an interesting article. Same with Chapter 12 [...].

Thankfully chapter 10 on Radiohead's "House of Cards" video was there - and here we are presented with true beauty in data - beautiful enough to create a music video out of!

I'm still on the fence with Chapter 13 - What Data Doesn't Do. It was an interesting chapter but it felt both too long and too short at the same time. I almost felt that in the author, Coco Krumme, were to write a book on this topic, I'd want to read it. However her essay was not the right vehicle.

Finally, the last chapter - "Connecting Data" was a truly inspiring piece; one that offers up paths for the future. I am sure a few start ups will form over the questions posed in by Segaran (or maybe the questions to the questions).

Overall there were enough strengths to overcome the weak chapters. My main complaints are trivial; poor binding of the book, too many PhD candidate papers and not enough from out in the trenches. I'd love to see something from Stonebreaker here; its hard to talk about beautiful data and not have him in it. Or forget [...]and talk about many eyes. Or map reduce. Still, "Beautiful Data" succeeds. It opened up my mind to different possibilities for data representation and usage. Read more ›

30 of 30 people found the following review helpful

3.0 out of 5 stars Beautiful cover, that's for sure November 8, 2009

By Dimitri Shvorob

Format:Paperback

... Contents are less impressive. O'Reilly bring together a heterogeneous group of authors and let them fend for themselves, with no editorial effort to unite their stories. Some authors hold their own, presenting interesting analyses and visualizations, or just interesting tales, others are less successful. (The spectrum of statistical expertise, for example, is bounded by Andrew Gelman and a graduate student believing that normality is a requirement of the central-limit theorem). 'Interesting' is a good thing, but for $40 I would like 'useful'. An appealing leisure read, but not much more, I am afraid.

17 of 19 people found the following review helpful

5.0 out of 5 stars Excellent overview of new approaches to harnessing and displaying data to support knowledge communication August 2, 2009

By Techie Evan

Format:Paperback

This book tells you what's possible now and what's on the horizon when it comes to data representation, collection, management, processing, analysis, sharing, and display. Very little code is provided because each chapter is mostly a conceptual discussion of approaches to tackling various kinds of challenges involving data, the lifeblood of any application. My favorite chapters are: 4, 5, 7 and 20. Below are my short notes for each chapter to give you some idea of the book's contents.

Ch. 1 Seeing Your Life in Data by Nathan Yau
Hoping to better understand their impact on and exposure to the environment, participants in one of Yau's projects download software onto their phones that then upload GPS data to servers as they go about their daily activities. One of Yau's early challenges was to summarize the data and make it meaningful to the participants: for example, what does it mean to emit 1,000 kilograms of carbon in a week? What he found helpful and not so helpful in data visualization are instructive.

Ch. 2 The Beautiful People: Keeping Users in Mind When Designing Data Collection Methods by Jonathan Follett and Matthew Holm
When there is no explicit profit to be made, how do you convince a person to take the time to answer your survey questions?

Ch. 3 Embedded Image Data Processing on Mars by J.M. Hughes
Like everything else onboard a spacecraft, the computing system is custom built with minimalism and other stringent specifications (e.g., withstand radiation) in mind. How does one harness limited resources to get the job done?

Ch. 4 Cloud Storage Design in a PNUTShell by Brian Cooper, Raghu Ramakrishnan, and Utkarsh Srivastava
Yahoo! engineers have a very challenging job.... Web pages containing potentially complex social data must load and update quickly regardless of where the data may be mastered in servers distributed across the world. Learn why they jettisoned some conventional database concepts in favor of: flexible schemas, timeline consistency-driven data updates, etc.

Ch. 5 Information Platforms and the Rise of the Data Scientist by Jeff Hammerbacher
The author mentions that according to IDC, the digital universe will expand to 1,800 exabytes by 2011 (1 exabyte = 1 billion gigabytes) and the vast majority of that data will not be managed by relational databases. The Facebook Information Platform described in this chapter can manage structured and unstructured data in an integrated manner, and can extract useful information from terabytes of data in seconds. Similar platforms built at Fox Interactive Media and Microsoft are also described briefly.

Ch. 6 The Geographic Beauty of a Photographic Archive by Jason Dykes and Jo Wood
The Geograph British Isles Project aims to collect geographically representative photographs and information for every square kilometer of great Britain and Ireland. Learn new data visualization techniques!

Ch. 7 Data Finds Data by Jeff Jonas and Lisa Sokol
Technologies similar to those already used in, say, fraud surveillance can be adapted for other more mundane applications.

Ch. 8 Portable Data in Real Time by Jud Valeski
How can companies facilitate the sharing of and access to social data without having to invest on an inordinate amount of infrastructure?

Ch. 9 Surfacing the Deep Web by Alon Halevy and Jayant Madhaven
Web contents that lie hidden behind HTML Forms are part of the Deep Web that search engines have not indexed very well but that may partially change soon.

Ch. 10 Building Radiohead's House of Cards by Aaron Koblin with Valdean Klump
The author helped produce a video for the music group entirely from visualization of data, and without the use of cameras or lights. Google Code urls given. You gotta see the interesting video!!

Ch. 11 Visualizing Urban Data by Michal Migurski
Learn how to visualize trends in urban crime, using maps and data mashups

Ch. 12 The Design of Sense.us by Jeffrey Heer
The combination of interactive visualization and social interpretation can help an audience more richly explore a data set.

Ch. 13 What Data Doesnt't Do by Coco Krumme
Data doesn't stand alone. In real-world decision-making, information is rarely packaged neatly and data isn't free from interpretive biases.

Ch. 14 Natural Language Corpus Data by Peter Norvig
Natural language tasks like word segmentation or spelling correction can be handled using probabilistic models built from processed large data sets.

Ch. 15 Life in Data: The Story of DNA by Matt Wood and Ben Blackburne
The human genome has been well annotated and 40 other species have been sequenced. With each new discovery, however, more questions are raised, and more research data is generated. The need for efficient sequence search, alignment, and assembly tools, as well as safe housing for the millions of genomes, will continue to grow. Learn how scientists are rising to the challenge.

Ch. 16 Beautifying Data in the Real World by Jean-Claude Bradley, et al.
How online publishing of scientific data can be improved upon

Ch. 17 Superficial Data Analysis: Exploring Millions of Social Stereotypes by Brendan O'Connor and Lukas Biewald
Ch. 18 Bay Area Blues: The Effect of the Housing Crisis by Hadley Wickham, Deborah F. Swayne, and David Poole
Ch. 19 Beautiful Political Data by Andrew Gelman, Jonathan P. Kastellec, and Yair Ghitza
These chapters show you data analyses in action: how to prep data, smooth out the effects of noisy or outlier data, etc.

Ch. 20 Connecting Data by Toby Segaran
We need to break down information silos but how? The use of Semantic Web and/or Collective Reconciliation techniques are discussed. Read more ›

16 of 21 people found the following review helpful

3.0 out of 5 stars Good content, lousy print quality September 1, 2009

By Thomas W. Gonzalez

Format:Paperback|Amazon Verified Purchase

While the content of this book is interesting and informative, I am struck with what lousy print quality it is. For a $40+ book you would expect a hardback, or at least a paperback with thick stock pages and color plates that actually look good. It was hard for me to appreciate the content when it felt like each page (or the cover) was going to rip because they were such thin and poor quality stock. The color plates are washed out and pixelated. I was expecting the same high quality we got with "Beautiful Code". O'Reilly usually does a much better job. That said, if these types of aesthetics don't bother you (although with a title like "Beautiful Data" I would question that it wouldn't) the book itself is an interesting read.

3 of 3 people found the following review helpful

2.0 out of 5 stars A selection of mostly uninteresting stories loosely related to data March 30, 2011

By hvu

Format:Paperback

Most of the stuff here really didn't hold my interest, I was looking something more closely related to practical engineering work. But it does have a nice looking cover.

5 of 6 people found the following review helpful

5.0 out of 5 stars Outstanding Case Studies in Data Capture, Processing & Visualization August 6, 2009

By Ira Laefsky VINE™ VOICE

Format:Paperback

From the title, I might have guessed that this was another pretty coffee table book on Information Visualization--Basically, an art book unless you already had the insight and talent to apply its principles to your own work in Data Representation. But, I should have expected (and I did receive) much more from O'Reilly's efforts in this domain. While the book is indeed beautiful, it more importantly provides a set of carefully described case studies in all phases of the data capture, processing, analysis, communication and visualization life cycle. Detailed descriptions are given of the motivation and design of data capture, analysis and design system in fields as diverse as personal energy consumption (and carbon footprint), mars explorer robotics, high quality market research, interpretation of U.S. Census statistics, and the visualization of DNA databases.

The case study methodology points out the necessity of designing all phases of the data capture, processing, analysis and representation process around the goals, open questions and constraints of the client organization, or user/consumer of the data
whose decisions are being informed. The thinking and design process behind these cases of beautiful data are fully described--this will enable you (or an untalented artist such as myself) to design systems which answer the questions and support the decisions of the individual or organization who needs this data.

--Ira Laefsky