Building Web Reputation Systems [Paperback]

Randy Farmer (Author), Bryce Glass (Author)

Book Description

Publication Date: March 23, 2010

What do Amazon's product reviews, eBay's feedback score system, Slashdot's Karma System, and Xbox Live's Achievements have in common? They're all examples of successful reputation systems that enable consumer websites to manage and present user contributions most effectively. This book shows you how to design and develop reputation systems for your own sites or web applications, written by experts who have designed web communities for Yahoo! and other prominent sites.

Building Web Reputation Systems helps you ask the hard questions about these underlying mechanisms, and why they're critical for any organization that draws from or depends on user-generated content. It's a must-have for system architects, product managers, community support staff, and UI designers.

• Scale your reputation system to handle an overwhelming inflow of user contributions
• Determine the quality of contributions, and learn why some are more useful than others
• Become familiar with different models that encourage first-class contributions
• Discover tricks of moderation and how to stamp out the worst contributions quickly and efficiently
• Engage contributors and reward them in a way that gets them to return
• Examine a case study based on actual reputation deployments at industry-leading social sites, including Yahoo!, Flickr, and eBay

Editorial Reviews

About the Author

F. Randall "Randy" Farmer has been creating online community systems for over 30 years, and has co-invented many of the basic structures for both virtual worlds and social software. His accomplishments include numerous industry firsts (such as the first virtual world, the first avatars, and the first online marketplace). Randy worked as the community strategic analyst for Yahoo!, advising Yahoo properties on construction of their online communities. Randy was the principal designer of Yahoo's global reputation platform and the reputation models that were deployed on it.

Bryce Glass is a principal interaction designer for Manta Media, Inc. Over the past 13 years, he's worked on social and community products for some of the web's best-known brands (Netscape, America Online and Yahoo!).

Product Details

• Paperback: 336 pages
• Publisher: Yahoo Press; 1 edition (March 23, 2010)
• Language: English
• ISBN-10: 059615979X
• ISBN-13: 978-0596159795
• Product Dimensions: 7 x 0.8 x 9.1 inches
• Shipping Weight: 12 ounces (View shipping rates and policies)
• Average Customer Review: 4.1 out of 5 stars  See all reviews (9 customer reviews)
• Amazon Best Sellers Rank: #389,909 in Books (See Top 100 in Books)

Most Helpful Customer Reviews

23 of 23 people found the following review helpful

4.0 out of 5 stars Not a Classic, But a Worthwhile Read July 19, 2010

By Eric Goldman

Format:Paperback

By Eric Goldman

For the past couple of years, I have been researching how we regulate reputation systems. As part of researching other disciplines' approaches to reputation systems, I was pleasantly surprised to find this book, which discusses web reputation systems from a technical/product development standpoint. I'm not aware of other books directly on point, so that alone makes the book noteworthy.

The word "reputation" is a complex and nuanced word. This book defines reputation as "information used to make a value judgment about an object or a person." Notice how this definition treats reputation as actionable information (i.e., making a "judgment"). I favor that approach; my work also uses an actionable definition of reputation.

Their definition equally treats both objects and people as having "reputation," and this does not work. In general, people are dynamic, i.e., they can change behavior; while content is static, i.e., an item of content does not change its character unless subsequently edited. This single definition of "reputation" created significant tension throughout the book. Recognizing this, the authors often bifurcated the discussion to separately address the process of establishing a person's "reputation" (which they confusingly called "karma"). However, the book primarily focuses on grading and sorting content items, especially user-generated content, and I personally would not describe content items as having a "reputation." As a result, I think the book is mistitled. It principally addresses content filtering, not "reputation" as I use the term.

Although this analytical tension pervades the book, the book nevertheless contained a lot of useful insights about both content filtering and establishing user trustworthiness. The authors have a lot of experience building filtering systems for different websites, so the book is packed with the kind of first-hand observations that only an insider can offer. There's no substitute for the voice of experience when designing Web 2.0 UGC systems, and this book provides an easy and accessible way to learn some of these tips and tricks.

The book emphasizes the authors' contributions to the reputation system at Yahoo Answers, and rightly so. Yahoo Answers has emerged into a bona fide success story and recently trumpeted its billionth answer. In my opinion, the book's high point is Chapter 10, a case study of how Yahoo Answers developed a new filtering and reputation system that helped turbocharge the Yahoo Answers community.

Although the book doesn't say this directly, two key lessons from Yahoo Answers' evolution are:

1) UGC websites should let users vote on content, but not all user votes should be weighted equally.

2) UGC websites do not need to publish all user-supplied content items in an equally prominent manner. Perhaps some content should be obscure/hard-to-find until other users validate it.

The book pitches these conclusions as novel, but they seemed fairly intuitive to me. We implemented a very similar system embodying these two points back in 2000-01 at Epinions. Epinions allowed users to grade each others' content; we weighted votes differentially based on users' credibility; and we displayed ungraded and poorly graded content only to registered users (a small fraction of our readers). The fact that the authors "discovered" these conclusions at Yahoo Answers shows the dire need for books like this to help websites implement best UGC management practices without reinventing the wheel.

The fact that the authors didn't acknowledge the Epinions precedent (and other systems like it) highlights another weakness of the book. There is a deep academic literature addressing the book's topics (especially on content filtering and user incentive systems), but the book barely acknowledges this literature. For example, several times the authors cite Dan Ariely's Predictably Irrational for descriptions of human psychology and foibles. That's a perfectly credible citation, but it should be one of many literature citations, not the only citation. Instead of dipping into the rich academic literature, the book almost exclusively relies on the authors' experience-based impressions. These impressions are a valuable information source that makes the book worth reading. However, because those impressions aren't tempered with more rigorous academic findings, it's not clear to me at all that the authors' conclusions represent true best practices...or even state-of-the-art.

Because of its many structural flaws, this edition will not become a classic. Nevertheless, I have enthusiastically recommended the book to several UGC start-ups because the book provides a good repository of high-value experience-based perspectives that are not readily available elsewhere. Even if the book's recommendations are debatable, it's a debate worth having.

5 of 5 people found the following review helpful

5.0 out of 5 stars A deep, insightful, and well-presented treatment of on-line reputation models August 3, 2010

By John C. Stepper

Format:Paperback|Amazon Verified Purchase

I'm researching different models to use for our corporate collaboration platform and I'm looking to implement a social recognition/reputation framework. Our framework needs to make sense for users and content in many different businesses and support functions. It's one thing to rate things on Amazon and Netflix and another to deploy a system inside an enterprise. (I'd also researched peter Reiser's "Community Equity" - [...])

This excellent book proved *extremely* useful. It dissects the different popular models - Amazon, eBay, Yahoo, Digg - as well as many others I was not familiar with. It analyzed each and clearly highlighted the pros and cons of each; how different models can be used (and abused) to meet different business objectives. Priceless.

More than a dry analysis, the book is filled with interesting (to me!) insights on the potential pitfalls of each model. I was often surprised at how things I took for granted had some hidden biases or other traps. The sections on the issues with leaderboards, for example, were clear and practical.

My only advice when reading is to skim the initial chapter on the "graphical language" for reputation models. It seemed unnecessary and too abstract. It almost put me off from reading the rest of the book but I needed help. And I'm really glad I continued. The remaining chapters got more and more engaging. The final case study on preventing abuse in Yahoo Answers reads like a page-turner. (Really!)

If you are building any kind of application with ratings or some kind of recognition/social capital/reputation for users, then read this book. It will help you avoid the mistakes made by others before you (you'll never forget the story of The Dollhouse Mafia). And it will help you meet your business objectives for including ratings and reputation in the first place.

(Note: The authors are also on twitter: @frandallfarmer, @soldierant, and @buildingrep.)

3 of 3 people found the following review helpful

4.0 out of 5 stars Designing Web Reputation Systems June 25, 2012

By Eric Jain

Format:Paperback

This book lays out the options you have for integrating reputation systems into a site, and helps make design choices (e.g. should I use thumb or star ratings).

The theoretical discussion of terms and concepts is well balanced with anecdotes and examples from real sites (typically Yahoo properties).

Not covered are algorithms (apart from one simplistic formula for factoring the number of ratings into an overall ranking), and much more could be said about how to deal with people gaming the system (incl trolls and spammers).