The Art of Capacity Planning: Scaling Web Resources [Paperback]

John Allspaw (Author)

Book Description

Publication Date: September 30, 2008 | ISBN-10: 0596518579 | ISBN-13: 978-0596518578 | Edition: 1

Success on the web is measured by usage and growth. Web-based companies live or die by the ability to scale their infrastructure to accommodate increasing demand. This book is a hands-on and practical guide to planning for such growth, with many techniques and considerations to help you plan, deploy, and manage web application infrastructure.

The Art of Capacity Planning is written by the manager of data operations for the world-famous photo-sharing site Flickr.com, now owned by Yahoo! John Allspaw combines personal anecdotes from many phases of Flickr's growth with insights from his colleagues in many other industries to give you solid guidelines for measuring your growth, predicting trends, and making cost-effective preparations.

Topics include:

• Evaluating tools for measurement and deployment
• Capacity analysis and prediction for storage, database, and application servers
• Designing architectures to easily add and measure capacity
• Handling sudden spikes
• Predicting exponential and explosive growth
• How cloud services such as EC2 can fit into a capacity strategy

In this book, Allspaw draws on years of valuable experience, starting from the days when Flickr was relatively small and had to deal with the typical growth pains and cost/performance trade-offs of a typical company with a Web presence. The advice he offers in The Art of Capacity Planning will not only help you prepare for explosive growth, it will save you tons of grief.

Editorial Reviews

About the Author

John Allspaw is currently Operations Engineering Manager at Flickr, the popular photo site. He has had extensive experience working with growing web sites since 1999. These include online news magazines (Salon.com, InfoWorld.com, Macworld.com) and social networking sites that experienced extreme growth (Friendster and Flickr). During his time at Friendster, traffic increased 5X. He was responsible for their transition from a couple dozen servers in a failing data center to over 400 machines across two data centers, and the complete redesign of the backing infrastructure. When he joined Flickr, they had 10 servers in a tiny data center in Vancouver; they are now located in multiple data centers across the US. Prior to his web experience, Allspaw worked in modeling and simulation as a mechanical engineer doing car crash simulations for the NHTSA.

Product Details

• Paperback: 156 pages
• Publisher: O'Reilly Media; 1 edition (September 30, 2008)
• Language: English
• ISBN-10: 0596518579
• ISBN-13: 978-0596518578
• Product Dimensions: 7 x 0.5 x 9.1 inches
• Shipping Weight: 11.2 ounces (View shipping rates and policies)
• Average Customer Review: 4.3 out of 5 stars  See all reviews (12 customer reviews)
• Amazon Best Sellers Rank: #182,211 in Books (See Top 100 in Books)

More About the Author

John has worked in systems operations for over fourteen years in biotech, government and online media. He started out tuning parallel clusters running vehicle crash simulations for the U.S. government, and then moved on to the Internet in 1997. He built the backing infrastructures at Salon.com, InfoWorld.com, Friendster, and Flickr. He is now VP of Tech Operations at Etsy, and is the author of "The Art of Capacity Planning" and "Web Operations" published by O'Reilly. He speaks from time to time at conferences on topics related to web operations, operations and development culture, infrastructure, and capacity planning.

He's a dad, guitarist, engineer, and wiseguy.

Most Helpful Customer Reviews

13 of 14 people found the following review helpful

5.0 out of 5 stars A complex art distilled October 16, 2008

By Adam Jacob

Format:Paperback

John Allspaw has done something that very few of his peers would have been able to do. He has taken a black art, Capacity Planning, and he turned it in to a series of steps that anyone can follow.

The book is filled with common case studies, for how to plan capacity for things like web server farms, database clusters, and caching layers. The real value is in watching how the author applies the same formula in each case, giving Systems Administrators and Executives the tools they need to do a better job of capacity planning in their own unique infrastructures.

As the earlier review says, it's a short book. In my opinion, that's a good thing: it's goal is to teach you how to perform capacity planning in any environment. If it was longer, it would have been full of more examples, which would likely only serve to lead the reader away from the core principles. You need to learn *how* to capacity plan an infrastructure, not get pat (and often incorrect) advice on how to measure your web farm.

The discussion on curve fitting and trend prediction is worth it alone - I'm aware of no other book on the topic that shows so clearly how to examine your data in service of capacity planning.

It's the process I'll follow from now on.

7 of 7 people found the following review helpful

3.0 out of 5 stars Good introduction to Capacity Planning for Web Operations March 16, 2009

By Sanket M. Naik

Format:Paperback

The Art of Capacity Planning is a good introduction to Capacity Planning for Web Operations that touches on the following topics:
* Why do you need capacity planning?
* What information should you gather for capacity planning and how?
* How to predict trends for your web applications?
* How and when to procure new hardware?
* How to create a sustainable capacity planning process?

As the author mentions in the preface, the book has a lot of common sense material. Most experienced enterprise web operations architects should be familiar with this material. But, it is refreshing to see this urban wisdom captured and printed in a book format. The book is unique in that it is not meticulously organized and illustrated like a text book or a reference guide. It provides a smattering of anecdotes, examples, gotchas, and tools from the author's experience in a rapidly growing start up environment at Flickr.

I am looking forward to a second edition of the book where the author can delve deeper into some missing aspects that are critical to capacity planning like log analysis and performance improvements. Enterprise web operations folks who are familiar with commercial tools like Sitescope, OpenView, Opsware, Gomez, etc. rather than free/open source tools and who manage a large number of diverse applications might have a learning curve to relate the examples in the book to their environment.

9 of 10 people found the following review helpful

5.0 out of 5 stars Chock full o' takeaways October 17, 2008

By Steve Souders

Format:Paperback

Right out of the gate, John covers a topic near and dear to my heart: metrics. His advice? "Measure, measure, measure." John reinforces this by including an incredible number of charts throughout the book. He goes on to say that our measurement tools need to provide an easy way to:
* Record and store data over time
* Build custom metrics
* Compare metrics from various sources
* Import and export metrics

As I read the book, I found myself nodding and thinking, "yes, yes, this is exactly what I learned!" Although it's been more than five years since I was buildmaster for My Yahoo!, I really resonated with the advice John provides, like this one: "Homogenize hardware to halt headaches". (You have to love the alliteration, too.)

In a thin book that's easy to read, John covers a large number of topics. He talks about load testing, with pointers to tools like Httperf and Siege. There are several sections that talk about caching architectures and the use of Squid. He provides guidelines when it comes to deployment, such as making all changes happen in one place, the importance of defining roles and services, and ensuring new servers start working automatically. At the end he even manages to cover virtualization and cloud computing, and how they come into play during capacity planning.

The Art of Capacity Planning is full of sage advice from a seasoned veteran, like this one: "The moral of this little story? When faced with the question of capacity, try to ignore those urges to make existing gear faster, and focus instead on the topic at hand: finding out what you need, and when." When I read a technical book, I'm really looking for takeaways. That's why I loved The Art of Capacity Planning, and I think you will, too.