on July 14, 2009
Performance is all about measurement, but it is also about understanding the performance characteristics of the system enough to make the measurements meaningful. Typical real-world enterprise computing configurations are a complex mixture of hardware and software that whose individual pieces need explanation and Henry Liu's book does an admirable job covering this important background. Starting with the hardware topics like Hyperthreading, Chipsets, the different kinds of RAID disk controllers, and network adapters are covered. It is harder for the book to do justice on the software configuration, because there is such a broad variety of software vendors and products, but the book does define basic architectures (e.g. Client-Server, 3 Tier, Service oriented Architecture). The book is also useful as a dictionary for the gaggle of terms and acronyms pervasive in the industry (OLTP, SOAP, WSDL, COTS ...).
My favorite part of the book is the chapter on performance testing. Here the author shows offers some sound, practical guidance on measuring real world applications. He goes into detail on how to use the windows Perfmon and TaskManager software to get useful performance data. Perhaps is most valuable advice is pointing users to the performance metrics that PerfMon and TaskManager CAN measure that are most important for finding system bottlenecks. He very briefly touches on performance issues in the databases themselves, I would have liked to see a more thorough exploration of this space (after all there really are only 3 big players in the database software world). Nevertheless I found this section quite useful. Throughout this section there are measurements of real-world systems that were analyzed and I found these examples to be useful.
For those more theoretically inclined, a third of the book is devoted to queuing theory and case studies that apply the theory to real world problems like deciding how much changes in the configuration (upgrading disks, CPUs, network) will have on overall response time. Each chapter comes with a set of problems and s reading list for further depth, making the book suitable for an advanced computer science class.
Finally the last third of the book is devoted to what Liu calls `API profiling', which is the instrumentation of the software with timestamps to methodically determine where the time is being spent. This is perhaps the part of the book that will age most quickly. It really is a sad testament on the state of performance tools that Liu had to build his own profiling infrastructure to get the detailed data he needed to perform investigations. Thankfully, this is changing. For example on Windows there is an instrumentation technology called Event Tracing for Windows (ETW) and tools that use it like the free download of XPERF which are destined to be even more important than PerfMon is today. Nevertheless, while many of the details of these later chapter will become superseded by better technology from software vendors (no longer necessary to `roll your own') the logic for interpreting this data will always be useful, and Liu's `Performance Maps' are a useful way of interpreting the data.
On the whole I found the book to be a to be a thoughtful overview of area of enterprise performance and scalability tuning that that its many examples lived up to its subtitle of `a Quantitative approach'. It is a happy addition to my library.
Vance Morrison - Performance Architect for the .NET Runtime
on December 5, 2010
This is my first book review ever. I wonder what took me so long; I guess the right book had to come along.
A couple of weeks ago I was in one of those situations that you probably know (and maybe you are in it right now?), where your job requires you to get something important done, but your foundations are not as solid as you would need them to be. In my context I was supposed to come up with a performance testing strategy for rather diverse and complicated system.
Dr. Liu's book helped me a lot. It provided me very good input on how to structure my performance test strategy. I like very much the concise style. After some pages you are convinced the author knows what he is talking about. When reading the book I had the impression the author takes me by the hand and guides me through the subject matter. Even if things get a bit tough (the Queuing Theory chapter) I had the confidence that I can make it through, if I spend sufficient time on it. There is plenty of practical advice and real world examples, which help too.
Of course there are still some open questions:
* The book focuses on performance optimizing/tuning individual transactions. Often, especially if you have middleware to deal with, you don't have such a pure environment, but resources are shared among a wide variety of transaction types. My thinking was that there should be a certain "background noise" setup when doing the measurements. When I emailed the author about this (and got a surprisingly quick response!), he said it is beneficial to know the absolute amounts or deltas of the resources consumed by a specific transaction type, to determine more accurately the hardware sizing for that type of transaction. For a more diverse environment of transaction types, it might make sense to set up the proper "background noise".
* Another question is related to the sequence of types of testing. He lists Performance Regression Testing first, before Performance Optimization/Tuning, Performance Benchmark Testing and Scalability Testing. I would think readers would assume that the types of testing are listed in the sequence they should be approached. To me it appears that the Performance Regression Testing should be last in the list.
* I also think that people like me would have benefited even more from the book if there were more guidelines to detect resource saturation. Maybe I haven't read the corresponding parts of the book yet, but so far I found: CPU queue longer than 2, CPU utilization larger than 80%, disk utilization, disk utilization larger than 20%. (Considering the responsiveness of the author to my email question, I wouldn't be surprised if he would add some more guidelines as a Comment to his review! :) )
In summary, this book transformed me and my little team from having a vague idea of how to approach the performance testing to one with a more solid conceptual foundation of how it should be done. (Let's see whether we get the job done now.)
So, this review got a bit long for being my first one (maybe I should also learn from the author his concise writing style, in addition to performance testing). Oh well. Was it any good?
on February 14, 2011
The challenge of making an effective software system involves defining all of the critical parts and making the appropriate changes to the system (hardware and software) that will sufficiently increase execution and reduce resource consumption. This is the essence of performance analysis.
Performance analysis is a critical component for developing any software solution. Poor performance can have severe consequences such as damaged customer relations, lost income, business or project failures, and increased costs for additional computer and personnel resources. A well defined performance plan provides a structured and repeatable approach that may be integrated throughout the system life cycle to best mitigate risk. Too early, and later changes to the system will be missed. Too late, and major components of the system may need to be totally reworked.
To tune or not to tune, that is the question... since you obviously can't test everything. Does the system meet its objectives for response time, throughput, and resource consumption? Can the system be easily scaled if workload increases? These objectives must be thoroughly understood and defined to insure that the system is successful. Processes that require performance analysis are typically based on the following profiles:
1. Processes that are time-critical because they require results in a very short time after supplying data (OLTP, on-line transaction processing).
2. Processes that are run frequently and consume considerable computer resources (batch processing).
3. Processes that consume excessive amounts of computer resources, even if they are not run frequently (migration processing).
4. Processes that run beyond acceptable time frames making them ineffective or infeasible. For example, an order that takes several hours or a migration process that takes several weeks.
Because good performance is so crucial to software systems, I strongly encourage anyone who develops software to read this book. Dr. Liu presents a quantitative approach to understanding the concepts of system performance which he substantiates with practical examples. For those interested in mathematics, he includes the numerical analysis to explain the details behind the theory. But if you are new to performance or quantitative analysis, don't let the numbers overwhelm you. This book also introduces basic fundamentals for performance engineering. It utilizes common performance terminology, theory, and techniques that all developers should strive to understand. If you would like more information about performance analysis from a different perspective, then I would also highly recommend Performance Solutions - A Practical Guide to Creating Responsive, Scalable Software by Connie U. Smith and Lloyd G. Williams.
on August 27, 2009
I strongly recommend Dr Liu's "Software Performance and Scalability: A Quantitative Approach" for anyone involved in designing, constructing, testing, or managing delivery of enterprise applications. Having started as a software engineer and now leading development organizations for the past fifteen years I can attest and relate to the challenges that Dr Liu introduces in his text. Too often performance is an afterthought and implemented with adhoc processes and untrained resources.
Dr Liu delivers a well structured book that provides the quantitative foundation necessary to actually build performance into a solution and test and just as importantly, effectively tune or troubleshoot through instrumentation. The book provides a very effective blend of hard quantitative studies with practical examples yielding a book that is extremely thorough, easy to read, and informative.
A must read for any software designer, developer, tester, or leader accountable for delivering performant solutions. A definite must have for anyone in, or studying to be in, the performance engineering field.
on June 19, 2009
I have worked in the computer and software industry for over 14 years and Dr. Liu gets it right with his book, "Software Performance and Scalability: A Quantitative Approach".
Although this book presents a quantitative approach, it is also extremely practical for anyone that is in the business of developing software that demands high performance and scalability. For example, the author correctly identifies that it's not enough to establish a test team to ensure that software performs and scales well. It's often too late to address or too complex to fix when performance defects are found during the test cycle. You have to engineer performance into the product as early as possible.
This book provides an excellent foundation of knowledge that can be used by QA engineers, software developers, and management to help understand misconceptions about performance and scalability problems and how to avoid them. What's special about this book is that the author presents the information in an easy-to-read fashion without being superficial. At the same time, the author also provides a good balance of real-life case studies and computer science.
I'd recommend this book to anyone who works in the software industry and has any stake in delivering products with optimum performance and scalability.
on December 1, 2009
This is one book which can cater for any section of software development work force (school/high school or undergraduate or graduate or instructor or software rofessional). The book is aptly organized and have a upbeat flow of information from beginning to ending. I would say any performance test engineer should read this book before submitting a performance test plan for any application. I would say any software developer should read this book before designing any feature. With my 14 years of experience API profiling is most over looked or given least attention in the initial design and people do patch up at end of the release. The chapter on API profiling framework should be eye opener for many brains. Amdahl's law and the case study speaks volume about the author and his experience. Effective optimization and tuning techniques can be a very handy especially for applications which are years old and still exist. Overall it was a complete quantitative, qualitative, practical approach to performance and scalability. I highly recommend this book to anyone who is serious about performance and scalability. - Bala (Bangalore, India).
on September 14, 2009
As a person who has no exposure to enterprise performance before, I greatly benefited from reading this book. It is an excellent educational textbook for me to grasp the basic concepts of software performance and scalability and master the quantitative approach to tackle any enterprise performance issues. I really like those usecase studies following each chapter, which help readers a lot to apply the abstract theory learned from each chapter to real-life scenarios. As a person who favors mathematical approaches, it is a really enlightening reading for me to see how Dr. Liu applies the queuing theory and classical probability models to solve enterprise performance and scalability measurement problems. Strong recommended for anyone who are interested in performance subject. In his future editions, I hope that Dr. Liu can cover more performance issues on virtualization environments.
on November 16, 2014
I bought this book two years ago. I have purchased several books in performance testing. By comparing all of them this is the best so far. The first couple of chapters goes through the entire fundamental of performance testing from the hardware to software. The last chapters will explain performance tuning. Every single thing you learn from this book will come with the case study and real life example. I am not performance tester, but after completing this book I am 100% confident in performance testing. I was able to find out the performance testing team was performing the load test on QA database which is 10 times smaller production. Therefore, the result of testing were invalid. That is just the one of the example the book helped me on my carrier.
on November 24, 2009
Performance and scalability are not the easiest engineering aspects. It requires careful thought, sincerity and knowledge of some basic representation techniques. Dr Liu does a great job in presenting these aspects. How to vary parameters, how to rely more on facts rather than emails/thoughts/perceptions, how to measure performance step by step, the importance of small but smart utilities are some of skills that are really worth picking up as soon as possible.
If you are serious about creating an enterprise class product which requires high performance, high scalability - and still should be reliable, this book would be a good addition to your library.
on September 1, 2009
I have greatly benefited by reading this book. Provides an excellent coverage of the subject. The book provides both the analytical and measurement aspects of performance and scalability. There aren't many references that cover these subjects. I felt the author could have covered some areas like - Virtualization and its effects on performance and scalability and some real world web services deployment scenarios.