on December 18, 2012
In this book and its predecessor, "Guerrilla Capacity Planning", Dr. Neil Gunther provides a useful introduction to the topic of measuring and modeling the scalability of parallel computer systems. The model that he advocates in his books is a useful starting point; however, this model fails to provide a sufficiently general basis for modeling the behavior of the wide variety of current parallel computer systems.
The "universal scalability law" to which he refers on p. 5 extends Amdahl's Law via the addition of a "queuing" term that models effects such as data exchange between parallel processes. And although Dr. Gunther suggests that this queuing term ought to grow linearly with the number of parallel processes, this queuing term depends on the specific communication architecture of the computer system and can grow non-linearly, for example, as log to the base two of the number of processes.
This logarithmic growth law can occur because one processor may not communicate directly with all other processors. Instead, one processor may send information to two other processors, and each of those two processors may send information to two more processors, and so forth. Therefore, in order to model the communication that occurs in such a communication cascade, the queuing term should grow as log(n).
Moreover, performance data that are obtained from current parallel computer systems do not always conform to Dr. Gunther's "universal" scalability "law" under other conditions. For example, a large volume of data that exceeds the capacity of the total cache memory when distributed across a few processors may well fit into total cache memory when distributed across a larger number of processors. Under these conditions, the scalability for the larger number of processors appears to grow "super-linearly" relative to the scalability of a few processors. However, on p. 5 of his book, Dr. Gunther states that "the data showed efficiencies of greater than 100%, which is simply not possible." Efficiencies of greater than 100% are not only possible, they are commonly observed due to cache memory effects. Thus, although Dr. Gunther's book is a useful introduction to the subject of measuring and modeling the behavior of parallel computer architectures, his universal scalability law should not be considered to be universal.
on May 17, 2012
My review of the 1st edition is below so I won't repeat it again.
The 2nd edition has several newly written introductory chapters to provide the needed
context for someone new to performance analysis, for example, the new introduction
to queueing theory is an wonderful exposition for a newbie. I only wished I had the
2nd edition when I bought the 1st edition.
Here is the link to more details on Dr. Gunther's site.
Now all we need is a Kindle edition.