Customer Reviews

Patterns for Parallel Programming

5 star:		(1)
4 star:		(5)
3 star:		(0)
2 star:		(0)
1 star:		(1)

 

 

Parallel programming has been around for years, in many different forms. It has usually been a specialty for supercomputing number crunchers and for the occasional OS geek. Now that traditional, single-processor solutions are hitting the wall, Moore's Law must grow in new directions: multithreaded processos, multi-cores, multi-processors, and wilder exotica. The hardware is entering the market now, and the software community is scrambling to develop the necessry skills for parallel program development. This book gives a fair introduction to a large range of the techniques available.

After getting the reader oriented to the basics of parallel programming, the authors lay out four "design spaces," or families of related patterns. Within each space, the authors present a handful of patterns using a common and reasonably familiar format: name, problem addressed, context, forces acting on the design, the solution, and examples of the pattern's usage. They identify spaces named Finding Concurrency, Algorithm Structure, Supporting Structures, and Implementation Mechanisms. Of course, these topics overlap to some extend, especially in the interplay of algorithm design and explitable parallelism, or in langauge and API primitives that blur support mechanisms available with the implemenation choices available to the programmer. The authors show how the pieces come together in familiar applications, including molecular dynamics and medical imaging applications. Appendices sketch the basic programming constructs available in three of the major parallelism toolkits around: OpenMP, MPI, and Java.

Although valuable, this book has a number of weaknesses. For example, they cite the Cooley-Tukey FFT algorithm as a winning example of "Divide and Conquer." It's a great example, but perhaps not the one most useful for didactic purposes. The FFT algortithm is brilliant, and based on deep insight combined with total fluency in handling combinations of trig functions. Programmers writing parallel code will always benefit in proportion to their understanding of the algorithms, but I would find it discouraging to think that mastery and creativity at Cooley and Tukey's level were requisite. The authors skip over the kinds of fine-grained parallelism and communication available in emerging platforms, such as FPGA- and GPU-based accelerators. At the opposite end of the spectrum, they also gloss over many of the management issues in grid computing, where parallelism is extremely coarse-grained. I also have reservations about some of the system primitives they identify as patterns - one may as well say that a "for" loop is a pattern, even though it's a primitive in nearly all programming languages.

Despite flaws, the authors do reasonably well at introducing a wide range of basics for writing parallel programs. The interested beginner will need a lot more information to put the ideas to use, but this is a fair start.

//wiredweird

This is an excellent introduction to parallel computing. It presents patterns for discovering what can be parallized, what data structures can be used, how to choose algorithms. Patterns are demonstrated by good examples showing benefits and trade-offs of different solutions. There is also a brief, but very useful introduction to common implementations: OpenMP, MPI, and regular procedural approach demonstrated with Java.

Some caution about what this book is not: this is not a general parallel programming designand patterns book (as I expected from the title). The focus of this book is parallel computing (i.e. scalable _calculations_, often scientific). There is somewhat more to parallel _programming_ than this book covers.

Still, I found this book very good and useful, even though I expected broader coverage.

Normally design pattern books are things that you dip into rather than read end to end, simply because they can be very dry reading. Not this one - as long as you have an interest in parallel programming, reading this end to end should be easy. But that's not to say that you couldn't just dip in to the bits that are most applicable to your work - I'm sure you could.

Many of the examples given of where each pattern is used are in industry sectors other than where I work, but with such good descriptions of each pattern it is easy to picture where they are used other than the examples given and to identify where you have used them yourself without previously knowing that you were using a "named" pattern even if you have been doing it that way for years.

Much of the material in this book is stuff that is hard to find elsewhere. I've heard bits of it at Intel seminars or touched on in Intel books (e.g. the Threading Building Blocks book), but otherwise have not seen this stuff in print, even though many people (possibly unknowingly) are implementing the same ideas in code.

Excellent book. I've knocked one star off though, simply because the authors work on the premise that almost everyone is using one of OpenMP, MPI or Java. In practice, there are still an awful lot of people implementing such systems using C++ with either native threading APIs or third party libraries wrapping those threading APIs.

"Patterns for Parallel Programming" (PPP) is the outcome of a collaboration between Timothy Mattson of Intel and Beverly Sanders & Berna Massingill (who are academic researchers). It introduces a pattern language for parallel programming, and uses OpenMP, MPI, and Java to flesh out the related patterns.

The Good: this volume discusses both shared-memory and distributed-memory programming, all between one set of covers. It also makes use of a general-purpose programming language and is therefore of interest both to computational scientists who are interested in clusters, and to programmers interested in multiprocessors (these days that covers pretty much everyone). More generally, PPP offers valuable advice to those interested in robust parallel software design. The authors cover a number of topics that are an essential part of parallel-programming lore (e.g. the 1D and 2D block-cyclic array distributions in Chapter 5). In other words, they codify existing knowledge, which is precisely what patterns are supposed to do. To accomplish this, they make effective use of a small number of examples (like molecular dynamics and the Mandelbrot set). That allows them to show a specific problem as approached both from different design spaces, and also from different patterns within one design space. This book follows in the footsteps of the illustrious volume "Design Patterns" by the Gang of Four (GoF). In chapters 3, 4, and 5, Mattson, Sanders, and Massingill introduce a number of patterns using a simplified version of the GoF template. Despite the structural similarities between the two books, PPP is more readable than the GoF volume. This is probably because it introduces a pattern language ("an organized way of navigating through a collection of design patterns to produce a design"), not just a collection of patterns. Essentially, the writing style is a linear combination of narrative and reference: it can be read cover-to-cover, or not. Finally, the three appendices contain introductory discussions of OpenMP, MPI, and concurrency in Java, respectively. They can be read either as the need arises, or before even starting the book: though limited in scope, they are pedagogically sound.

The Bad: despite being easier to read from start to finish than the GoF classic, this book is still constrained by its choice to catalog patterns. As a result, the recurring examples lead to repetition, since they have to be re-introduced in each example section. Also, given that the book was published in 2004, a few implementation-related topics are somewhat out-of-date (e.g., OpenMP 3.0 was not around at the time). Importantly, the book predates the recent explosion of interest in general-purpose GPU programming, so it doesn't mention, say, texture memory. However, more fundamental things like data decomposition, which the book does explain, are related to any parallel programming environment. On a different note, even though the book is generally readable, from time to time the authors resort to the "just look at the code and figure it out" technique: the best-known example is in chapter 4 when they discuss ghost cells and nonblocking communication. Furthermore, even though the authors have been for the most part clearheaded when naming the different patterns, I found their decision to call two distinct patterns "Data Sharing" and "Shared Data" (in the "Finding Concurrency" and "Supporting Structures" design spaces, respectively) quite confusing and therefore unfortunate. Also, the Glossary is very useful, in that it explains many terms either discussed in the text (e.g. "False sharing") or not (e.g. "Copy on write", "Eager evaluation"), but it is far from complete (e.g. "First touch", "Poison pill", and "Work stealing", though mentioned in the main text, are not included in the Glossary). Finally, I think the authors overstate the case when they claim that "the parallel programming community has converged around" Java: Pthreads would have been an equally (if not more) acceptable choice.

All in all, this book provides a good description of many aspects of parallel programming. Most other texts on parallel programming either are class textbooks or focus on a specific technology. In contradistinction to such books, "Patterns for parallel programming" strikes a happy medium between focusing on principles and discussing practical applications.

Alex Gezerlis

This is a very good book: It will start teaching you how to think about parallel programming and will help you get started in this area.

Why only four stars you may ask? The trouble is that after over 40 years knowledge about parallel programming is still weak. The scientific computation folks have their (often heavy duty) tricks of the trade, but, as another reviewer pointed out, parallel computing is much more and is starting to address much broader areas.

This book will help you wade through the maze of confusion and will help you get oriented - that is of a huge help. Then you need to practice...

A little dry and a little repetitive but only to a small degree. The subject is (necessarily) approached from several different 'points of view' so some repetition is to be expected, but this should not discourage you from buying and reading this book, it is one of the most readable and affordable books on this topic. I highly recommend this book.

When I bought this book, I was hoping that the word 'patterns' in its title is only there to make it buzzword compliant. But sadly not. It is one of those completely useless pattern books, that long-windedly explain what should you do, without telling the how, and the why. Moreover all that explanations are about things, that you find out during the first day, when you actually sit down, and try to do some parallel programming.