Customer Reviews

Parallel Programming with MPI

5 star:		(8)
4 star:		(5)
3 star:		(0)
2 star:		(0)
1 star:		(0)

 

 

This book is a great introduction not only to MPI but to parallel programming itself. Initially, one learns essential concepts of parallel programming, including hardware and network topologies. An MPI version of ``Hello World!'' gently introduces the reader to the mechanics of running MPI programs (if you haven't set up your parallel computing environment yet, this may well be the most time consuming chapter of the book!) In what follows, the author lays out the basics of each topic in parallel programming, followed by MPI-specific details (i.e. which MPI function is used in that situation). Particularly worthy of praise is the example-driven approach, whereby in each chapter a problem is presented (like numerical integration or matrix multiplication), and the author shows how to solve them with techniques taught in that chapter. Some problems are rehashed in other sections/chapters, in order to show different PP techniques to solve them. The book doesn't suffer from the black-box syndrome of hiding implementation details: the reader is usually shown the inner workings of, say, each communication technique, while being advised to use MPI's built-in functions instead of rolling out his own versions.

Topics covered include collective communication methods, I/O, debugging, design, performance, parallel algorithms and libraries, and advanced topics in communication. A handy appendix on MPI functions wraps up the book.

As an aside, I will be forever grateful to this book due to its clear exposition of the butterfly broadcast method, which helped me understand the Cooley-Tukey iterative version of the FFT (which uses the same butterfly idea), something I had been struggling with for a couple of years. I would have gladly bought the book for this explanation alone.

As the book description indicates, the text is based on the C language. The Fortran MPI function calls are slightly different (syntatically) than the C equivalents. Fortran users should be aware that these differences are not discussed in the text. However, the sample programs from the text are available in Fortran from the author's web page (referenced in the book).

Aside from this, Parallel Programming With MPI is an easy reading book which introduces the material through working examples. The book also contains a nice introduction to high performance computing and optimization issues. I still have a lot to learn , but I'm glad I bought it!

Pacheco's book is a strong, gently paced introduction to a very complex API. MPI, the message passing interface, is the most common coordination tool for parallel scientific computing. When a Blue Gene programmer has 1,000 or 100,000 processors all working on different parts of one calculation, there's a big problem in getting partial results from where they're computed to where they're needed. That's what MPI is for.

When the problem is so complicated, the solution is also complicated. Pacheco does a good job of breaking MPI down into digestible pieces, starting with the basic send and receive primitives that you'll use most often. He presents each new part of the API in terms of some problem to be solved, keeping a concrete and practical tone to this book. He gradually adds more pieces in terms of more practical exercises: broadcasts and reductions, scatter and gather, data structuring, communicators, and asynchronous IO.

Along the way, Pacheco introduces algorithms that even experienced uniprocessor programmers may not be familiar with, including bitonic sorting and Fox's algorithm for matrix multiplication. This isn't gratuitous intellectual showmanship. It's a pointed demonstration that, when communication barriers change the computation landscape, old paths to solutions may not be the best routes any more. After finishing with the MPI API itself, Pacheco presents debugging techniques and common kinds of support libraries, as well as basic techniques for analyzing the potential and actual acceleration possible for a given problem.

If you're serious about MPI, you'll need the official standard for understanding the fussy details of these complex APIs. That's a pretty brutal way for a beginner to get going, though. This introduces not only the basic concepts of MPI, but also the basics of how to think about highly parallel programming. And, as multi-threaded multi-core multi-processor systems become common, that's an increasing percentage of all programming.

//wiredweird

The book is written very well and goes through how the MPI functions work and all their parameters in pretty fine detail. He even talks about the simpler things many books overlook. The only thing I wish was in this book, and this has nothing to do with the quality of the book, is some C++ reference or talk of the C++ MPI calls. This book is written with examples in C only (I think FORTRAN one may be available online), but the theory he teaches and the design of the programs will work for any language, of course. It would be nice to have a good reference chapter that lists the FORTRAN MPI functions and the C++ ones too though.

Very seldom do I find a programming book being as detail as this one. Definitely worth reading if you are new to MPI.

I read this book over the past week, covering chapters 1 through 6, skimming 7-10, and reading 11 through the final chapter 16. Its basically applied MPI programming, done up very well and clearly, starting with architectural history & motivations and leading into a simple numerical integrator example program in chapter 4 (chapter 3 was the MPI `hello, world'). The coding used is C, and I wrote my own integrator after finishing chapter 4 to also explore floating point numbers in calculations, loop control, and to integrate arbitrary functions on arbitrary intervals with adjustable resolution. The integrator is developed more fully throughout the book wherein MPI performance issues of the original design are pointed out and polished off as additional MPI functions and techniques are introduced. Some of these techniques included tree-structured initialization & broadcasts, data communication optimizations (such as derived types, packing / unpacking, etc), and guidance as to when certain techniques would be more useful than others offered by MPI. Communications are further advanced later in the book where the important non-blocking forms and more advanced concepts are brought to light & illustrated. Empirical analysis of algorithmic performance occupies two full chapters and is very interesting, including a detailed look into Amdahl's law. Its an excellent example of why we should keeps our eyes open in research. Program design & troubleshooting are also covered, but I only skimmed those chapters. Several parallel algorithms and some parallel libraries are also treated well in the text. At only 362 pages, (minus appendices) this book is a quick read and a superb lab manual. If you are a software developer just now getting into MPI, this book will certainly accelerate you onto MPI with the confidence that you can do anything with it. Just give it a week of your time. Its the perfect self-study manual.
5-stars

A necessary volume for anyone working with MPI. After reading the books by Snir and Gropp this is a great follow-on which demystifies a few complex concepts in MPI.

This book is an excellent walk-through of MPI programming, with extensive coverage of more general topics in parallel programming. It makes an excellent companion to a course on parallel programming, and is also just generally a good read.

I was looking for a good introduction into MPI to parallelize some software I had written. Somehow, no online resources seemed to cover the topic well, so $30 seemed like a worthwhile investment, compared to my time. I got this book and the more recent one by Quinn (PP in C with MPI and OpenMP). This one's the hands-down winner. I basically scanned it in bed for three nights, and two weeks later my code is running like a charm. Just the right mix of reference and tutorial, very little distraction, and a pleasant read throughout.

Highly recommended.

For the undergraduate or graduate student who programs in C, this book is a well-written and informative introduction to MPI programming. However, for the old-school Fortran programmer, the book offers little guidance. Typical students today will be writing code in C (or its derivatives), and so the choice of programming language is logical, but I have still spent a lot more time online looking up MPI Fortran code and syntax than using the book.

Overall, the text itself is solid and readable. A Fortran version of the text would be nice, but the online code snippets are good enough to get you started.