Parallel Programming with MPI [Paperback]

Peter Pacheco (Author)

Book Description

Publication Date: October 15, 1996 | ISBN-10: 1558603395 | ISBN-13: 978-1558603394 | Edition: 1

A hands-on introduction to parallel programming based on the Message-Passing Interface (MPI) standard, the de-facto industry standard adopted by major vendors of commercial parallel systems. This textbook/tutorial, based on the C language, contains many fully-developed examples and exercises. The complete source code for the examples is available in both C and Fortran 77. Students and professionals will find that the portability of MPI, combined with a thorough grounding in parallel programming principles, will allow them to program any parallel system, from a network of workstations to a parallel supercomputer.

* Proceeds from basic blocking sends and receives to the most esoteric aspects of MPI.
* Includes extensive coverage of performance and debugging.
* Discusses a variety of approaches to the problem of basic I/O on parallel machines.
* Provides exercises and programming assignments.

Editorial Reviews

Review

"...the detailed discussion of many complex and confusing issues makes the book an important information source for programmers developing large applications using MPI."
--L.M. Liebrock, ACM Computing Reviews

From the Back Cover

"...the detailed discussion of many complex and confusing issues makes the book an important information source for programmers developing large applications using MPI."
-- L.M. Liebrock, ACM Computing Reviews

A hands-on introduction to parallel programming based on the Message-Passing Interface (MPI) standard, the de-facto industry standard adopted by major vendors of commercial parallel systems. This textbook/tutorial, based on the C language, contains many fully-developed examples and exercises. The complete source code for the examples is available in both C and Fortran 77. Students and professionals will find that the portability of MPI, combined with a thorough grounding in parallel programming principles, will allow them to program any parallel system, from a network of workstations to a parallel supercomputer.

Features:

+ Proceeds from basic blocking sends and receives to the most esoteric aspects of MPI.
+ Includes extensive coverage of performance and debugging.
+ Discusses a variety of approaches to the problem of basic I/O on parallel machines.
+ Provides exercises and programming assignments.

See all Editorial Reviews

Product Details

• Paperback: 500 pages
• Publisher: Morgan Kaufmann; 1 edition (October 15, 1996)
• Language: English
• ISBN-10: 1558603395
• ISBN-13: 978-1558603394
• Product Dimensions: 7 x 0.9 x 10 inches
• Shipping Weight: 1.5 pounds (View shipping rates and policies)
• Average Customer Review: 4.4 out of 5 stars  See all reviews (15 customer reviews)
• Amazon Best Sellers Rank: #570,708 in Books (See Top 100 in Books)

More About the Author

Discover books, learn about writers, read author blogs, and more.

Most Helpful Customer Reviews

26 of 26 people found the following review helpful

5.0 out of 5 stars Very insightful! April 21, 2003

By Decio Luiz Gazzoni Filho

Format:Paperback

This book is a great introduction not only to MPI but to parallel programming itself. Initially, one learns essential concepts of parallel programming, including hardware and network topologies. An MPI version of ``Hello World!'' gently introduces the reader to the mechanics of running MPI programs (if you haven't set up your parallel computing environment yet, this may well be the most time consuming chapter of the book!) In what follows, the author lays out the basics of each topic in parallel programming, followed by MPI-specific details (i.e. which MPI function is used in that situation). Particularly worthy of praise is the example-driven approach, whereby in each chapter a problem is presented (like numerical integration or matrix multiplication), and the author shows how to solve them with techniques taught in that chapter. Some problems are rehashed in other sections/chapters, in order to show different PP techniques to solve them. The book doesn't suffer from the black-box syndrome of hiding implementation details: the reader is usually shown the inner workings of, say, each communication technique, while being advised to use MPI's built-in functions instead of rolling out his own versions.

Topics covered include collective communication methods, I/O, debugging, design, performance, parallel algorithms and libraries, and advanced topics in communication. A handy appendix on MPI functions wraps up the book.

As an aside, I will be forever grateful to this book due to its clear exposition of the butterfly broadcast method, which helped me understand the Cooley-Tukey iterative version of the FFT (which uses the same butterfly idea), something I had been struggling with for a couple of years. I would have gladly bought the book for this explanation alone.

13 of 13 people found the following review helpful

4.0 out of 5 stars Comments for Fortran Users March 4, 1997

By A Customer

Format:Paperback

As the book description indicates, the text is based on the C language. The Fortran MPI function calls are slightly different (syntatically) than the C equivalents. Fortran users should be aware that these differences are not discussed in the text. However, the sample programs from the text are available in Fortran from the author's web page (referenced in the book).

Aside from this, Parallel Programming With MPI is an easy reading book which introduces the material through working examples. The book also contains a nice introduction to high performance computing and optimization issues. I still have a lot to learn , but I'm glad I bought it!

12 of 12 people found the following review helpful

4.0 out of 5 stars Great for MPI beginners November 9, 2006

By wiredweird HALL OF FAMETOP 500 REVIEWER

Format:Paperback

Pacheco's book is a strong, gently paced introduction to a very complex API. MPI, the message passing interface, is the most common coordination tool for parallel scientific computing. When a Blue Gene programmer has 1,000 or 100,000 processors all working on different parts of one calculation, there's a big problem in getting partial results from where they're computed to where they're needed. That's what MPI is for.

When the problem is so complicated, the solution is also complicated. Pacheco does a good job of breaking MPI down into digestible pieces, starting with the basic send and receive primitives that you'll use most often. He presents each new part of the API in terms of some problem to be solved, keeping a concrete and practical tone to this book. He gradually adds more pieces in terms of more practical exercises: broadcasts and reductions, scatter and gather, data structuring, communicators, and asynchronous IO.

Along the way, Pacheco introduces algorithms that even experienced uniprocessor programmers may not be familiar with, including bitonic sorting and Fox's algorithm for matrix multiplication. This isn't gratuitous intellectual showmanship. It's a pointed demonstration that, when communication barriers change the computation landscape, old paths to solutions may not be the best routes any more. After finishing with the MPI API itself, Pacheco presents debugging techniques and common kinds of support libraries, as well as basic techniques for analyzing the potential and actual acceleration possible for a given problem.

If you're serious about MPI, you'll need the official standard for understanding the fussy details of these complex APIs. That's a pretty brutal way for a beginner to get going, though. This introduces not only the basic concepts of MPI, but also the basics of how to think about highly parallel programming. And, as multi-threaded multi-core multi-processor systems become common, that's an increasing percentage of all programming.

//wiredweird