Customer Reviews

Programming Massively Parallel Processors: A Hands-on Approach (Applications of GPU Computing Series)

5 star:		(3)
4 star:		(12)
3 star:		(5)
2 star:		(1)
1 star:		(0)

 

 

This book is a much better introduction to programming GPUs via CUDA than CUDA manual, or some presentation floating on the web. It is a little odd in coverage and language. You can tell it is written by two people with different command of English as well as passion. One co-author seems to be trying very hard to be colorful and looking for idiot-proof analogies but is prone to repetition. The other co-author sounds like a dry marketing droid sometimes. There are some mistakes in the codes in the book, but not too many since they don't dwell too long on code listings. In terms of coverage, I wish they'd cover texture memories, profiling tools, examples beyond simple matrix multiplication, and advice on computational thinking for codes with random access patterns. Chapters 6, 8, 9, and 10 are worth reading several times as they are full of practical tricks to use to trade one performance limiter for another in the quest for higher performance.

I think this book was written with the beginner in mind - if you're new to CUDA and having issues with understanding NVIDIA's documentation on the subject then this is the book to get. The author(s) took time to clarify and solidify some of the more difficult terms to understand e.g. memory bandwidth utilization, optimizing strategies but there are shortcomings in the book and two i could think of are typos (this really an issue cos it happens to every other book i've read) and the other would be using more examples to solidify concepts and illustrating them.

In a nutshell, a great beginner's book but not a handbook sort of book.

This book fills a nice gap between the SDK samples, technical specifications, and online course content. If you are just getting started with GPGPU computing, this book leads you smoothly through the computation model, hardware architecture, and the programming model required to take advantage of the hardware.

As others have pointed out, this is not a large book and fairly expensive. But, for the first book on the market it's surprisingly useful, effective, and readable. Definitely recommended for newcomers to the platform. Experienced GPGPU developers should only pick it up as a "hand out" for the people you need to train up, though.

As a beginning text this book has a significant advantage that beginning texts written for MPI, OpenMP, and so on don't have: there are 200 million CUDA-capable GPUs already deployed, and the odds are pretty good that most readers either have, or can readily get access to, a computer on which they can meaningfully learn parallel programming. If you are new to parallel programming and have access to a Tesla GPU, this book is a fine place to start your education. Readers already comfortable with parallel programming will find clear explanations of the Tesla GPU architecture and the performance implications of its hardware features, as well as a solid introduction to the principles of programming in CUDA, though they'll probably do a lot of skimming over the already-familiar basics.

One of the problems with many parallel programming books is that they take too general of an approach, which can leave the reader to figure out how to implement the ideas using the library of his/her choosing. There's certainly a place for such a book in the world, but not if you want to get up and running quickly.

Programming Massively Parallel Processors presents parallel programming from the perspective of someone programming an NVIDIA GPU using their CUDA platform. From matrix multiplication (the "hello world" of the parallel computing world) to fine-tuned optimization, this book walks the reader through step by step not only how to do it, but how to think about it for any arbitrary problem.

The introduction mentions that this book does not require a background in computer architecture or C/C++ programming experience, and while that's largely true, I think it would be extremely helpful to come into a topic like this with at least some exposure in those areas.

Summary: this book is the best reference I've found for learning parallel programming "the CUDA way". Many of the concepts will carry over to other approaches (OpenMP, MPI, etc.), but this is by and large a CUDA book. Highly recommended.

I enjoyed reading this book and I highly recommend it, but with a number of caveats.

First of all this is an introductory book to CUDA language for GPU programming. If you are already a CUDA practitioner, there is nothing you can learn from this book.

On the other side of the coin, if you have never written software before, you will be overwhelmed by the concepts, examples and the exercises in the book. The ideal audience for this book should have a minimum of a year's worth of hands-on programming experience in C. Those who only programmed before in scripting languages (like Perl, Python), and even those who programmed in Java may struggle with this book initially. CUDA is not a very high level language. If you are not used to manipulate data and code interaction at a low level and not used to clean-up after yourself, you need to learn C first before you tackle CUDA.

Assuming you know C, and assuming you are not an expert in parallel programming, this book is for you. However, be warned that the title of the book is misleading: The book does not deal with the general problem of "Programming Massively Parallel Processors". It talks about only one type of parallel environment "single program multiple data" or SPMD. In fact most of the book specifically deals with GPUs designed by Nvidia. You will not learn other parallel programming models such as SIMD or MIMD. Each of these models has its strengths and weaknesses, and which one does best depends on the application. It is my personal bias (not shared by everyone) that getting exposed to MIMD before SPMD is better for learning and understanding the synchronization concepts.

The book does not discuss one of the most important problems for parallel programming: "sequential equivalence" (SE). Proving that your program will produce identical results running on a sequential machine and in a parallel environment is very difficult in general. Most parallel programming environments (CUDA included) provide some set of tools to aid in this regard. With these tools, you may not get a proof, but you at least need to get a high probability of assurance that your program is SE. This is typically a topmost worry for any parallel programmer, and it should have been discussed in the book to expose the audience to a new set of problems one faces in the parallel realm.

Executive Summary - if you really want to dig into CUDA, go to the "CUDA Zone" on NVidia's web site. Also, this book concentrates on using CUDA on a single GPU.

I think the target audience of this book is an undergraduate taking a CUDA or parallel programming class with the university supplying access to a pre-installed CUDA development system.

This book is very readable (compared to the usual stuff programmers read). I particularly enjoyed the parts about GPU architecture and how various CUDA commands and structures map onto the architecture.

As far as "hands-on" ... ummmm ... no. The code snippets look like they were taken from a linux system (or maybe windows with posix in there) but there isn't any real discussion about setting up a programming environment. To me, a true "hands-on" book should have the reader creating and running a "hello world" app ASAP. This book doesn't deliver that.

An experienced professional (not just an MCSE or script kiddie) might enjoy this book, but not if the goal is to sling code under a deadline. In that case the CUDA zone is your friend.

the book is frankly overhyped and for $69b barely 200 pages book is overpriced. while i was looking for something solid to read beyond cuda programming guide, sort of text book, the book falls quite short it it.
99.9% of information listed there is already available on internet, it seems that book has been worked on in a rush. there are some interesting insights there about memory allocation and thread assignment i would have expected book to pack a lot more punch.

If you haven't done much low-level programming, and you plan to use the NVIDIA CUDA architecture, this book will help you get started. The pace is slow. Things like indexing into multi-dimensional spaces are explained in excruciating detail. However, the author does a good job of breaking down the levels of parallelism that the CUDA architecture can provide.

There's quite a bit of info online from the GPU conference last fall that will take you farther.

In the 1970s, supercomputers filled entire rooms. In the 1980s they were shrunk to the size of breadboxes and immersed in liquid coolant to keep them from melting down. Today a PC graphics card costing a few hundred dollars packs many times the raw processing power of the most expensive 1980s supercomputer.

The latest generation of computer graphics cards has an internal structure (architecture) that supports programming for non-graphical supercomputing. (Most supercomputing applications do resemble computational graphics in that they deal with geometry in two or more dimensions.) This is not a general-purpose computer architecture. A Graphics Processing Unit (GPU) is a massively parallel computer with hundreds or thousands of floating point processors and an operating system designed to manage tens or hundreds of thousands of threads. The architecture dictates how a program may be structured, and how the programmer's choices in structuring affect the actual computing power that can be extracted.

Programming Massively Parallel Processors presents the machine architecture for the recent generation of high-end NVIDIA GPUs. (NVIDIA is one of two companies specializing in GPUs and holding a major market share.)

The presentation reflects the book's origin as an academic course. Over and over it proceeds from theory (machine architecture) to application. Chapter questions focus on implications of the architecture and tools: What would happen if you did computation X in this way? Minimum time is spent on the mechanics of writing code and large examples are relegated to the appendices. It would be a mistake to look to this book for cookbook solutions. You are expected to master the whole magilla and nothing less. In fairness, the subject demands it.

The reader will need the equivalent of undergraduate engineering mathematics (vectors, calculus, etc.) but none of the "Discrete Math" topics often associated with computer science.

Most of the book concentrates on the CUDA extensions to the C programming language. This is a proprietary solution, not open source. Near the end of the book, other programming environments are described. A quick pass with a search engine indicates that there are open-source efforts for GPU programming, but they are still in their infancy.

This text's strengths are its clear and comprehensive presentation of a topic counter to the ordinary way of programming. Its weaknesses are the limited amount of source code in the main exposition and its focus on a solution limited to proprietary systems rather than an open-source solution.

One small disappointment: Amdahl's Law is a well-known statement of the limits of parallel performance improvement. The limit is explored here, but Gene Amdahl is not credited.

Kudos to the engineers at NVIDIA who designed the hardware architecture and operating system. It harnesses enormous parallelism with excellent flexibility and minimal (though significant) complexity to the programmer.