CUDA Application Design and Development [Paperback]

Rob Farber (Author)

Book Description

Publication Date: November 14, 2011 | ISBN-10: 0123884268 | ISBN-13: 978-0123884268 | Edition: 1

As the computer industry retools to leverage massively parallel graphics processing units (GPUs), this book is designed to meet the needs of working software developers who need to understand GPU programming with CUDA and increase efficiency in their projects. CUDA Application Design and Development starts with an introduction to parallel computing concepts for readers with no previous parallel experience, and focuses on issues of immediate importance to working software developers: achieving high performance, maintaining competitiveness, analyzing CUDA benefits versus costs, and determining application lifespan.

The book then details the thought behind CUDA and teaches how to create, analyze, and debug CUDA applications. Throughout, the focus is on software engineering issues: how to use CUDA in the context of existing application code, with existing compilers, languages, software tools, and industry-standard API libraries.

Using an approach refined in a series of well-received articles at Dr Dobb's Journal, author Rob Farber takes the reader step-by-step from fundamentals to implementation, moving from language theory to practical coding.

• Includes multiple examples building from simple to more complex applications in four key areas: machine learning, visualization, vision recognition, and mobile computing
• Addresses the foundational issues for CUDA development: multi-threaded programming and the different memory hierarchy
• Includes teaching chapters designed to give a full understanding of CUDA tools, techniques and structure.
• Presents CUDA techniques in the context of the hardware they are implemented on as well as other styles of programming that will help readers bridge into the new material

Editorial Reviews

Review

The book by Rob Faber on CUDA Application Design and Development is required reading for anyone who wants to understand and efficiently program CUDA for scientific and visual programming. It provides a hands-on exposure to the details in a readable and easy to understand form. Jack Dongarra, Innovative Computing Laboratory, EECS Department, University of Tennessee

GPUs have the potential to take computational simulations to new levels of scale and detail. Many scientists are already realising these benefits, tackling larger and more complex problems that are not feasible on conventional CPU-based systems. This book provides the tools and techniques for anyone wishing to join these pioneers, in an accessible though thorough text that a budding CUDA programmer would do well to keep close to hand. Dr. George Beckett, EPCC, University of Edinburgh

With his book, Farber takes us on a journey to the exciting world of programming multi-core processor machines with CUDA. Farber's pragmatic approach is effective in guiding the reader across challenges and their solutions.   Farber's broader presentation of parallel programming with CUDA ranging from CUDA in Cloud and Cluster environments to CUDA for real problems and applications helps the reader learning about the unique opportunities this parallel programming language can offer to the scientific community. This book is definitely a must for students, teachers, and developers! Michela Taufer, Assistant Professor, Department of Computer and Information Sciences, University of Delaware

Rob Farber has written an enlightening and accessible book on the application to CUDA for real research tasks, with an eye to developing scalable and distributed GPU applications.  He supplies clear and usable code examples combined with insight about _why_ one should use a particular approach.  This is an excellent book filled with practical advice for experienced CUDA programmers and ground-up guidance for beginners wondering if CUDA can accelerate their time to solution. Paul A. Navrátil, Manager, Visualization Software, Texas Advanced Computing Center

The book provides a solid introduction to the CUDA programming language starting with the basics and progressively exposing the reader to advanced concepts through the well annotated implementation of real-world applications. It makes a first-rate presentation of CUDA, its use in the implementation of portable and efficient applications and the underlying architecture of GPGPU/CPU systems with particular emphasis on memory hierarchies. This is complemented by a thorough presentation both of the CUDA Tool Suite and of techniques for the parallelisation of applications. Farber's book is a valuable addition to the bookshelves of both the advanced and novice CUDA programmer. Francis Wray, Independent Consultant and Visiting Professor at the Faculty of Computing, Information Systems and Mathematics at the University of Kingston

At a brisk pace, "CUDA Application Design and Development" will take one from the basics of CUDA programming to the level where real-time video processing becomes a stroll in the park. Along the way, the reader can get a clear understanding of how the hybrid CPU-GPU computing idea can be capitalized on, and how a 500-GPU configuration can be used in large scale machine learning problems.  Wasting no time on obscure issues of little relevance, the book provides an excellent account of the CUDA execution model, memory access issues, opportunities to increase parallelism in a program, and how advanced profiling can squeeze performance out of a code.  Rob provides a snapshot of everything that is relevant in CUDA based GPU computing in a style honed through a long series of Dr. Dobb's articles that have delighted scores of CUDA programmers.  His followers will be delighted once again. Dan Negrut, Associate Professor, University of Wisconsin-Madison, NVIDIA CUDA Fellow

About the Author

Rob Farber has served as a scientist at the Irish Center for High-End Computing, U.S. national labs in Los Alamos, Berkeley, and the Pacific Northwest, and external faculty at the Santa Fe Institute. His articles have appeared in Dr. Dobb's Journal and Scientific Computing, among others.

Product Details

• Paperback: 336 pages
• Publisher: Morgan Kaufmann; 1 edition (November 14, 2011)
• Language: English
• ISBN-10: 0123884268
• ISBN-13: 978-0123884268
• Product Dimensions: 7.5 x 1.1 x 9.2 inches
• Shipping Weight: 11.2 ounces (View shipping rates and policies)
• Average Customer Review: 3.2 out of 5 stars  See all reviews (5 customer reviews)
• Amazon Best Sellers Rank: #639,615 in Books (See Top 100 in Books)

More About the Author

Rob Farber has served as a scientist worldwide at the Irish Center for High-End Computing, U.S. national labs in Los Alamos, Berkeley, and the Pacific Northwest, and external faculty at the Santa Fe Institute. A co-founder of two successful start-ups and a consultant to Fortune 100 companies, his articles have appeared in Dr. Dobb's Journal, The Code Project, and Scientific Computing, among others. He recently completed his book "CUDA Application Design and Development", which is now available for purchase.

Most Helpful Customer Reviews

11 of 13 people found the following review helpful

5.0 out of 5 stars Details on Memory Coalescing and Linux Tools (Profiler + GDB) January 12, 2012

By Yeoching Thia

Format:Kindle Edition|Amazon Verified Purchase

I desperately need some insights about CUDA memory performance, and there are two chapters discussing memory coalescing which are very useful to me. I need to write a very simple kernel to convert 3 bytes RGB into 4 bytes RGBA so that OpenGL could render it correctly, but what I have done naively is to spawn too many threads and by doing so it slow down the code so much, so much so that I wondered what I have really done.

Later I came to know that CUDA memory utilization is my bottleneck. Being a traditional programmer, I never pay attention to how I access memory. Now with the insights on memory coalescing in this book, I get the real education about how GPU really works.

On the other hand, I use CUDA under Linux, it is important to know that CUDA has provided a profiling tool to diagnose memory bottleneck, and cuda-gdb could use to step through my CUDA kernel. This book also spend quite some details to illustrate how to use nvcc. This is also up to date to discuss Fermi architecture (Compute Capability 2.x with L1, L2 cache) and CUDA Toolkit 4.0 .

Get this book if you need real insights from a real practitioner on CUDA. Also for the entry, you should get CUDA by Example (Jason Sanders and Edward Kandrot) as well. You would be so glad as I did.

5 of 5 people found the following review helpful

2.0 out of 5 stars uneven quality - i don't think it' s worth the price December 13, 2012

By S. Saha

Format:Paperback

The book is a mixture of good and bad. On the good, the author knows a great deal about the subject and does a pretty good job of explaining GPU architecture. Chapters 1 and 4-7 are very useful and definitely worth reading for any cuda developers. Chapter 12 is also fun.

The bad- I think large parts of the book are straight from other works by the author especially from a series of articles he wrote for Dr. Dobb's. The summary for Chapter 8 even refers to the chapter as "this article". So, often concepts are explained twice or used before they're explained in a later chapter e.g. thrust::reduce is used from chapter 1, finally explained in chapter 8 i believe.

Chapters 2 and 3 should be avoided at all costs especially all the code. Read it at your own risk. The explanation of it is fragmented and random design choices and variable naming just adds to the confusion. The code in the other chapters is better. As a rule of thumb, if the code takes up more than 1 page skip it and any discussion of it - there's no benefit to reading it.

There's also an extremely uneven assumption of reader knowledge throughout the book - the author at one point spends a page and a half explaining why testing is important. It takes him 3 lines to explain that someone pointed out his code could come up with a number so large that the variable couldn't hold it. Why didn't he just say "overflow" ? This is just after discussing pre-processor directives and simplex methods!

So yeah, read the Dr. Dobb's articles if they're freely available. There's some good content in this book but a lot of it can be found in the cuda docs and websites/blogs. Given those resources, I feel this book isn't worth the price.

2 of 2 people found the following review helpful

4.0 out of 5 stars Easy to read and clear enough to follow December 3, 2012

By YL

Format:Paperback|Amazon Verified Purchase

It's a rookie level CUDA book. It includes the info you need to know as a CUDA starter and you can start to program very soon. At meantime, I suggest read David B. Kirk's <<Programming Massively Parallel Processors>> as well, which gives more fundamental understanding of how CUDA works at a higher level.