Customer Reviews

The Art of Concurrency: A Thread Monkey's Guide to Writing Parallel Applications

5 star:		(2)
4 star:		(5)
3 star:		(1)
2 star:		(2)
1 star:		(0)

 

 

This book is kind of a dull read for (in my opinion) interesting material. The writing style is informal, but self-importantly so (lots of "I did this" and "I've said this before"). Even discounting that, the writer cannot make the subject very interesting. Partly because he eschews figures or flowcharts or itemized steps for walls of text, partly because the writing itself is disjointed and not very good. Remember, informal != good.

Plus, the code is sloppy. Almost everywhere a main routine has a loop pthread_create'ing new threads, "new" is used to allocate and free is used to deallocate a pointer. This is the most egregious one; one can take issue with a lot of others (for example, why is the code C++ if 99% of it is C? Why is the pointer allocated in the main thread but freed in the worker thread?).

The thing I liked about the book is that it covers a lot of relevant topics, including modern ones such as MapReduce. It should reward someone with the patience to overlook its flaws.

If you are a relative beginner, and not dealing with inherited code, then this book provides a happily patronising and sloppily-coded "taster" introduction to writing algorithms in OpenMP or Intel Thread Building Blocks, with a little coverage of pThreads, and a very small amount of Windows Threading. Java, Erlang, and CUDA/OpenCL are completely absent. Compiler support was sparse and the C++0x standard not ratified at time of writing, so no examples are given of the way lambda functions make it easier to write and use TBB algorithms.

Thread-local storage is mentioned, but no example code is given, so there is insufficient information to actually use it -- the same is true for many other indexed items. (TLS has four entries in the index, but the useful paragraph on p43 is not in the index at all).

If you already know enough to use the libraries it covers, then the only useful thing from this book is going to be the hints, tips and experiences. Unlike a previous good reviewer, these are what I think should be the golden core of the book (there are plenty of better books on parallel algorithms). But like the previous reviewer, I was left extremely disappointed by an opportunity lost. With hints and tips associated with arbitrary algorithms and scorecards throughout the book, most of their potential benefit is lost.

Debugging tools are summarily dealt with on pages 258 and 259, and profiling tools take up the following three pages. Verification and correctness do not make it to the index, although there is a single entry for "testing for correctness", which refers to Design Step 3 on page 10, which says that you should do this, but simply refers you to the six pages of the tools chapter above. Step 3 finally exits reality saying: "With a good solid design, you should be able to keep the number of threading or logic errors to a minimum, so not much verbiage is spent on finding or correcting errors in [concurrent] code". Chapter Three "Proving Correctness and Measuring Performance" does spend a few pages on correctness, but mainly refers to the Ben-Ari book. On actually measuring performance it says: "For serial applications, you can measure an application's speed using a stopwatch. Simply time the run from start to finish." The author then explains how you can measure performance improvement by comparing the time taken for the first version with that of subsequent versions. "It is the same with concurrent programs."

Sample sentences that give you a flavour for the author's style: "While I've never written one or seen an implementation of one, creating a barrier that could work with a different number of threads each time it was used sounds like such a complex and daunting task", "I recommend reading the chapters from Ben-Ari's text (See if you can get it out of the library..", I've not had the chance to work with the task constructs in OpenMP, so there may be other details...", "I won't create a threaded version of this code, but you're welcome to give it a try on your own..", "und so veiter [sic] (or 'and so on' for my English readers)". "

The author is a course architect for Intel specializing in multithreaded progamming and training.

This is a new book on concurrent programming that splits the difference between academic tomes on the subject and cookbook code dump type texts.

The author assumes that readers have some basic knowledge of data structures and algorithms and asymptotic efficiency of algorithms (Big-Oh notation) that is typically taught in an undergraduate computer science curriculum. Readers familiar with Introduction to Algorithms should do the trick. He also assumes that the reader is an experienced C programmer (he uses C throughout the book) and knows something about OpenMP, Intel Threading Building Blocks, POSIX threads, or Windows Threads libraries and has a good idea of which of these tools will be used in their own situations. The author does not focus on using one programming paradigm here, since, for the most part, the functionality of these overlap. Instead he presents a variety of threading implementations across a wide spectrum of algorithms that are featured in the latter portion of the book.

The current product description does not show the table of contents, so I do that next:

Chapter 1, "Want to go faster" anticipates and answers some of the questions you might have about concurrent programming. This chapter explains the differences between parallel and concurrent, and describes the four-step threading methodology. The chapter ends with some background on concurrent programming and some of the differences and similarities between distributed-memory and shared-memory programming and execution models.

Chapter 2, "Concurrent or Not Concurrent" collects a lot of the author's wisdom on initial approaches that apply to a large percentage of code you're likely to encounter. Also, the author lets you know that not every bit of computation can be made concurrent, no matter how hard you try. There are examples of the kinds of algorithms and computations that are not friendly to concurrency in Section 2.2. When any of those examples can be modified to allow for concurrent execution, hints and tips are included about how to do that.

Chapter 3, "Proving Correctness and Measuring Performance" takes a look at topics related to the final two steps of the threading methodology. The first is knowing when your concurrent algorithms will run correctly or at least having a good idea that you've done a good job of designing an error-free concurrent algorithm. The second topic covers some of the ways you can measure how well your concurrent code is executing in parallel.

Chapter 4, "Eight Simple Rules" should give you more success in writing the best and most efficient threaded implementation of your applications. In upcoming chapters, when discussing the design and implementation of specific algorithms, the author drops in a reference to one or more of these eight rules.

Chapter 5, Threading Libraries, reviews some of the details of the threading libraries used in subsequent chapters to implement the algorithms. The author assumes that you are already familiar with at least one of these threading methods. Proficiency is not expected, but you should have at least tried your hand at some threaded coding examples before.

If you are unfamiliar with any of the threading libraries, this chapter should provide you with enough details to understand the algorithms implemented with such a library and any library-specific features that are used. Otherwise, this should be a review. You can skip over this chapter and come back when you might have a question about syntax or threading without fear of loss of continuity.

Chapter 6, is entitled "Parallel Sum and Prefix Scan". Summing the elements of an array or finding all partial sums of the elements in an array are basic algorithmic problems. The solution to these problems is easy to describe in a single sentence or two. The concurrent versions of these algorithms, known as parallel sum and prefix scan respectively, are simple and easy to understand. Since they are so simple, these problems have been extensively analyzed and are used as bellwether algorithms within the parallel programming community. Description, design, analysis, and implementation of these two algorithms is intended to prepare you for the rest of the algorithms contained in the text.

Chapter 7, "Map Reduce", is about an algorithmic framework of the same name, like divide-and-conquer or backtracking, rather than a specific algorithm. The pair of operations, map and reduce, is found in LISP and other functional languages. MapReduce has been getting a lot of interest lately as an algorithmic framework that can be executed concurrently.

Chapter 8, "Sorting" examines concurrent sorting. First considered are compare-exchange sorts such as bubble sort. These are sorting algorithms that use the results from comparing two keys to determine the relative order of the elements with those keys. Movement of data items will be based on those results and will be the exchange of the positions of the two items under consideration. The final algorithm considered is radix sort, which compares bits within keys to determine movement of data.

Chapter 9, "Searching", discusses two algorithms you can use to search through a collection of data and shows how to make them concurrent to decrease the time needed to locate items of interest. For simplicity, it is assumed that all keys within a data set to be searched are unique. Strategies for dealing with multiple duplicate keys are only mentioned briefly.

Chapter 10, "Graph Algorithms", talks briefly about what a graph is and how it is represented and then how it is traversed using both depth-first and breadth-first search algorithms. It is very well illustrated.

Chapter 11,"Threading Tools", mentions some debugging and performance tools that you can use on threaded applications. The set of tools covered in this chapter is not exhaustive and is meant to show you ways to track down problems in a much quicker manner than if you performed the hunt yourself.

All in all, this is a very useful yet not over-academic book. I would say that you should probably have read something along the lines of Using OpenMP: Portable Shared Memory Parallel Programming (Scientific and Engineering Computation), or at least read that book in parallel with this one. If you're not interested in OpenMP, you would of course read a book on a different technique for concurrent programming.

This is a gentle book for readers without a CS background, as the author said in the preface "I work for a computer technology company, but I'm the only computer science degree-holder on my team", Clay has been involved with parallel computation for over twenty years, so he is certainly qualified to write such a book.

My favorite chapters are 2, 3 and 4, as other reviewers commented, this book has a very informal style, something goes like this:

"For those of you who haven't immediately curled up into a fetal position or flopped onto the floor kicking and screaming at the thought of formal proofs of algorithms (and for those of you who did but have now picked the book up again), let me assure you that I consider this a vital part of concurrent algorithm design and it's probably not as bad as you think. The more time you spend creating correct concurrent algorithms from the get-go, the less time you will spend chasing down errors that only show up on those Thursdays when the dates are prime numbers."

But Clay does make his point very clear, so overall I think the book very well serves its purpose.

"The Art of Concurrency: A Thread Monkey's Guide to Writing Parallel Applications", by Clay Breshears, is a very good book about writing non-embedded, application type, parallel applications and translating applications from serial to concurrent.

Between an introductory and intermediate level book on concurrency, assuming "some basic knowledge of data structures and algorithms and asymptotic efficiency of algorithms (Big-Oh notation)", as stated in his preface. The first five chapters were very good in explaining concurrency: introduction, concurrent or not-concurrent, measuring performance, rules for design, and libraries. The next five chapters examined concurrent algorithms: parallel scan and prefix scan, mapreduce, sorting, searching, and graph applications. The final chapter described threading tools.

My favorite chapters were the introduction, rules for design, and algorithm chapters. The sections "THE TWO-MINUTE PRIMER ON CONCURRENT PROGRAMMING" and "Four Steps of a Threading Methodology" were great; as was the chapter on rules for design.

The code is in C, with plenty of examples going from serial to concurrent. The code is not really geared towards one threading library; I used OpenMP in Visual Studio 2008 CLI/C++.

The book is an enjoyable read, with lots of figures (figures that an instructor would love to display) and code to keep a reader's interest.

Another great feature of the book was its nine page comprehensive glossary relating to concurrency. Though the index was not as large as I would like, the book's layout make's it not difficult in finding information (In my experience, a comprehensive index is key when using any type of technical book as a reference at work).

Sloppy piece of work. Content overview is good but the code is sloppy and writing style is bad. I would not recommend this.

This book provides an introduction to concurrency of threads and tasks with shared memory, not SIMD concurrency (such as with a vector unit or GPU) or message passing with distributed memory (such as with a cluster). The first half of the book covers the concepts involved in multithreaded programming and how to parallelize algorithms. The second half covers particular parallel algorithms with implementations in C++.

The first several chapters were confusing because they made no reference to the memory model, which is odd for a book that covers shared-memory concurrency. In particular, the book assumed that when one thread writes a global variable, that all other threads would see the new value immediately. In reality, compilers can reorder results and store results in registers unless variables are marked as volatile or there is an explicit memory barrier.

The book assumes the reader has experience with a task or thread library. It does introduce OpenMP, Intel Threaded Building Blocks, and Pthreads, but the book could offer more explanation for readers not experienced with these implementations of concurrency. The book does not use C++ lambda expressions, which are useful when using Intel TBB, and it does not cover Java concurrency.

The C++ code listings in the book are quite helpful and can be understood by any C programmer, but they could be better. One nitpick in particular is that the code for Quicksort always picks the leftmost element as the pivot, which will result in a stack overflow when sorting a large, nearly sorted list. The book provides no coding exercises, which I often find helpful when learning a new language or library.

"The Art of Concurrency," by Clay Breshears, is an outstanding book on concurrent algorithms and multi-threaded programming. Breshears hits the sweet spot between rigor and readability, and manages to make a terrifically difficult subject accessible, if not the to layman, at least to motivated and experienced developers.

While I cannot imagine a more accessible book on the subject, make no mistake: reading this book requires some serious head work and commitment. If you stay the course, you will review the details of algorithms that you vaguely remember (quick sort; Floyd's algorithm) and some that may be entirely new to you (PRAM). But the rewards are significant: not just a sense of accomplishment, but confidence that you could transform a gnarly serial algorithm into a parallel version that would run rings around the original. There is also the intangible benefit of having been forced to look at problems sideways, with the result that next time you need to engineer a solution, you will be less stuck in a rut and more apt to find an elegant, out-of-the-box solution.

Although there are some eminently practical chapters on searching for potential race conditions, this book is not a cookbook of thread solutions: few developers will need to implement parallel binary searches, parallel sorts, or parallel graph traversal. Yet reading this book should make a developer better prepared to resolve more mundane concurrency issues. And it goes down easy: the author has a charming voice, loves his area of expertise, and takes delight in leading the reader through this subject.

Without a doubt, this is one of the best books on programming that I have read.

With CPUs growing in power by adding additional core as opposed to just getting "faster", learning how to take advantage of parallel programming is a must. The book "The Art of Concurrency: A Thread Monkey's Guide to Writing Parallel Applications" by Clay Breshears works great as a reference and guide for determining when parallelization may be possible, how it could be done, and what to look out for.

The book introduces the reader to parallel programming with a set of useful rules and guidelines to follow to plan for optimizing algorithms by distributing workloads through concurrent programming. Much of the remainder of the book enumerates some common tasks and how to make them concurrent. One of the best parts of the common task listing is the scorecard for evaluating the quality of the implementation. The scorecard includes the useful performance factors of "efficiency" and "scalability". It also includes the important details of "simplicity" and "portability", important when evaluating methods for maintainable code.
The common threading tools OpenMP, Intel Thread Building Blocks, and POSIX threads are described in the early chapters and sprinkled throughout the examples in a useful manner, providing exposure to different ways one might implement concurency; not everyone needs to re-invent the wheel when optimizing tasks.

I thought this was an excellent book. The previous reviewer was correct, it's an outstanding overview of how to create parallel algorithms without being too academic. My only real complaint is that the font was too small. Perhaps it's my age (I'm 46), but frankly if I'm going to spend hours with a book and use it as a reference, I want the font to be easy on my eyes. If in the second edition, the author makes the font bigger and adds a few more pages, then this will *the* book to read on parallel algorithms.