Superforecasting Paperback

There are two kind of pop-science books; one deep and thoughtful based on years of research, one quick and dirty written by a ghost-writer. This book is of the latter kind. Tetlock wrote Expert Political Judgment: How Good Is It? How Can We Know? about a decade ago. That book was deep and thoughtful. I had expected his new book to be an update with ten more years of research and consulting. Sadly, I am greatly disappointed. The book could have been written totally without additional research input. It starts with a couple of chapters of the history of the standard controlled experiment. There is about 50 pages of real content in the 330 pages of the book.There is a lot of content directly lifted from the web (e.g. Fermi-forecasting, Auftragstaktik) - kind of Malcolm Gladwell style, some insight and some misinterpretation.

The style is **extreme pop-science**. What do I mean with that? Far too many pages, plentiful descriptions of minute irrelevant details of individuals (so called human interest points - I guess that is what they teach in creative writing), never any figure or number (e.g. 67% is changed to two thirds), all difficult material removed or put in a footnote. And how come a book with two authors use the pronoun "I" all the time?

The researcher has run a forecasting tournament for several years. He has loads of data, but he does not provide any analysis in the book. He refers to his research in footnotes, but no explanation or description at all. Instead we get statements like 80% of superforecasters are more intelligent than average. What is wrong with running a regression to find out what characteristics are important? Why spend five chapters going through the characteristics of superforecasters? In the end, apparently, two characteristics stand out. (1) Continual updating of forecasts, (2) Being intelligent. That fact is told after around 200 pages of tedious writing. Wtf? I can reluctantly accept dumbing down the book, but it is inexcusable that the footnotes does not include some further help to the reader that wants more depth.

The author likes to give minute details of the superforcasters. Personally, I don't care that Brian likes Facebook updates of cats, that John is retired because he is sick and that he now likes to collect stuff or that Steve is and old colleague of the author that likes opera. Who reads and enjoy this written muzak? It goes on chapter after chapter. We "meet" 15-20 superforecasters.

There is a lot about the superforecasters in the book, but the title of the book is "Superforecasting". This is a seriously misleading title. It makes you believe that you will learn tools to become a great forecaster. You get some, mostly general, points in an eight page appendix. With the researcher's experience, I would have expected a lot of practical advice.

What is good about the book?
(1) The key message that experts are lousy forecasters and do not want accountability is very important, but that was already in the author's earlier book.
(2) Some useful anecdotes that you probably should pick up if you are teaching/presenting on the topic.
(3) Odd bits of information. I liked the discussion of how the German military used what we consider modern management already 100 years ago. As mentioned earlier, there are 50 pages of really good material in the book.

I bought the hard-cover edition. If you make notes with a normal pencil, be careful because it easily pierces the paper.

The book is worth two stars. If you are en educator and want a few anecdotes, read the book. Others should give it a pass. Instead sign up to the author's forecasting tournament. You learn more by trial and error learning. I signed up two years ago and it is a useful experience. You can also check the video features on edge.org. Then spend time reading better books. A few rigorous pop-science books:
* Another forecasting perspective is Steenbarger's Trading Psychology 2.0: From Best Practices to Best Processes (Wiley Trading). It is about trading in the market, but it covers many of the topics from a different perspective. Worth reading his earlier books too.
* And if you haven't read Thinking, Fast and Slow, that is a more important book (but also too fluffy for my linking).
* You should also read Taleb's The Black Swan: Second Edition: The Impact of the Highly Improbable: With a new section: "On Robustness and Fragility" (Incerto), but don't buy his fluffy version of the same topic Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets (Incerto)

Everyone wants to be able to predict the future, whether they are buying stocks, choosing a mate, or deciding how the next presidential election will go, but what, if anything, can we do to improve our ability to predict? Wharton School professor Philip Tetlock has been studying that question since the Reagan era and has observed forecasters from pundits and intelligence analysts to filmmakers and pipe fitters to try to learn why some people are better at making predictions than others. In this book, he describes his work and that of others and presents some techniques that may help all of us make better decisions.
As someone who enjoys reading about topics like decision-making, forecasting, and behavioral economics, I too often find myself reluctantly concluding, “That was well-presented, but there is nothing here I have not heard before.” For a reader new to the subject, it is good that Superforecasting delves into the ideas of people like psychologist Daniel Kahneman, whose description of the biases in judgment that impede our ability to make good decisions and forecasts earned him a Nobel Prize in Economics, and Tetlock appropriately covers topics like these.
I was pleased, though, he also presented some interesting work I was not familiar with, such as the author’s own Expert Political Judgment project to study whether some people really are better predictors than others and, if so, how they differ from the less successful experts, and the Good Judgment Project that was part of an effort to improve intelligence estimating techniques funded by IARPA (the intelligence community’s equivalent of DARPA). I was also especially amused by a contest run in 1997 by the Financial Times at the suggestion of behavioral economist Richard Thaler. People were to guess a number between 0 and 100, and the winner would be the person whose guess comes closest to TWO-THIRDS of the average guess of all contestants. If thinking about this contest begins to make your head spin, read this book. If it sounds pretty simple to you, then you should DEFINITELY read this book; the answer will surprise you!
The history of science was also interesting and often surprising, such as the idea of randomized controlled trials, which are taken for granted today, not being used until after World War II. The book introduces us to people like meteorologist Edward Lorenz, the author of the classic paper asking whether the flap of a butterfly’s wings in Brazil can set off a tornado in Texas, and physician Archie Cochrane, an early advocate for randomized trials and a scientific approach to medical decisions who nonetheless was driven by his human biases to make a decision about his own health that subjected him to a mutilating surgery and could have cost him his life.
After studying and identifying a group of superforecasters and their characteristics, Tetlock asked the natural question: Are superforecasters born, or can we all become superforecasters? As a good scientist, he concludes he cannot answer that question with certainty, but he does lay down some habits of mind that are very likely (Give me a probability here, Phil!) to improve anyone’s ability to make predictions and improve the resulting decisions.
If your aim is to improve your own ability to make predictions, Tetlock will both give you valuable advice and explain how following that rather simple-sounding advice may be harder than you think. I predict you’ll find the book both enjoyable and informative.

I almost gave this four stars because I got bogged down a bit in the beginning, but then the book really took off.

The concept of people forecasting future events was intriguing, but discovering how the effective ones did it was absolutely fascinating. It's not a book that is going to tell you about future events like some talking head, but you will discover some keys based off of test groups to help you forecast more accurately. It's a great book that not only discusses people predicting the future, but how our own bias and perspective can skew our ability to see clearly. The last portion of the book really delved into the power of teamwork and how the RIGHT team can forecast with incredible accuracy.

If you're looking for a book to engage your intellect and challenge some preconceived notions, this is a great one for you.