Dan Gardner essentially presents the amazing work of Philip Tetlock who precisely documented in “Expert Political Judgment” over a couple of decades. Tetlock uncovered that the vast majority of experts predictions in politics and other social sciences were highly inaccurate and often no better than randomness.
To study this issue I would encourage one to go to the source and study “Expert Political Judgment.” It is a seminal social science study. However, this book does read like a 300 page scientific paper that it is. There are a lot of quantitative methods including tables of regression outputs, statistics, Bayesian trees, and related graphs. Tetlock style is lively but it assumes you are comfortable with the mentioned quantitative methods.
Dan Gardner renders Tetlock’s study much more digestible for the majority who may not stomach the quantitative intensity of Tetlock’s message. And, Gardner does an excellent job of doing that. He uses a journalistic narrative style but he does not dumb down the issue.
Gardner also broadens the criticism of expert judgment by reviewing the actual records of well-known pundits. This is in contrast to Tetlock who was extremely careful not to offend anyone. He had taken great care throughout his entire study of rendering the identity of all forecasters anonymous. In essence, he had constructed thousands of prediction contests that he challenged numerous pundits and experts to participate in over years. And, he understood that the minute he would embarrass any forecasters by rendering their results somewhat public, it would be the end of his study. Gardner not burdened by having to complete a study on the topic feels a lot freer to criticize (rightfully) whoever he wants by scrutinizing their actual prediction records and uncovering how dismal that record is.
Throughout the book, Gardner discloses how poor the record of so many well-known pundits and experts is. He focuses an entire chapter by narrowing his gaze on a few such experts (chapter 7, ‘When Prophets Fail’). These include: James Howard Kunstler, Robert Heilbroner, Lord William Rees-Mogg, Paul Ehrlich, Oswald Spengler, and George Friedman among many others within the book. They essentially all predicted the same thing including an overall desperate world stuck in a Malthusian trap with exploding population growth and a scarcity of resources including food. Oil and other commodities would invariably become scarce and exorbitantly expensive. Living standards and life expectancies would plummet, etc. In other words, our civilization is on the edge of a cliff. Also, in all cases the US would loose its supremacy and various countries or areas would take over the lead. The upcoming world leading hegemon is at times Russia or Japan or Europe, and now it is mainly China (until the next prospective candidate). Some of their earlier predictions suggested this near-end of civilization would hit us in the 1970s. When it did not quite occur predictions were revised to the 1980s, then the 1990s, then again. Yet, the US is still standing. Oil is not scarce. Population growth has stabilized. And, civilization as we know it is still moving forward (Big Data, Artificial Intelligence, Machine Learning, robot automation, drones, driverless cars, etc.).
Gardner refers to all the flaws affecting our human brain rendering most of us to be incapable of making accurate predictions. These include overconfidence, hindsight bias, confirmation bias, cognitive dissonance, anchoring, recency, and many other cognitive flaws the vast majority of us have. The study of those flaws has been extensively covered not only by Philip Tetlock, but also by Daniel Kahneman and Amos Tversky. Related studies will contribute to the foundation of a new field: behavioral economics.
In addition to all those common human flaws, forecasters can be divided in two very different thinking tribes: the Hedgehogs and the Foxes (Philip Tetlock’s framework). The Hedgehog is by far the dominant one. The vast majority of well-established experts are Hedgehogs. They are articulate, formidable debaters, very confident and definite. They just about know for sure exactly what is going to happen and why. Their message is typically fairly simple supported by what appears to be irrefutable and comprehensible logic. The Hedgehogs also never change their mind. They have developed a single theory that explains everything everywhere every time. And the Media and the public love them. In turn, the Hedgehogs are very famous, and make a lot of money. The Foxes are just the opposite. They are extremely rare among experts (Gardner mentions Robert Shiller, George Soros; I have also heard that Mark Zandi is considered one… and that’s about it). The Foxes are often shy, not so articulate, and insecure. They believe the world is way to complex and unpredictable to abide by any deterministic theory or law advanced by the Hedgehogs. They accumulate a lot of information including contradicting one. And, they change their mind much more frequently. As a result, they seem hesitant, unconvincing and not persuasive.
The tragedy is that the Hedgehogs we so love and find persuasive are far worse forecasters than the Foxes we more often than not ignore. The Hedgehogs are also far worse at recognizing that they are bad forecasters. They do not recognize their mistakes. And, they are excellent at persuading us that even though their forecast was wrong, they were directionally correct. They just claim to be ahead of their time. And, that eventually they will be proven correct in the near future.
Gardner also expands on why the future is unpredictable by referring to various theories. These include Chaos Theory, including the famous Butterfly Effect. A very small change can cause incremental sequential changes leading to a final very big change. Also, the concept of feedback loop that affects much of social sciences renders the future unpredictable. In essence, we do not forecast in a vacuum. We actually react to forecasts so as to preempt them from occurring. These trigger a new set of forecasts with a subsequent set of feedback loop negating reactions, etc. Also, many events are affected by non-linear effects and confounding variables as well as sheer randomness. The combination of those are not readily modeleable; and, they certainly do not conform to any simple trend extrapolations that usually represent the main tool of many forecasters especially hedgehogs.
Gardner mentions oil prices, economic growth, and stock markets as phenomena that seem forecastable. But, in reality are most challenging to predict. They are all affected by feedback loops and numerous random ad hoc shocks that are just not forecastable.
In the last chapter, Gardner outlines how Foxes think in contrast to Hedgehogs. Fox thinking really boils down to this famous quote from John Maynard Keynes that Gardner mentions at the start of chapter 7: “When the facts change, I change my mind. What do you do, sir?” More explicitly Fox thinking has three elements: 1) aggregation; 2) metacognition; and 3) humility.
Aggregation entails incorporating information from many different sources, including conflicting sources with opposite biases (political or otherwise). It entails using different data source and structure (quantitative, narrative, social media… in today’s parlance using both structured and unstructured data). This effort may render the aggregator somewhat confused, directionless, and uncertain. And, that is exactly where Foxes like to be. Some of them actually get a bit anxious when they find themselves confident about an outcome (do they cross the line into the overconfidence territory of the Hedgehogs). And, this leads to their third characteristic: humility. They like to doubt themselves, re-check and re-evaluate their assessments. They like to revise the direction or probability of their assessments very frequently (Keynes quote). They are also very introspective about their own thinking. How did they reach that conclusion? What logic? Did they avoid classic biases (confirmation bias, recency, hindsight bias, etc.)? And, that is where the second element comes in: metacognition. The latter describes this introspective way of thinking.
Several years ago, Philip Tetlock founded an institute for Fox thinking: the Good Judgment Project. This is a web based forum open to everyone who wants to measure their forecasting skills. Forecasting challenges are issued all the time, and results and accuracy are readily tracked. Tetlock uncovered that a very small percentage of the people are super forecasters. He tested them in various environments against various elite teams (CIA analytical units, MIT and Stanford teams). And, they routinely trounced everyone else. They even beat prediction markets. And, Tetlock wrote an entire book on the subject “Superforecasting” published in 2015. As can be expected, Tetlock goes way beyond Gardner’s very small introduction on Fox thinking. But, Gardner in a clear and digestible language with some very good example gives a good introduction on the subject.
Download the free Kindle app and start reading Kindle books instantly on your smartphone, tablet, or computer - no Kindle device required.
Read instantly on your browser with Kindle for Web.
Using your mobile phone camera - scan the code below and download the Kindle app.
Customers who bought this item also bought
Page 1 of 1 Start overPage 1 of 1
About the author
Follow authors to get new release updates, plus improved recommendations.
Products related to this item
Page 1 of 1Start overPage 1 of 1
Reviews with images
I really like it and recommend it for your skeptical library
I bought this book in the UK from Amazon. I really like it and recommend it for your skeptical library. However I do have sa critical point to raise about Gardner's view that, based on current evidence, predicting the future in human affairs is rather a fool's errand.If myths are no longer good enough for explaining natural phenomena why should we continue accepting them when it comes to explaining human behavior?Perhaps we should be seeking massive investment beyond the scale of CERN if we wish to move beyond current mythical post-hoc rhetorical explanations for cause and effect in our diverse societies? What is more important, after all: (a) discovering interesting, useful and deadly stuff like how to split the atom and the existence of the Higgs Bosun particle or (b) accurately and consistently predicting when the next nuclear strike or holocaust will take place or the next economic disaster will occur unless we know how to effectively intervene?Perhaps predicting the future is not so difficult as we think. One promising route worth exploring further might be to better evaluate the effectiveness of PR, propaganda, public information campaigns and other attitude change initiatives, and to improve evidence based practice in this area. After all, what do such behavior change initiatives seek to achieve if not to influence the future? And in case you were thinking that I've fallen foul of failing to consider Popper's fallacy of induction let me end on a question: isn't the entire notion of the fallacy of induction based on past evidence of the dangers of relying upon inductive logic, which ironically means that it is based on the very fallacy it seeks to reveal?Dr Mike Sutton is author of 'Nullius in Verba: Darwin's greatest secretNullius in Verba: Darwin's greatest secret
Top reviews from other countries
Andrew Lloyd Gordon
Made me think differently about the information I'm exposed to
Reviewed in the United Kingdom on June 15, 2011
