Expert Political Judgment: How Good Is It? How Can We Know? 4th Printing Edition
Use the Amazon App to scan ISBNs and compare prices.
Enter your mobile number or email address below and we'll send you a link to download the free Kindle App. Then you can start reading Kindle books on your smartphone, tablet, or computer - no Kindle device required.
To get the free app, enter your mobile phone number.
Frequently bought together
"This book is a landmark in both content and style of argument. It is a major advance in our understanding of expert judgment in the vitally important and almost impossible task of political and strategic forecasting. Tetlock also offers a unique example of even-handed social science. This may be the first book I have seen in which the arguments and objections of opponents are presented with as much care as the author's own position."―Daniel Kahneman, Princeton University, recipient of the 2002 Nobel Prize in economic sciences
"Before anyone turns an ear to the panels of pundits, they might do well to obtain a copy of Phillip Tetlock's new book Expert Political Judgment: How Good Is It? How Can We Know? The Berkeley psychiatrist has apparently made a 20-year study of predictions by the sorts who appear as experts on TV and get quoted in newspapers and found that they are no better than the rest of us at prognostication."---Jim Coyle, Toronto Star
"Phillip E. Tetlock does a remarkable job . . . applying the high-end statistical and methodological tools of social science to the alchemistic world of the political prognosticator. The result is a fascinating blend of science and storytelling, in the the best sense of both words."---William D. Crano, PsysCRITIQUES
"Philip Tetlock has just produced a study which suggests we should view expertise in political forecasting--by academics or intelligence analysts, independent pundits, journalists or institutional specialists--with the same skepticism that the well-informed now apply to stockmarket forecasting. . . . It is the scientific spirit with which he tackled his project that is the most notable thing about his book, but the findings of his inquiry are important and, for both reasons, everyone seriously concerned with forecasting, political risk, strategic analysis and public policy debate would do well to read the book."---Paul Monk, Australian Financial Review
"This is a marvelous book―fascinating and important. It provides a stimulating and often profound discussion, not only of what sort of people tend to be better predictors than others, but of what we mean by good judgment and the nature of objectivity. It examines the tensions between holding to beliefs that have served us well and responding rapidly to new information. Unusual in its breadth and reach, the subtlety and sophistication of its analysis, and the fair-mindedness of the alternative perspectives it provides, it is a must-read for all those interested in how political judgments are formed."―Robert Jervis, Columbia University
"The definitive work on this question. . . . Tetlock systematically collected a vast number of individual forecasts about political and economic events, made by recognised experts over a period of more than 20 years. He showed that these forecasts were not very much better than making predictions by chance, and also that experts performed only slightly better than the average person who was casually informed about the subject in hand."---Gavyn Davies, Financial Times
"This book is a major contribution to our thinking about political judgment. Philip Tetlock formulates coding rules by which to categorize the observations of individuals, and arrives at several interesting hypotheses. He lays out the many strategies that experts use to avoid learning from surprising real-world events."―Deborah W. Larson, University of California, Los Angeles
"Why do most political experts prove to be wrong most of time? For an answer, you might want to browse through a very fascinating study by Philip Tetlock . . . who in Expert Political Judgment contends that there is no direct correlation between the intelligence and knowledge of the political expert and the quality of his or her forecasts. If you want to know whether this or that pundit is making a correct prediction, don't ask yourself what he or she is thinking--but how he or she is thinking."---Leon Hadar, Business Times
"Winner of the 2006 Woodrow Wilson Foundation Award, American Political Science Association"
"[This] book . . . Marshals powerful evidence to make [its] case. Expert Political Judgment . . . Summarizes the results of a truly amazing research project. . . . The question that screams out from the data is why the world keeps believing that "experts" exist at all."---Geoffrey Colvin, Fortune
"Mr. Tetlock's analysis is about political judgment but equally relevant to economic and commercial assessments."---John Kay, Financial Times
"Woodrow Wilson Foundation Award, American Political Science Association"
"Winner of the 2006 Grawemeyer Award for Ideas Improving World Order"
"Winner of the 2006 Robert E. Lane Award, Political Psychology Section of the American Political Science Association"
"This book is just what one would expect from America's most influential political psychologist: Intelligent, important, and closely argued. Both science and policy are brilliantly illuminated by Tetlock's fascinating arguments."―Daniel Gilbert, Harvard University
"Tetlock uses science and policy to brilliantly explore what constitutes good judgment in predicting future events and to examine why experts are often wrong in their forecasts." ― Choice
From the Inside Flap
"This book is a landmark in both content and style of argument. It is a major advance in our understanding of expert judgment in the vitally important and almost impossible task of political and strategic forecasting. Tetlock also offers a unique example of even-handed social science. This may be the first book I have seen in which the arguments and objections of opponents are presented with as much care as the author's own position."--Daniel Kahneman, Princeton University, recipient of the 2002 Nobel Prize in economic sciences
"This book is a major contribution to our thinking about political judgment. Philip Tetlock formulates coding rules by which to categorize the observations of individuals, and arrives at several interesting hypotheses. He lays out the many strategies that experts use to avoid learning from surprising real-world events."--Deborah W. Larson, University of California, Los Angeles
"This is a marvelous book--fascinating and important. It provides a stimulating and often profound discussion, not only of what sort of people tend to be better predictors than others, but of what we mean by good judgment and the nature of objectivity. It examines the tensions between holding to beliefs that have served us well and responding rapidly to new information. Unusual in its breadth and reach, the subtlety and sophistication of its analysis, and the fair-mindedness of the alternative perspectives it provides, it is a must-read for all those interested in how political judgments are formed."--Robert Jervis, Columbia University
"This book is just what one would expect from America's most influential political psychologist: Intelligent, important, and closely argued. Both science and policy are brilliantly illuminated by Tetlock's fascinating arguments."--Daniel Gilbert, Harvard University
- Publisher : Princeton University Press; 4th Printing edition (July 25, 2005)
- Language : English
- Hardcover : 352 pages
- ISBN-10 : 0691123020
- ISBN-13 : 978-0691123028
- Item Weight : 1.38 pounds
- Dimensions : 6.25 x 1 x 9.25 inches
- Best Sellers Rank: #2,457,844 in Books (See Top 100 in Books)
- Customer Reviews:
Top reviews from the United States
There was a problem filtering reviews right now. Please try again later.
If you’ve already been exposed to a cliff-notes version of the book on a podcast or in other readings, I’d still pick it up: It contains much more than any summary could do justice, with each chapter containing more substantive content than most books. Tetlock’s familiarity with a breadth of sources from statistics (not philosophic of science (his home discipline is psychology) is a delight.
The dialectical structure of the book itself was instructive in how communicate memorably:
+ First, two positions debate the thesis of the book—the skeptic who doubts the possibility of good judgment in the political domain, and the meliorist who holds out hope for the possibility of improvement.
+ After the evidence is presented, the central result is personified in two characters who are put on trials and judged—the hedgehogs who know one big thing, and the foxes who know many small things. These are the personifications of the most salient factors determining forecasting accuracy and probabilistic coherence. There is much special pleading.
+ Finally, the results of the book are put under debate by four voices—the radical relativist, the moderate relativist, the moderate neopositivist, and the radical neopositivist. Suffice it to say that Tetlock takes the wind out of would-be naysayers and says everything that could said against his own position better than most people could say it. At the end of the day, the moderates come together and cooler heads prevail.
Holding public discourse to the intersubjective standards of clarity under the norm of proffering predictions as reputational bets will be hard, but it is possible, and it is needed.
Tetlock is true to his subject—good judgment—in seeing many sides of the case against his hypotheses, and giving them the strongest reading he can, not in order to defeat them, but to see what he can learn from them. This trait makes me happy when I see it in anyone.
I’ll paraphrase a quote that once made me chortle, yet somehow feels very appropriate here. As Ken Binmore once said in the preface to William Sandholm’s seminal “Population Games and Evolutionary Dynamics”: “He puts the beef on the table.”
If you have the option, I’d get 2017 edition for an extraordinary summative new preface by Tetlock.
Were the experts better at anything? Well, they were pretty good at making excuses. Here are a few: 1. I made the right mistake. 2. I'm not right yet, but you'll see. 3. I was almost right. 4. Your scoring system is flawed. 5. Your questions aren't real world. 6. I never said that. 7. Things happen. Of course, experts applied their excuses only when they got it wrong... er... I mean almost right... that is, about to be right, or right if you looked at it in the right way, or what would have been right if the question were asked properly, or right if you applied the right scoring system, or... well... that was a dumb question anyway, or....
Not only did experts get it wrong, but they were so wedded to their opinions that they failed to update their forecasts even in the face of building evidence to the contrary. And then a curious thing happened -- after they got it wrong and exhausted all their excuses, they forgot they were wrong in the first place. When Tetlock did follow-up questions at later dates, experts routinely misremembered their predictions. When the expert's models failed, they merely updated their models post hoc, giving them the comforting illusion that their expert judgment and simplified model of social behavior remained intact. Compare this with another very complex system -- predicting the weather. In this latter case, there is a very big difference in the predictive abilities of experts and lay persons. Meteorologists do not use over-simplified models like "red in the morning, sailor's warning." They use complex modeling, statistical forecasting, computer simulations, etc. When they are wrong, weathermen do not say, well, it almost rained; or, it just hasn't rained yet; or, it didn't rain, but predicting rain was the right mistake to make; or, there's something wrong with the rain guage; or, I didn't say it was going to rain; or, what kind of a question is that?
Political experts, unlike weathermen, live in an infinite variety of counterfactual worlds; or as Tetlock writes, "Counterfactual history becomes a convenient graveyard for burying embarrassing conditional forecasts." That is: sure, given x, y, and z, the former Soviet Union collapsed; but if z had not occurred, the former Soviet Union would have remained intact. Really? Considering the expert got it wrong in the first place, how could they possibly know the outcome in a hypothetical counterfactual world? At best, this is intellectual dishonesty. At worst, it is fraud.
But some experts did better than others. In particular, those who were less dogmatic and frequently updated their predictions in response to countervailing evidence (Tetlock's "foxes") did much better than the opposing camp (termed "hedgehogs"). The problem is that hedgehogs climb the ladder faster and have positions of greater prominence. My Machiavellian take? You might as well make dogmatic pronouncements because all the hedgehogs you work for aren't any better at predicting the future than you are -- they're just more sure of themselves. So, work on your self-confidence. It is apparently the only thing anyone pays any attention to.
Top reviews from other countries
Most academics with an interesting, new theory like this write two books. A magnum opus stuffed with references and experimental detail for the academic community, and another, shorter, more easily digestible work for the lay reader. This is what Bruce Bueno de Mesquita did for "The Logic of Political Survival" when he wrote "The Dictator's Handbook". Sadly Tetlock has failed to come out with the second type of book, so we are left to wade through a fairly dry academic tome which, while having its moments, is a bit too footnote heavy for me.
I urge you to skim through this, maybe using a library copy, but I would counsel against buying it, unless you are of an academic cast of mind.
(Archilochos, griechischer Lyriker, 680 bis 645 v. Chr.)
Philip Tetlock ist gelernter Psychologe und Prof. für Leadership an der Univ. Kalifornien. Er forscht darüber, welche Faktoren für menschliche Weitsicht und Blindheit verantwortlich sind. Von 1985 bis 2003 befragte er 284 ausgewählte amerikanische Experten über den Lauf der Welt. Der Prognosehorizont war meistens 2 bis 5 Jahre, in Einzelfällen aber auch bis zu einer Dekade. Die Teilnehmer mussten z.B. beantworten, ob in einem bestimmten Land die jetzige Regierung nach den nächsten bzw. übernächsten Wahlen noch immer am Ruder ist (in autoritären Regimen, ob sie weg geputscht wird). Andere Fragen waren etwa, ob sich die Provinz Quebec von Kanada loslösen wird, ob zwischen Indien und Pakistan ein Krieg ausbricht. Ob das Wachstum des Bruttonationalproduktes, die Staatsverschuldung oder der Zinssatz der Notenbank höher, niedriger oder gleich ausfallen wird, die Preise für wichtige Rohstoffe nach oben, unten gehen oder gleich bleiben, ob die Internet-Börsenblase innerhalb des Prognosehorizontes platzt. Die Experten mussten nicht nur die Richtung angeben, sondern auch, für wie wahrscheinlich sie die einzelnen Szenarien – z.B. die Abspaltung Quebecs – hielten.
Ein roter Faden in Tetlock's Experimenten ist: Die Experten sind sich – egal auf welchem Gebiet – viel zu sicher. Wenn sie etwas als „praktisch sicher“ einstufen, dann kommt es höchstens mit 70% Wahrscheinlichkeit vor. Es sind auch Wunder gar nicht so selten. Es treten Ereignisse ein, die die Experten für (denk-)unmöglich gehalten haben. Generell ist die Prognose „kräht der Hahn am Mist bleibt das Wetter so wie es ist“ ziemlich gut. Sie hätte gegen die Experten klar gewonnen. Die Experten überschätzen krass die Wahrscheinlichkeit von Wendungen zum Guten oder Schlechten. Dies gilt insbesondere für die Igel.
Die Unterscheidung zwischen Igel (Engl. Hedgehog) und Fuchs geht auf einen – im englischen Sprachraum – allgemein bekannten Aufsatz des Philosophen Sir Isaiah Berlin (1909 bis 1996) zurück. Nach Berlins Auffassung versuchten die Igel, ein allumfassendes System menschlicher Handlungen, der Geschichte und von moralischen Werten zu entwickeln. Die Füchse hingegen tendierten eher dazu, überall eine Vielfalt zu sehen. Ein Fuchs verfolgt viele Ziele, oft ohne inneren Zusammenhang oder sogar widersprüchlich. Typische Igel sind für Berlin Plato, Pascal, Hegel, Dostojewski, Nietzsche oder Proust, Füchse hingegen Shakespeare, Aristoteles, Erasmus von Rotterdam, Goethe, Puschkin oder Joyce. Es gibt natürlich auch Mischformen. Der von Berlin verehrte Tolstoi war seiner Auffassung nach ein Fuchs, der gerne ein Igel gewesen wäre. Für die Welt der Experten könnte man den Typus des Hedgehogs auch etwas vereinfachend mit „Fachidiot“ beschreiben. Der Fuchs ist der umtriebige bunte Hund, der alles und nichts kann.
Der oben geschilderte Zusammenhang ist die zentrale Botschaft des Buches. Tetlock kann sich auch überzeugend belegen. Daneben gibt es eine Reihe von anderen Zusammenhängen. Z.B. Stars die besonders oft in den Medien zitiert werden liefern sehr schlechte Prognosen ab. Das hängt mit der Vorliebe der Medien für Igel zusammen.
Inhaltlich verdient das Buch 6 Sterne. Allerdings ist es ziemlich akademisch-öd geschrieben. Tetlock wollte damit offensichtlich einen akademischen Klassiker verfassen. Das ist ihm gelungen. Für das breite Publikum ist seine Darstellung aber eher ermüdend und wenn man sich nicht in Statistik auskennt teilweise auch unverständlich. Ich kenn mich in Statistik aus, habe mich aber durch manche Passagen eher durchgekämpft. Vieles könnte man ohne inhaltliche Abstriche einfacher, flotter sagen. Aber dann wärs kein akademischer Klassiker geworden.
Wer an einer gut lesbaren Zusammenfassung interessiert ist wohl mit dem Artikel "Trau keinem Igel" in der Dezember 2014 Ausgabe von Chrilly's Monatlicher Goldreport besser bedient (Nach "Chrilly's Monatlicher Goldreport" googeln).
P.S.: Das Wort Akademie bezieht sich auf den Treffpunkt von Platos Gelehrtenkreis. Gerade Plato hat aber grossen Wert auf die Lesbarkeit seiner Werke gelegt und viele seiner Gedanken in Dialogform präsentiert.