Wrong: Why experts* keep failing us--and how to know when not to trust them *Scientists, finance wizards, doctors, relationship gurus, celebrity CEOs, ... consultants, health officials and more Hardcover – June 10, 2010

Author David Freedman is a good writer and "WRONG" is a pretty good read. The book focuses on the many ways that experts, in numerous fields, can mess us up by passing on wrong information and advice. As you might guess, Freedman's warning is to beware of experts, particularly those claiming to know it all. He even tackles the question "is this book wrong" - a smart touch, given the subject.

Where WRONG comes up a bit short is on solutions -- how to tell questionable advice from the trustworthy kind. I wondered why a book that's so good at describing the problem couldn't muster a few more creative answers toward the end.

Freedman might have made a stronger case by weaving in some of the things that experts do "right." The contrast would have better highlighted their true failures while also helping readers know when to trust expert advice and when to question it. And that's critical if you want to use the info he shares to make better choices. Instead, Freedman suggests we can't trust most experts most of the time. That's a bit misleading and not as helpful as it could be in a world where most of us rely pretty heavily on experts for a range of basic services.

The research I've seen suggests the need to be especially wary when experts weigh in on topics outside their comfort zones or where answers to problems are unknown. By contrast, experts do better than non-experts on problems that have known solutions. There are always exceptions but this makes sense in general. For example, plumbers are experts who have seen thousands of drain clogs and know how to treat. Mechanics have a wealth of experience diagnosing common car problems. And ditto for many of the common problems a family doctor encounters or a tax accountant sees. Do these experts make mistakes? Sure. But, compared to the rest of us, they tend to do pretty well on problems in their fields that are well understood. Freedman has to know this but doesn't say it clearly, so readers may easily come away thinking otherwise.

Of course, unscrupulous people from doctors to auto-mechanics can also knowingly mislead us. Freedman addresses ethics in several places but tends to lump together experts who give us wrong info on purpose with those who do it by accident or ineptitude. The bad advice may look similar in these cases, but ethical wrongs spring from different problems and point to different solutions. Again, it would have made for a more compelling conclusion if Freedman had pushed harder on possible solutions here and in related areas. Maybe he wanted to do more but just ran out of time before publication. Overall, a good analysis but the payoff & solutions could have been more constructive.

If your job requires you to make decisions based on advice or research studies, then you should read Wrong by David Freedman. The book takes a look at the state of studies and the unsettling observation that a surprising minority of studies is inaccurate, flawed or just plain wrong. The book looks at several types of studies with a concentration on the medical studies we hear so much about and so often hear conflicting advice. For example, red wine prevents heart disease when another study shows no relationship.

The book is Freedman's investigation and exploration of the reasons behind the why these studies are wrong. The book takes the reader on a systematic investigation of the forces that lead to the publication of inaccurate studies from the need to simplify study finding, the bias of publishing only positive findings, to the social pressures that suppress whistleblowers. Freedman paints a comprehensive picture of the weakeness of the scientific research, including research conducted by Nobel Laureates.

Freedman also takes a look at business research and business books which suffer from these same weaknesses and biases. He points out the structural weakness of the two major basis of business books - that today's `winners' offer immutable lessons for everyone else, or that companies need radical new approaches to address new issues. That discussion, in Chapter 6, should be required reading for every business guru and person offering advice. Readers should also go back to Clayton Christensen's HBR article Why Hard-Nosed Executives Should Care About Management Theory that was published in September 2003 to round out their understanding of business research.

Freedman provides practical advice on characteristics of different types of advice. Below is a summary of the statements to give you an idea of how comprehensive the book is, each point is discussed in detail in the last chapter and this cements the value of the book for a researcher or those who make decisions based on research.

Characteristics of less trustworthy advice include:
- Advice that is simplistic, universal, and definitive
- Advice that is supported by only a single study, or many small or less careful ones, or animal studies.
- Advice that is groundbreaking
- Advice that is pushed by people or organizations that stands to benefit from its acceptance.
- Advice that is geared to preventing future occurrences of a prominent recent failure or crisis.

Characteristics of expert advice we should ignore:

- Advice that is mildly resonant
- Advice that is provocative
- Advice that gets a lot of positive attention
- Advice that other experts embrace
- Advice that appears in a prestigious journal
- Advice that is supported by a big, rigorous study
- Advice backed by experts that boast impressive credentials

Characteristics of more trustworthy expert advice

- Advice that does not trip other alarms
- Advice that is a negative finding
- Advice that is heavy on qualifying statements
- Advice that is candid about reputational evidence
- Advice tat provide some context for research
- Advice that provides perspective
- Advice that includes candid, blunt comments

There are to many points here to repeat in this review, but this is the most valuable part of the book and one of the reasons I am recommending the book - particularly for those who engage in research or are making decisions based on research they either commission or review.

STRENGTHS

Wrong offers a comprehensive view of different types of studies from medical, business, public policy, etc. This gives the book broad appeal and the careful reader insight into the overall modern research process.

The discussion on the wisdom of crowds (Chapter 4 - the Idiocy of Crowds) is perhaps one of the better responses to the current craze of social media. This book is definitely worth the read.

The book offers numerous examples, predominantly of medical studies, which the reader can remember hearing about. The South Korean scientist cloning human stem cells, the debate about the connection between red wine and health are among the flawed science and study Mr. Freedman uses to support his analysis.

The book is blunt and direct and the author calls out people by name and points out the weakeness of their past work.

The author gives you direct references to other skeptical scientists and people who have the job of verifying studies and assuring the quality of research. This has led me to their work, something I never would have found on my own.

CHALLENGES

This book is not for everyone and while its premise, that many scientific studies are based on poor research, biased, or just plan wrong will have popular appear, the book is not own written for the mainstream. People who do research or make decisions based on research or give advice will get the most out of this book. That is something I do and I found the book very powerful.

The book occasionally drifts into areas and alleys that seem to be more for the author's benefit than the readers. The middle chapters particularly go into detail on things that I did not find particularly interesting. This required me to work my way through chapters five, seven and eight which I found a bit heavy and indirect.

The book has a strong bias and purpose in pointing out the weaknesses and outright failures of research and the research process. The readers have to remind themselves that this is a book about an issue rather than research on the research process. If you know the bias, then you can get a lot out of this book.