Customer Reviews

Junk Science Judo: Self-Defense Against Health Scares and Scams

5 star:		(11)
4 star:		(7)
3 star:		(0)
2 star:		(5)
1 star:		(7)

 

 

BOOK REVIEW

Junk Science Judo. Self-Defense Against Health Scares and Scams by Steven J. Milloy, Cato Institute, Washington, DC, USA, 2001, ix-xii + 218 pp.

A superb manual for understanding health claims and detecting fraud, "Junk Science Judo" is written in a punchy, easy-to-read style that allows the mathematically challenged, like myself, to interpret the usual bell-curve statistics that are used for environmental health threats and diet threats. One may also use this approach to judge the benefits of prescription drugs or alternative treatments. He shows how to look for p values of 0.05 or less, and he shows how to use the 95% confidence intervals (CI) to see whether the low or high limit crosses the reference value of 1.00. If it does, Milloy suggests disregarding any result that is claimed.

In epidemiological or ecologic studies, Milloy suggests discarding those results in which the relative risk (RR) is between 0.5 and 2.00, whichever applies. Milloy goes far beyond the usual cautions that an association is not necessarily a cause. He is contemptuous of, but not totally dismissive of epidemiology. For Milloy, the descending pecking order of research on health hazards is: clinical trials, cohort studies, case-control studies, and ecologic studies. He describes publication bias, confounding, and "tainted experts". He reminds us that mice are not little people, and that there is a safe dose of everything.

Milloy encourages intelligent contact with the sources of health advice to support or refute the recommendations, and he recommends a number of sources of supposedly reliable health information. His criticism of medical journals, which includes the defects of peer-review, is the most detailed I have seen in print, and very well-taken, in my opinion. He skewers broadcast media on their emphasis on ratings rather than accuracy or balance. He deplores the motivation of many activists. Many specific examples are given, from Alar to radiation.

This book is extremely valuable for helping anyone who is not a medical or epidemiological specialist to judge the value of health or health-threat claims.

Now the bad news. Milloy does not question the use 1-tailed statistics, or the failure of many relationships to follow a bell-curve (Kauffman, 2001). Milloy failed to warn of the common deception of reporting an effect of something on a certain health condition without the inclusion of total death rates. Milloy's fine example of prostate cancer "non-prevention" by eating foods high in selenium (p164) may mislead people into ignoring the benefits of selenium supplementation, which has been shown in a clinical trial to lower the RR of all cancers to 0.83 (95% CI 0.47-0.85), and is most effective against lung, prostate, and colorectal cancers (Clark et al., 1996). Milloy downplays the dangers of trans fats (Oomen et al., 2001), but his own example shows that the highest two quintiles of consumption are not healthful (p165). Milloy made the mistake of writing that the absence of a biological explanation for the claimed effects of electric and magnetic fields means that they have no effects (p76); readers of JSE know that such judgments based on lack of knowledge are faulty. ....
Joel M. Kauffman
...

References

Clark, L. C. et al. (17), (1996). Effects of Selenium Supplementation for Cancer Prevention in Patients with Carcinoma of the Skin. A Randomized Controlled Trial. Journal of the American Medical Society, 276, 1957-1963.

Kauffman, J. M. (2001). Article of Interest. Journal of Scientific Exploration, 15(4), 575-576.

Oomen, C. M., Ocké, M. C., Feskens, E. J. M., van Erp-Baart, M.-A. J., Kok, F. J., & Kromhout, D. (2001). Association between trans fatty acid intake and 10-year risk of coronary heart disease in the Zutphen Elderly Study: a prospective population-based study. The Lancet, 357, 746-751.

Hardly a day can pass without hearing about some new threat to our health from something in our food or in the air that we breathe. In his lively style, Steven J. Milloy has written a "self-defense guide" to help the average consumer know when to be alarmed or when to proceed with life as usual.

Junk Science Judo is filled with examples of questionable conclusions drawn from data from a wide variety of sources. Milloy relates how zealous lawyers can draw false or misleading inferences from scientific data in order to extract large rewards from lawsuits against defendants with deep pockets, often with collaboration from those in the "scientific" community who stand to profit from their "research" (what better way to justify large government grants to continue research into some promising area?). Most telling are the cases of the lawsuits involving breast implants and Agent Orange, where a total absence of evidence was ignored by courts and juries eager to punish unpopular defendants and to reward those who were definitely disadvantaged (but probably not as a result of either the implants or Agent Orange).

Despite a generally good job of writing a primer on how not to be taken in by faulty conclusions or misleading inferences of cause and effect, Chapter 11 of this book contains one significantly dubious conclusion of the author himself. In a discussion of the statistical technique of meta-analysis (a technique where the results of numerous small studies are combined to simulate one large study), Milloy repeatedly takes stabs at this respected technique. His criticisms appear to be based mainly on questions raised in an editorial in a medical journal where the editor cited poor data quality and publication bias (i.e., only research studies showing significant results tend to get published) as reasons for questioning conclusions from meta-analysis. Although the editor is correct, it does not follow that all meta-analysis is therefore useless. Given enough well-executed studies without bias in the findings, meta-analysis can be a very useful and enlightening tool.

I highly recommend that all consumers of information either purchase this book or at least borrow it from someone, and read it thoroughly. I suspect that you will never again be able to read uncritically any account of some purported new crisis brought on by zealous lawyers, journalists or politicians. When some "scientific" conclusion appears to be counterintuitive, it is most likely incorrect. Subsequent research may, in fact, show that the conclusions were incorrect, but (since this is not exciting) the correction is likely not to be brought to the public's attention. Since it is impossible to prove a negative, the burden of proof should be on the scientific community to exercise caution in releasing insignificant findings to the press. Junk science may sell newspapers or gain an audience for the evening news, but it is devastating when it causes major disruptions in the lives of people who either aren't given the whole picture or who can't interpret the information when it is presented.

The best defense against junk science is an informed public. Read this book and you will definitely become better informed!

Some parts of this book literally left my jaw hanging open. The endless examples of high-level fraud in government agencies and special interest groups are enough to make any logical thinker sick.

Bear in mind that this book is probably not for the trained scientist. The author takes time to explain basic research concepts for the casual reader, and also interjects his own attitude to make the book more fun to read. However, for those readers who do want to check up on the author's science, the 17-page listing of 314 cited studies should leave no question that the author beckons the audience to follow his own advice by examining his research.

If I had one complaint about this book, it would be that the author often relies too heavily on quoted material. I think the author is trying to show that he was not skewing the cited studies, as would the individuals he criticizes in the book. While effective in this regard, the practice sometimes results in a monotonous pattern. The author's comments on the quoted studies become predictable midway through the book as the reader learns to identify the glaring problems in cited research methods.

In the end, this book was an excellent read for the choir to which it preaches. I would recommend it as an excellent resource for anybody who wants ammunition to fight junk science, or to back up political debate with embarrassing proof of the ineptitude on which many health/environmental policies are based. Anybody who cares about ridding public policy of special interest politics should read through this book once and keep it handy for later use as a reference tool.

I purchased this book after spending much time on Mr. Milloy's web site, www.junkscience.com, and found it to be an excellent primer on the way information is manipulated to serve various purposes.

The book is well-written, and an easy read, with an eye toward the scientific novice. But the book focuses not so much on science, but on statistics, which are often used as a surrogate for science. Indeed, Mr. Milloy never claims to be a scientist; his biography notes that he has a degree in statistics, making him well-qualified to criticize the misuse of such numbers.

I recently had the opportunity to use the information gleaned from "Junk Science Judo." A local group used a study as evidence that cranberry bogs present a cancer risk, i.e., the hypothesis of the study was that living near cranberry bogs exposed local residents to enough pesticides to cause cancer. While the authors did find a correlation between living near cranberry bogs and a particular kind of brain cancer, it was only a correlation.

This was an ecologic epidemiological study (the worst kind according to Mr. Milloy). The authors never established that the study subjects were exposured to the pesticides used on cranberry bogs; blood or urine samples were never obtained, but a formula was used to estimate exposure levels.

There were other problems; the sample size was too small (only a dozen individuals had brain cancer) to be of value; cancer rates were no higher for those individuals who had known exposures to pesticides (such as pest control or agricultural workers) than for the control group; pesticide use on the cranberry bogs could only be guessed at - the authors did not know what pesticides were used, when they were used or in what quantities.

In short, the study raised more questions than answers. But when I critizied the study and the use of it, the only thing anyone could say was, "Pesticides are toxic." Well, duh. But as Mr. Milloy notes, the dose makes the poison.

Ironically, most of the bad reviews of this book make an issue out of motivation, and appear to be criticizing Mr. Milloy personally, instead of the book and its contents.

Scientists can be activists, and when they are they often hide behind their scientific training, claiming to be impartial even as they advocate a certain point of view (believe me, I've encountered them). And just because a corporation pays for a study does not mean that that study is any less relevant than one paid for by an environmental organization. I've gone so far as to research some of the examples Mr. Milloy uses (such as the Love Canal) and found them to be true.

My point - and that of Mr. Milloy's book - is to look at the claims made with a critical eye; examine the science behind them. In "Junk Science Judo," Mr. Milloy provides us with the information needed to determine if the claims made are true, exaggerated, or completely out of the ballpark. This book is not a collection of blind rants, but a close examination of how public health claims are made and justified (rightly or wrongly).

Get the book, but keep an open mind, because there is no doubt that it will challenge some of your basic assumptions. If you're a natural skeptic, it will give you a foundation upon which to base your skepticism. If you're not a skeptic, it will open your eyes.

Every day American readers are given scare stories about health issues in news accounts based on some statistics and little analysis. Many accounts are circulated by special interest groups or lawyers. Milloy's background in law and health lends to his ability to point out the fallacies of 'junk science' and tips on how the average reader can separate junk science from real science.

Steven Milloy is a journalist, and pundit whose principled stand on behalf of scientific logic benefits all of us. He stands side by side with Stephen Jay Gould and Michael Schermer in debunking the outrageous claims of pseudo science. Milloy attempts to provide a framework to evaluate the nonsensical claims made by individuals and organizations with agendas which are independent of the facts. It is this framework he calls "junk science judo." Actually what he provides us is the kind of suspicious phraseology and statistical trickery often employed by scammers. In most cases it would take a thorough review of the claimants' research techniques and statistical interpretation to spot the scam. We come away suspicious of almost any claim. All in all, though, Milloy is readable and he is right. Don't miss his column, "Junk Science," on the Opinion page of FoxNews.com.

Postscript: With the passage of time I've become aware of a blind spot in Milloy's vision. It doesn't appear in this book, but it does repeatedly in his columns and website. He denies global warming, which is reliably documented by science. Where junk science enters the debate is in the extent of human causality. To date there is no credible evidence for human causality; it is an issue pressed by political populists and our competitors in global trade. Global warming is good science; human causality is junk science. Science shows that if humans disappeared from the Earth, global warming would continue, and likely continue to accelerate. Milloy is so incensed by the assertion of human causality that he denies warming altogether. In his acceptance of the politicization of the debate, he plays into the hands of his adversaries. His willingness to distort facts in support of this cause unnecessarily introduces doubt about his objecivity in other issues.

This book does a very good job of explaining, in easy to understand terms, how to tell good science from bad, how to identify the different types of studies and methods, what to keep an eye out for, and how to differentiate between anecdotes, statistics, and science.

I find it quite ironic that the only negative reviews here are of the ad hominem variety of which the author warns us to be wary.

If you're tired of being frightened and victimized by the never ending procession of doom-and-gloom prognostications, this book is for you. It lucidly explains how statistics and "scientific" studies are routinely manipulated by groups with their own agendas who are willingly assisted by the media who are all too happy to be given a shocking headline for public consumption. After reading this book, you will never react to alarmist news reports in the same way.

And, those who have no valid counter arguments use ad hominem attacks.

If your "Statistics for Health Professionals" class was as incomplete as mine, then, you need this book. Finally, you will get a clear explanation of p-values, confidence intervals and study designs. Lesson 4: "Epidemiology is Statistics" alone was worth the price of the book. However, don't miss out on the other 11 lessons. It's just like the 12 steps up from alcoholism. By learning and living these rules, you will at last be able to free yourself from those maddening health scares that bombard us daily. Milloy writes in a sassy, bold as brass style that entertains whilst it informs. Personally, I like that sort of thing. This book is an easy read, and if you really, really want to, you can find and read the original studies that are discussed. Then you will appreciate an author who can tackle complex subjects in a straightforward manner, and be funny in the bargain.

"When the enemy comes welcome him, when he goes, send him on his way," is a tenet of the practice of the art of judo. Milloy will show you the enemy, and equip you with the means of speedily dispatching them to the junk pile where they belong. You will truly welcome the next scientific study you encounter.

Whether you are a newbie in science, or do research, Milloy's work is exceptionally relevant, and at the same time intensely interesting.

Sadly, there seem to be very few people doing exactly what is needed in this growing area of scientific vandalism, and I applaud Milloy's courage in writing this work and others on the same or similar topic. He shows great creativity in the style and format used to write this work. It is exceptionally interesting to read, and captivates. My copy was on my office desk for exactly three days, and then without a routing list made it way through some of the science staff.

I wish I had kept the statistics on this, and could plot the exact point that this work seeded an intense discussion. Perhaps a critical mass of people had been reached, and the fallout was exceptional. I suppose it will simply take more time as more and more people read, and understand the difficulties Milloy cites. It would be a shame to re-live the problems of yesteryear, and with Milloy showing the problems before they become epidemic, a double shame.

Awareness if so very difficult to acquire, but Milloy does help open eyes. I would heartily recommend this book.