on June 8, 2002
Junk Science Judo. Self-Defense Against Health Scares and Scams by Steven J. Milloy, Cato Institute, Washington, DC, USA, 2001, ix-xii + 218 pp.
A superb manual for understanding health claims and detecting fraud, "Junk Science Judo" is written in a punchy, easy-to-read style that allows the mathematically challenged, like myself, to interpret the usual bell-curve statistics that are used for environmental health threats and diet threats. One may also use this approach to judge the benefits of prescription drugs or alternative treatments. He shows how to look for p values of 0.05 or less, and he shows how to use the 95% confidence intervals (CI) to see whether the low or high limit crosses the reference value of 1.00. If it does, Milloy suggests disregarding any result that is claimed.
In epidemiological or ecologic studies, Milloy suggests discarding those results in which the relative risk (RR) is between 0.5 and 2.00, whichever applies. Milloy goes far beyond the usual cautions that an association is not necessarily a cause. He is contemptuous of, but not totally dismissive of epidemiology. For Milloy, the descending pecking order of research on health hazards is: clinical trials, cohort studies, case-control studies, and ecologic studies. He describes publication bias, confounding, and "tainted experts". He reminds us that mice are not little people, and that there is a safe dose of everything.
Milloy encourages intelligent contact with the sources of health advice to support or refute the recommendations, and he recommends a number of sources of supposedly reliable health information. His criticism of medical journals, which includes the defects of peer-review, is the most detailed I have seen in print, and very well-taken, in my opinion. He skewers broadcast media on their emphasis on ratings rather than accuracy or balance. He deplores the motivation of many activists. Many specific examples are given, from Alar to radiation.
This book is extremely valuable for helping anyone who is not a medical or epidemiological specialist to judge the value of health or health-threat claims.
Now the bad news. Milloy does not question the use 1-tailed statistics, or the failure of many relationships to follow a bell-curve (Kauffman, 2001). Milloy failed to warn of the common deception of reporting an effect of something on a certain health condition without the inclusion of total death rates. Milloy's fine example of prostate cancer "non-prevention" by eating foods high in selenium (p164) may mislead people into ignoring the benefits of selenium supplementation, which has been shown in a clinical trial to lower the RR of all cancers to 0.83 (95% CI 0.47-0.85), and is most effective against lung, prostate, and colorectal cancers (Clark et al., 1996). Milloy downplays the dangers of trans fats (Oomen et al., 2001), but his own example shows that the highest two quintiles of consumption are not healthful (p165). Milloy made the mistake of writing that the absence of a biological explanation for the claimed effects of electric and magnetic fields means that they have no effects (p76); readers of JSE know that such judgments based on lack of knowledge are faulty. ....
Joel M. Kauffman
Clark, L. C. et al. (17), (1996). Effects of Selenium Supplementation for Cancer Prevention in Patients with Carcinoma of the Skin. A Randomized Controlled Trial. Journal of the American Medical Society, 276, 1957-1963.
Kauffman, J. M. (2001). Article of Interest. Journal of Scientific Exploration, 15(4), 575-576.
Oomen, C. M., Ocké, M. C., Feskens, E. J. M., van Erp-Baart, M.-A. J., Kok, F. J., & Kromhout, D. (2001). Association between trans fatty acid intake and 10-year risk of coronary heart disease in the Zutphen Elderly Study: a prospective population-based study. The Lancet, 357, 746-751.
Hardly a day can pass without hearing about some new threat to our health from something in our food or in the air that we breathe. In his lively style, Steven J. Milloy has written a "self-defense guide" to help the average consumer know when to be alarmed or when to proceed with life as usual.
Junk Science Judo is filled with examples of questionable conclusions drawn from data from a wide variety of sources. Milloy relates how zealous lawyers can draw false or misleading inferences from scientific data in order to extract large rewards from lawsuits against defendants with deep pockets, often with collaboration from those in the "scientific" community who stand to profit from their "research" (what better way to justify large government grants to continue research into some promising area?). Most telling are the cases of the lawsuits involving breast implants and Agent Orange, where a total absence of evidence was ignored by courts and juries eager to punish unpopular defendants and to reward those who were definitely disadvantaged (but probably not as a result of either the implants or Agent Orange).
Despite a generally good job of writing a primer on how not to be taken in by faulty conclusions or misleading inferences of cause and effect, Chapter 11 of this book contains one significantly dubious conclusion of the author himself. In a discussion of the statistical technique of meta-analysis (a technique where the results of numerous small studies are combined to simulate one large study), Milloy repeatedly takes stabs at this respected technique. His criticisms appear to be based mainly on questions raised in an editorial in a medical journal where the editor cited poor data quality and publication bias (i.e., only research studies showing significant results tend to get published) as reasons for questioning conclusions from meta-analysis. Although the editor is correct, it does not follow that all meta-analysis is therefore useless. Given enough well-executed studies without bias in the findings, meta-analysis can be a very useful and enlightening tool.
I highly recommend that all consumers of information either purchase this book or at least borrow it from someone, and read it thoroughly. I suspect that you will never again be able to read uncritically any account of some purported new crisis brought on by zealous lawyers, journalists or politicians. When some "scientific" conclusion appears to be counterintuitive, it is most likely incorrect. Subsequent research may, in fact, show that the conclusions were incorrect, but (since this is not exciting) the correction is likely not to be brought to the public's attention. Since it is impossible to prove a negative, the burden of proof should be on the scientific community to exercise caution in releasing insignificant findings to the press. Junk science may sell newspapers or gain an audience for the evening news, but it is devastating when it causes major disruptions in the lives of people who either aren't given the whole picture or who can't interpret the information when it is presented.
The best defense against junk science is an informed public. Read this book and you will definitely become better informed!
on January 1, 2002
Some parts of this book literally left my jaw hanging open. The endless examples of high-level fraud in government agencies and special interest groups are enough to make any logical thinker sick.
Bear in mind that this book is probably not for the trained scientist. The author takes time to explain basic research concepts for the casual reader, and also interjects his own attitude to make the book more fun to read. However, for those readers who do want to check up on the author's science, the 17-page listing of 314 cited studies should leave no question that the author beckons the audience to follow his own advice by examining his research.
If I had one complaint about this book, it would be that the author often relies too heavily on quoted material. I think the author is trying to show that he was not skewing the cited studies, as would the individuals he criticizes in the book. While effective in this regard, the practice sometimes results in a monotonous pattern. The author's comments on the quoted studies become predictable midway through the book as the reader learns to identify the glaring problems in cited research methods.
In the end, this book was an excellent read for the choir to which it preaches. I would recommend it as an excellent resource for anybody who wants ammunition to fight junk science, or to back up political debate with embarrassing proof of the ineptitude on which many health/environmental policies are based. Anybody who cares about ridding public policy of special interest politics should read through this book once and keep it handy for later use as a reference tool.
on December 10, 2004
I purchased this book after spending much time on Mr. Milloy's web site, [...] and found it to be an excellent primer on the way information is manipulated to serve various purposes.
The book is well-written, and an easy read, with an eye toward the scientific novice. But the book focuses not so much on science, but on statistics, which are often used as a surrogate for science. Indeed, Mr. Milloy never claims to be a scientist; his biography notes that he has a degree in statistics, making him well-qualified to criticize the misuse of such numbers.
I recently had the opportunity to use the information gleaned from "Junk Science Judo." A local group used a study as evidence that cranberry bogs present a cancer risk, i.e., the hypothesis of the study was that living near cranberry bogs exposed local residents to enough pesticides to cause cancer. While the authors did find a correlation between living near cranberry bogs and a particular kind of brain cancer, it was only a correlation.
This was an ecologic epidemiological study (the worst kind according to Mr. Milloy). The authors never established that the study subjects were exposured to the pesticides used on cranberry bogs; blood or urine samples were never obtained, but a formula was used to estimate exposure levels.
There were other problems; the sample size was too small (only a dozen individuals had brain cancer) to be of value; cancer rates were no higher for those individuals who had known exposures to pesticides (such as pest control or agricultural workers) than for the control group; pesticide use on the cranberry bogs could only be guessed at - the authors did not know what pesticides were used, when they were used or in what quantities.
In short, the study raised more questions than answers. But when I critizied the study and the use of it, the only thing anyone could say was, "Pesticides are toxic." Well, duh. But as Mr. Milloy notes, the dose makes the poison.
Ironically, most of the bad reviews of this book make an issue out of motivation, and appear to be criticizing Mr. Milloy personally, instead of the book and its contents.
Scientists can be activists, and when they are they often hide behind their scientific training, claiming to be impartial even as they advocate a certain point of view (believe me, I've encountered them). And just because a corporation pays for a study does not mean that that study is any less relevant than one paid for by an environmental organization. I've gone so far as to research some of the examples Mr. Milloy uses (such as the Love Canal) and found them to be true.
My point - and that of Mr. Milloy's book - is to look at the claims made with a critical eye; examine the science behind them. In "Junk Science Judo," Mr. Milloy provides us with the information needed to determine if the claims made are true, exaggerated, or completely out of the ballpark. This book is not a collection of blind rants, but a close examination of how public health claims are made and justified (rightly or wrongly).
Get the book, but keep an open mind, because there is no doubt that it will challenge some of your basic assumptions. If you're a natural skeptic, it will give you a foundation upon which to base your skepticism. If you're not a skeptic, it will open your eyes.
on February 24, 2014
Good, well written documentation about phony and insubstantial scares generated by charlatans and picked up by a gullible press and passed on as fact to the unsuspecting public. Everyone should read the sections on statistics to understand the results of experiments that prove nothing, yet are hyped as scientific proof of dangers of one kind or another.
on February 21, 2013
The book, "Junk Science Judo" by Steven Milloy prepares you to be able to identify the junk science that gives you a false sense of security, reduces your wealth and peace of mind, and may get you fired. The fundamental purpose of this book is to defend you, friends, relatives, and your business from junk science. The definition of junk science in the book is, "Falty scientific data and analysis used to advance a special interest" (Milloy 34). There is a ton of junk science in the world as Milloy points out, such as, deception about second hand smoke, chlorinated water, breast implants, over the counter drugs, organic food, and so on. There are overwhelming amounts of people that have a false sense of security about their health from absorbing junk science.
Milloy describes the types of people responsible for junk science in his book. The people involved in junk science range from small time con artists to as big as people inside government agencies. Basically anyone could be a potential candidate for posting junk science, because it is so easy to do. One example of junk science is the claim that grapefruit juice is heart healthy. There is no solid evidence that supports that claim. The con artists posting junk science go out of their way to establish a cause and effect relationship with what they are trying to prove. These people are interested in their own succession, and should not be considered scientists at all.
A really captivating point in the book is that news channels, such as CNN, often report pseudoscience. Reporters are not supposed to make up lies, but the information they broadcast is not always true. Milloy brings up a good point that the news does not like debunking there own news. This book teaches you to have doubts about even the most reliable sources. Often times, many studies that back up a claim are done on small sample sizes, which often makes them useless to the population as a whole. Frequently activists put their own desires before the facts, and will say anything to promote their claim. Junk science even gets through the judicial system, and pollutes the courts and juries. Millions upon millions of dollars have been lost due to junk science legislation.
After reading "Junk Science Judo" what you think is true, may not be true to you anymore. You will see the doubt in many of the claims that you have come across in your life. Think about this question for instance, how could you personally tell if eating cereal gave you cancer? You can't. You have to take someone else's word for it, and they could either have reliable or unreliable data. The fact of the matter is that often times you hear of the unreliable data. You will even realize that politicians use pseudoscience to get elected. Although it is wrong in every way, pseudoscience often means an easy route to the top; that is if you don't get caught. After reading this book you will understand the importance of the scientific method, taking an idea from a theory to a scientific law, and how generations builds upon each other's claims.
This book was easy to read in the sense that there was a lot to catch your attention, and never would you feel stuck on one topic or event. There are countless examples of junk science in this book. Each lesson teaches you more and more about how to defend yourself against junk science. The length of the book made it not discouraging to get through. It wasn't too long, or filled with any dull or useless information. Even if you are a person who is stubborn, Milloy's point that you ought to be, "Better safe than sorry." (Milloy 148) will probably end up saving your skin one day.
The book "Junk Science Judo" shows you the key default assumptions in health risk assessments. Milloy teaches you to become aware that many researchers like to skip the testing stage, and get a conclusion with no proof. One fact is that, "Theories, anecdotes, and assumptions aren't proof of anything" (Milloy 225). I really enjoyed how much this book taught me, and prepared me for a world filled with pseudoscience. For instance when you hear big numbers, it often comes with a large amount of falsity. Statistics are not proof of anything, and many people like to use them because they lack the proof they need. As Milloy states, "Statistics are the lingua franca of junk science." (Milloy 212), which is the truth.
If you are a person who is for animal rights, there are some facts that might be upsetting toward the end of the book or in Lesson 9 to be exact. Junk scientists often get societies attention using Bioassays, or animal experiments that are often times wrongly performed. Basically the idea in this part of the book is that rats differ from humans, and what is claimed to be harmful to them, cannot necessarily be claimed to harm people, but often is. The book goes over all of the epidemiologic basic studies, such as clinical trials, cohort studies, case-control studies, and ecologic studies. There are many charts and examples in the book to give you an idea of what goes on.
By reading "Junk Science Judo" you will realize that much of the junk science you retain has been engraved in your mind throughout your life, and it will be a challenge to distinguish the pseudoscience from actual science. I agree with the author that by the end of the book you will be a black belt in junk science judo. I would highly recommend reading this book because of the life lessons it taught me.
on November 8, 2001
Every day American readers are given scare stories about health issues in news accounts based on some statistics and little analysis. Many accounts are circulated by special interest groups or lawyers. Milloy's background in law and health lends to his ability to point out the fallacies of 'junk science' and tips on how the average reader can separate junk science from real science.
on January 14, 2005
Steven Milloy is a journalist, and pundit whose principled stand on behalf of scientific logic benefits all of us. He stands side by side with Stephen Jay Gould and Michael Schermer in debunking the outrageous claims of pseudo science. Milloy attempts to provide a framework to evaluate the nonsensical claims made by individuals and organizations with agendas which are independent of the facts. It is this framework he calls "junk science judo." Actually what he provides us is the kind of suspicious phraseology and statistical trickery often employed by scammers. In most cases it would take a thorough review of the claimants' research techniques and statistical interpretation to spot the scam. We come away suspicious of almost any claim. All in all, though, Milloy is readable and he is right. Don't miss his column, "Junk Science," on the Opinion page of FoxNews.com.
Postscript: With the passage of time I've become aware of a blind spot in Milloy's vision. It doesn't appear in this book, but it does repeatedly in his columns and website. He denies global warming, which is reliably documented by science. Where junk science enters the debate is in the extent of human causality. To date there is no credible evidence for human causality; it is an issue pressed by political populists and our competitors in global trade. Global warming is good science; human causality is junk science. Science shows that if humans disappeared from the Earth, global warming would continue, and likely continue to accelerate. Milloy is so incensed by the assertion of human causality that he denies warming altogether. In his acceptance of the politicization of the debate, he plays into the hands of his adversaries. His willingness to distort facts in support of this cause unnecessarily introduces doubt about his objecivity in other issues.
If you're tired of being frightened and victimized by the never ending procession of doom-and-gloom prognostications, this book is for you. It lucidly explains how statistics and "scientific" studies are routinely manipulated by groups with their own agendas who are willingly assisted by the media who are all too happy to be given a shocking headline for public consumption. After reading this book, you will never react to alarmist news reports in the same way.
on January 28, 2002
And, those who have no valid counter arguments use ad hominem attacks.
If your "Statistics for Health Professionals" class was as incomplete as mine, then, you need this book. Finally, you will get a clear explanation of p-values, confidence intervals and study designs. Lesson 4: "Epidemiology is Statistics" alone was worth the price of the book. However, don't miss out on the other 11 lessons. It's just like the 12 steps up from alcoholism. By learning and living these rules, you will at last be able to free yourself from those maddening health scares that bombard us daily. Milloy writes in a sassy, bold as brass style that entertains whilst it informs. Personally, I like that sort of thing. This book is an easy read, and if you really, really want to, you can find and read the original studies that are discussed. Then you will appreciate an author who can tackle complex subjects in a straightforward manner, and be funny in the bargain.
"When the enemy comes welcome him, when he goes, send him on his way," is a tenet of the practice of the art of judo. Milloy will show you the enemy, and equip you with the means of speedily dispatching them to the junk pile where they belong. You will truly welcome the next scientific study you encounter.