236 of 243 people found the following review helpful
on May 29, 2011
Format: Kindle EditionVerified Purchase
"The Theory That Would Not Die" is an enjoyable account of the history of Bayesian statistics from Thomas Bayes's first idea to the ultimate (near-)triumph of Bayesian methods in modern statistics. As a statistically-oriented researcher and avowed Bayesian myself, I found that the book fills in details about the personalities, battles, and tempestuous history of the concepts.
If you are generally familiar with the concept of Bayes' rule and the fundamental technical debate with frequentist theory, then I can wholeheartedly recommend the book because it will deepen your understanding of the history. The main limitation occurs if you are *not* familiar with the statistical side of the debate but are a general popular science reader: the book refers obliquely to the fundamental problems but does not delve into enough technical depth to communicate the central elements of the debate.
I think McGrayne should have used a chapter very early in the book to illustrate the technical difference between the two theories -- not in terms of mathematics or detailed equations, but in terms of a practical question that would show how the Bayesian approach can answer questions that traditional statistics cannot. In many cases in McGrayne's book, we find assertions that the Bayesian methods yielded better answers in one situation or another, but the underlying intuition about *why* or *how* is missing. The Bayesian literature is full of such examples that could be easily explained.
A good example occurs on p. 1 of ET Jaynes's Probability Theory: I observe someone climbing out a window in the middle of the night carrying a bag over the shoulder and running away. Question: is it likely that this person is a burgler? A traditional statistical analysis can give no answer, because no hypothesis can be rejected with observation of only one case. A Bayesian analysis, however, can use prior information (e.g., the prior knowledge that people rarely climb out wndows in the middle of the night) to yield both a technically correct answer and one that obviously is in better, common-sense alignment with the kinds of judgments we all make.
If the present book included a bit more detail to show exactly how this occurs and why the difference arises, I think it would be substantially more powerful for a general audience.
In conclusion: a good and entertaining book, although if you know nothing about the underlying debate, it may leave you wishing for more detail and concrete examples. If you already understand the technical side in some depth and can fill in the missing detail, then it will be purely enjoyable and you will learn much about the back history of the competing approaches to statistics.
41 of 44 people found the following review helpful
on June 6, 2011
Format: HardcoverVerified Purchase
This book moves through the history (so far) of the development and application of Bayes rule. It is a good story, and the book is well written. Unfortunately, it is somewhat mixed in the manner material is presented. For example, the author provides significant detail on the application of the rule to activities such as code cracking and finding submarines but she then goes on to list a large number of more recent applications with very little historical background. Maintaining consistency of depth for each application discussed would have significantly improved the "story". I would recommend this book to anyone who is interested in the history of science, statistics and mathematics, but be prepared for a "patchy" read.
326 of 399 people found the following review helpful
on June 17, 2011
Format: HardcoverVerified Purchase
Sharon Bertsch Mcgrayne is a talented science writer whose portraits of great scientists of the past are incisive and entertaining. However, she evidently believes that one must studiously avoid dealing with any serious scientific issues in entertaining a popular audience. For this reason, this book is a total failure. Why should a reader care about the history of an idea of which he or she has zero understanding? Mcgrayne turns the history of Bayes rule into a pitched battle between intransigent opponents, but we never find out what the real issue are.
In fact, Bayes rule is a mathematical tautology, being the definition of conditional probability. Suppose A is an event with probability P(A) and B is an event with probability P(B). Let C be the event "both A and B occur." Then the conditional probability P(A|B) of event A, given that we know that B has occurred, just P(C)/P(B). Moreover, if a decision-maker knows P(A), P(B), and P(C), and discovers that B occurred, then he should revise the probability that A occurred to P(A|B) = P(C)/P(B). Why? Well, suppose we have a population of 1000 individuals, where the probability that an event E is true of an individual is P(E), where E is any one of A, B, and C. Then the expected number of individuals for which B is true is 1000*P(B). Of these, the number for which A is also true is 1000*P(A). Therefore, the probability that an individual satisfies A, given that he satisfies B, is 1000*P(A)/1000*P(B) = P(A|B).
For instance, suppose 5% of the population uses drugs, and there is a drug test that is correct 95% of the time: it tests positive on a drug user 95% of the time, and it tests negative on a drug nonuser 95% of the time. If an individual tests positive, we can show using Bayes rule that the probability of his being a drug user is 50%. To see this, let A be the event "subject uses drugs," and let B be the event "subject test positive for using drugs." First, what is the probability P(B) of event B? Well, take a random subject. With probability 1/20 he is a drug user, so with probability (19/20)(1/20)=19/400 he is a drug user testing positive. With probability 19/20 he not a drug user, so he is a non-user testing positive with probability (1/20)(19/20)=19/400. Thus P(B) = 19/400+19/400=38/400. Let event C be "subject uses drugs and tests positive for using drugs." This probability is (1/20) times (19/20) = 19/400. Thus P(A|B) = P(C)/P(B) = 1/2.
If this seems mystifying, consider the following interpretation. Suppose we test 10000 people. The expected number of drug users will be 500, and 95% of them, or 475, will test positive for drug use. But 9500 people will be non-drug users, and 5% of them will erroneously test positive for drug use, which is 475 people. Thus, 50% of those who test positive for drugs are actually drug users.
The real brilliance of Bayes Rule lies in the fact that sometimes we want to find P(A|B) when we don't know either P(C) or P(B), but we do know P(B|A) and P(A). For instance, want to know P(A|B), meaning the probability that an individual who test positive is actually a drug user, but we only know the frequency P(A) of drug use in the population (5%) and the accuracy of the test, which is P(B|A) = 95% (a drug user tests positive with probability 0.95). Then we can write P(A|B)P(B) = P(C)=P(B|A)P(A). From the first and third terms we get P(A|B) = P(B|A)P(A)/P(B). In our case, this gives P(A|B) = 0.95(0.05)/P(B)=0.0475/P(B). But we can also calculate P(B) as follows.
Let N mean "A is false for the subject." Thus P(N) = 1-P(A) = 0.95. Then we have
P(B) = P(B|A)P(A) + P(B|N)P(N), as can be verified by simply multiplying out the right hand side of the equation. Thus in our case we have, given that we know that P(B|N) = 0.95 (the test accurately predicts that a non-user is a non-user with probability 0.95), so we have P(B) = 0.95(0.05) + 0.05(0.95) = 0.095. Thus P(A|B) = 0.0475/0.095 = 1/2.
Isn't this a simple and beautiful result? Only arithmetic and grade school algebra are used to arrive at this stunning result. By the way, for more on Bayes Rule, see my textbook Game Theory Evolving (Princeton 2009).
Now, who could dispute this analysis? It is clearly correct. So where does all of the vehement opposition to Bayes rule come from? The answer is that when a group of individuals (e.g., professional scientists) do not agree on P(A) then you cannot apply Bayes rule. You can however show that under many conditions, repeated observations of events A can lead to mutually acceptable values for P(A). For instance, suppose you know that the weight of a substance per ounce is variable and unknown, and each scientist i has his personal prior probability Pi that the weight is less than one gram per ounce. Suppose we take unbiased samples that are each about one ounce, and we take unbiased measurements of the weight. Then the long-run average of the sample weights will be accepted by all scientists as the updated probability. This is Bayesian updating.
However, it is not true that Bayesian updating always lead to convergence to a common probability distribution. See, for instance, papers by Mordecai Kurz, of Stanford University. Moreover, when observations are limited, the range of assessments of probabilities can be quite wide. This is why Bayes rule is considered "subjective." However, when we really know the probabilities, as in the case of the drug testing example, there is no controversy about the value of Bayes rule. It is extremely valuable, indeed indispensable, in such cases.
This book manages to obfuscate a very simple issue, turning sciences into a vast morality play. Now of course there are deep issues in the philosophy of probability that implicate Bayes rule, but one does not learn what they are from this book.
51 of 62 people found the following review helpful
on April 18, 2012
This book is disappointing, even annoying, primarily because, as at least one other reviewer pointed out, the author critically misrepresents the issue. Specifically, the central implication of the book is that anybody who uses Bayes's rule is a "Bayesian" or is using "Bayesian" logic. In fact, in order to solve probability problems, anybody with a bit of training in probability theory would use the definition of conditional probability (of which Bayes's rule is a corollary so trivial that it generally wouldn't be noticed as a theorem per se) when appropriate: this does _not_ make that person a Bayesian. So, contrary to the author's claim, people who used Bayes's rule (again, just the definition of conditional probability) throughout the 19th and 20th centuries (e.g., Poincare, Alan Turing) were not necessarily Bayesians. The logical extension of this error would be to conclude that every mathematician, scientist, engineer, and really every person who ever took a basic probability course between the years 1800 and 2012, was a Bayesian. For example, Kolmogorov formalized mathematical probability theory into more or less its present form in his 1933 book, and he notes without fanfare in the first 7 pages that his basic axioms lead trivially to Bayes's rule. Thus, according to the author's central false implication (that the Bayesian/frequentist issue is whether one's probability theory is consistent with Bayes's theorem), the mainstream foundations of 20th century probability theory are utterly Bayesian, and so there is no story to tell.
So what does it really mean to be Bayesian or frequentist, and what were the (very real) debates about? The real issue is how probability theory should be mapped onto _statistics_, with Bayesian statisticians taking one approach, and frequentist statisticians taking another. Critically, any statistician would have used Bayes's rule (the definition of conditional probability) when deriving results. However, they disagreed ferociously over issues such as whether or not to treat model parameters like random variables and compute posterior distributions for them (Bayesians), or to treat model parameters like fixed constants and estimate them with confidence intervals (frequentists). My understanding is that the "Bayesian" approach gets its name because it requires using Bayes's rule (the definition of conditional probability) in order to form the parameter's posterior, i.e.,
p(parameter|data) = p(data|parameter)p(parameter)/p(data),
whereas the fixed constant (frequentist) treatment does not. (Indeed, under the fixed constant treatment of the parameter, this application of Bayes's rule does not make sense, because the parameter is no longer a random variable, and thus has neither prior nor posterior distributions.) It is _not_ that Bayesians statisticians are the only statisticians that use the standard definition of conditional probability!
Other than how to treat model parameters, there are a few other statistical points at issue, and they generally touch deeply on fascinating questions about science and engineering. Moreover, the personalities involved were indeed ferocious, and there is surely an interesting history to be told. Unfortunately, this book butchered the opportunity by falsely claiming that people like Alan Turing had something to do with the Bayesian/frequentist debate, while at the same time not touching at all on any of the true and interesting issues.
17 of 19 people found the following review helpful
on August 26, 2011
Format: HardcoverVerified Purchase
While I completely agree with those reviewers who expressed disappointment with the lack of technical depth in the author's presentation, what I find even more disturbing is her abuse of reference material. References appear in two sections: in footnotes to each chapter, and then, again, as a bibliography with separate sections for each of the book chapters. This leads to some (expected and excusable) duplication, but there are indefensible gaps, in which footnotes may refer to books or articles that cannot be found in any of the bibliographies, let alone in those dedicated to the footnoted chapter. It is even more unfortunate that many of the references given do not permit the interested reader to fill in the technical and scientific detail of the applications the author so temptingly describes, or, in the case of the missing H-bomb, seems to describe. One is driven to the conclusion that almost none of the text pertains to actual application of Bayes' Rule, or, indeed, to any sort of statistical analysis.
It is quite true that the historical presentation is replete with biographical anecdotes, and they are a joy to those of us interested in the history of statistics (and philosophy and logic) - but, alas, that makes even more frustrating the utter absence of technical, mathematical detail.
Though much of the book purports to be a presentation of the Bayesian vs. Frequentist controversy, it deals with the latter no more deeply or fruitfully than the former. In brief, it is truly a "hands-off" presentation, hence immensely disappointing. If only it were the book the jacket blurbs describe!
40 of 49 people found the following review helpful
on October 7, 2011
As a practitioner who uses Bayes Rule on an almost daily basis, I was hoping to see examples of how it was used in other fields. Instead, this book is rich in details about everything around the subject, but very poor on details of the actual subject matter, even (in my view) ones which would be appropriate for a general audience.
What we're left with is a book of anecdotes, some of which are interesting, but which are mostly useless. The subtitle of the book simply remains unanswered.
20 of 24 people found the following review helpful
on March 12, 2012
The book was a Christmas present. I know, you can't look a gift horse in the mouth, but this one takes our forbearance a step too far. What a waste of time. Here are a few notes.
One, the author may have believed the old myth that each formula would cut her potential audience in half. In any case, the book has only two formulae, both of them a version of Bayes' theorem. This by itself would not be a problem were it not that the most common version of all - p(A/B)=p(B/A)*p(A)/p(B), or if you like p(C/E)=p(E/C)*p(C)/p(E), with C meaning Cause and E meaning Event- did not make it into the book.
Two, regarding the Laplace formulation of Bayes' theorem, the author writes about the formula's denominator : "The [SIGMA] sign in the denominator [...] makes the total probability of all causes add up to one" - and leaves it at that. The sentence is a badly worded truism AND a non sequitur. Yes, the sum of the probabilities of all possible causes is one, but that is not the point of the denominator in the Laplace formula. It is difficult to imagine somebody writing the sentence who is not a complete stranger to mathematics - but maybe the editor did it!
Three, the book covers a large number of cases that benefited, or are claimed to have benefited , from applications of Bayes' theorem. In covering the cases, the book never rises above the level of the banality <<that we learn from experience.>> There are no specifics whatsoever about the mechanics of applying the theorem - no explanation how the Bayes instrument works. If you want to be nice you can say that the book is all conclusory. However, if you want to understand the basics of Bayes' theorem you are in the wrong place.
Four, in nearly all cases, it is not clear what are the precise Cause(s) and Event(s). For example, in the case of fighting the U-boats, what precisely is the Cause and what precisely is the Event? I can guess but I did not read the book for guessing what the author may have had in mind. Worse, it may very well be that the author does not have enough understanding of the theorem or the cases to identify the precise Causes and Events.
Five, some of the claims in the book are on shaky ground. For example, the absence of an accidental explosion of a nuclear bomb is thanks to risk analysis more than to Bayesian analysis.
Six, somewhere in the book the author mentions a certain statistician of some name who turns from a Bayesian into a frequentist. The author sounds surprised - the overall tone of the book is that Bayesians have seen the Light. However, the surprise does not lead the author to question what there could be in Bayesian analysis that turned the particular statistician into a frequentist.
Seven, the author never gets into detail on how Bayesian analysis and frequentism are complementary and how they interact to the benefit of both. Naively, the "triumph" of Bayes means that somebody else must lose.
Eight, it is bad form for an author to put a particular individual in a positive light, and then for the individual to recommend the work on the jacket of the book.
Nine, what were the people at Yale University Press thinking when they decided to put their imprint on this work?
7 of 7 people found the following review helpful
on October 22, 2011
Format: HardcoverVerified Purchase
The 20th century ideological controversy between the Bayesian and frequentist schools of statistics has been studied in scholarly books, and appears as a chapter or so in numerous "popular science" style books on statistics or probability, but this is the first book length account I have read in that style. Alas, the author's chosen style is to be completely non-technical and to tell anecdotes about individuals. This makes pleasant enough bedtime reading, and some of the stories will likely be new even to professional statisticians. But stories ending with the individual "seeing the light" of Bayesianism with only superficial description of the actual content of their work rapidly become tiresome.
My own summary of the actual intellectual issues would have about 10 bullet points, of which the book touches upon maybe 5. For instance she correctly points out the important practical fact that, until computer terminals appeared on statisticians' desks around 1980, it was often impossible to do the Bayesian calculations, whereas applying known frequentist methodology involved only arithmetic and table look-up. But she completely neglects to tell the reader that the basic everyday difference between the viewpoints is not the dramatic one (Bayesians put priors on hypotheses, frequentists don't) but a much more mundane one. Do you view parameters in a model as unknown but fixed (frequentist) or as random, with some distribution you therefore need to specify (Bayes)? Academics continue to enjoy arguing over this, but it's actually a relatively minor issue. If the rest of your model ("likelihood") is realistic and you have enough data then usually both schemes will be reliable, and if not then neither will be reliable.
To see how far the book's air of triumphalism is at odds with contemporary thinking, a search on "what are the open problems in Bayesian statistics" will lead to a March 2011 article by Michael Jordan summarizing thoughts of leading academic Bayesians. Though technical in detail, the basic tone of the comments comes through as entirely moderate and reasoned: they are concerned with matters like computational feasibility, how to justify priors, understanding better how Bayes methods relate to frequentist methods and whether they should adapt some frequentist ideas. Not a single 'I'm going to Disneyland!" to be found anywhere.
28 of 35 people found the following review helpful
on July 26, 2011
Format: HardcoverVerified Purchase
The opening of this book is quite promising, offering to tell the intertwining stories of the social and intellectual histories of Bayes Rule and its modern incarnation, Bayesian analysis. Unfortunately, as several other reviewers have pointed out, the coverage of the mathematical and intellectual properties of Bayes Rule is grossly superficial. I also found much of the second half of the book to be quite poorly written, filled with paragraphs that begin by tossing out a factoid concerning someone who may or may not ever be mentioned again, then wandering about some vaguely related topic, and finishing up with a complete non-sequitor. Perhaps there was pressure to publish by a certain deadline; certainly the editing in the last part of the book suffered badly. I found the final two chapters virtually impossible to get through - they are farragoes of names, ideas, and allusions to achievements of Bayesian analysis, never explained in any depth and rarely connected in any comprehensible fashion. After a promising beginning, I found this book to be a major disappointment.
I am glad to see that a couple of the initial, glowing "reviews" - which consisted solely of blurbs from the book jacket - have been taken down.
27 of 34 people found the following review helpful
on November 9, 2011
Format: HardcoverVerified Purchase
This book is so awful that I am writing my first public review.
I was disappointed with the weak discussion of how the mathematics of Bayes' Theorem provided valuable insights in the different applications the author chose to present. I was disappointed in the constant process of setting up straw man baddies who ignorantly opposed our noble theorem. I was disappointed with the silly political asides that were only story-related in the author's mind. Overall, the book reads like a first draft, mind dump of all the research material the author gathered. The chapters and the various stories tend to open with a thrilling "now it will be revealed...." lead only to lose steam and eventually flatline in time to go to the next potboiler. If only the book had an editor willing to take it to the next level. As it is, the book is simply long, bland, and without insight.