Probability Theory: The Logic of Science [Hardcover]

E. T. Jaynes (Author), G. Larry Bretthorst (Editor)

Book Description

ISBN-10: 0521592712 | ISBN-13: 978-0521592710 | Publication Date: June 9, 2003

Going beyond the conventional mathematics of probability theory, this study views the subject in a wider context. It discusses new results, along with applications of probability theory to a variety of problems. The book contains many exercises and is suitable for use as a textbook on graduate-level courses involving data analysis. Aimed at readers already familiar with applied mathematics at an advanced undergraduate level or higher, it is of interest to scientists concerned with inference from incomplete information.

Editorial Reviews

Review

"...tantalizing ideas...one of the most useful and least familiar applications of Bayesian theory...Probability Theory [is] considerably more entertaining reading than the average statistics textbook...the conceptual points that underlie his attacks are often right on."
Science

"This is a work written by a scientist for scientists. As such it is to be welcomed. The reader will certainly find things with which he disagrees, but he will also find much that will cause him to think deeply not only on his usual practice by also on statistics and probability in general. Probability Theory: the Logic of Science is, for both statisticians and scientists, more than just 'recommended reading': it should be prescribed."
Mathematical Reviews

"...the rewards of reading Probability Theory can be immense."
Physics Today, Ralph Baierlein

“This is not an ordinary text. It is an unabashed, hard sell of the Bayesian approach to statistics. It is wonderfully down to earth, with hundreds of telling examples. Everyone who is interested in the problems or applications of statistics should have a serious look.”
SIAM News

"[T]he author thinks for himself...and writes in a lively way about all sorts of things. It is worth dipping into it if only for vivid expressions of opinion...There are many books on Bayesian statistics, but few with this much color."
Notices of the AMS

Book Description

Going beyond the conventional mathematics of probability theory, this study views the subject in a wider context. It discusses new results, along with applications of probability theory to a variety of problems. The book contains many exercises and is suitable for use as a textbook on graduate level courses involving data analysis. Aimed at readers already familiar with applied mathematics at an advanced undergraduate level or higher, it is of interest to scientists concerned with inference from incomplete information.

Product Details

• Hardcover: 758 pages
• Publisher: Cambridge University Press (June 9, 2003)
• Language: English
• ISBN-10: 0521592712
• ISBN-13: 978-0521592710
• Product Dimensions: 9.9 x 7.1 x 1.6 inches
• Shipping Weight: 3.4 pounds (View shipping rates and policies)
• Average Customer Review: 4.8 out of 5 stars  See all reviews (21 customer reviews)
• Amazon Best Sellers Rank: #78,590 in Books (See Top 100 in Books)
  • #35 in Books > Professional & Technical > Professional Science > Physics > Mathematical Physics

More About the Author

Discover books, learn about writers, read author blogs, and more.

Customer Reviews

Most Helpful Customer Reviews

137 of 140 people found the following review helpful:

5.0 out of 5 stars The most important book on probability theory in decades, August 28, 2003

By 

Kevin S. Van Horn "ksvh" (Orem, UT USA) - See all my reviews
(REAL NAME)   

This review is from: Probability Theory: The Logic of Science (Hardcover)

Reading this book is an exhilarating intellectual adventure. I found that it shed light on many mysteries and answered questions that had long troubled me. It contains the clearest exposition of the fundamentals of probability theory that I have ever encountered, and its chatty style is a pleasure to read. Jaynes the teacher collaborates fully with Jaynes the scientist in this book, and at times you feel as if the author is standing before you at the blackboard, chalk in hand, giving you a private lesson. Jaynes's advice on avoiding errors in the application of probability theory -- reinforced in many examples throughout the book -- is by itself well worth the price of the book.

If you deal at all with probability theory, statistics, data analysis, pattern recognition, automated diagnosis -- in short, any form of reasoning from inconclusive or uncertain information -- you need to read this book. It will give you new perspectives on these problems.

The downside to the book is that Jaynes died before he had a chance to finish it, and the editor, although capable and qualified to fill in the missing pieces, was understandably unwilling to inject himself into Jaynes's book. One result is that the quality of exposition suffers in some of the later chapters; furthermore, the author is not in a position to issue errata to correct various minor errors. Volunteer efforts are underway to remedy these problems -- those who buy the book may want to visit the "Unofficial Errata and Commentary" website for it, or check out the etjaynesstudy mailing list at Yahoo groups.

74 of 75 people found the following review helpful:

5.0 out of 5 stars Invaluable, June 27, 2003

By 

brainowner (NYC, USA) - See all my reviews

This review is from: Probability Theory: The Logic of Science (Hardcover)

This book has been on the web in unfinished form for a number of years and has shaped my scientific thinking more than any other book. I believe it constitutes one of the most important scientific texts of the last hundred years. It convincingly shows that "statistics", "statistical inference", "Bayesian inference", "probability theory", "maximum entropy methods" , and "statistical mechanics" are all parts of a large coherent theory that is the unique consistent extension of logic to propositions that have degrees of plausibility attached to them. This is already a theoretical accomplishment of epic proportions. But in addition, the book shows how one actually solves real world problems within this frame work, and in doing so shows what a vastly wider array of problems is addressable within this frame work than in any of the forementioned particular fields.
If you work in any field where on needs to "reason with incomplete information" this book is invaluable.

As others have already mentioned, Jaynes never finished this book. The editor decided to "fill in" the missing parts by putting excercises that, when finished by the reader, provide what (so the editor guesses) Jaynes left out. I find this solution a bit disappointing. The excercises don't take away the impression that holes are left in the text. It would have been better if the editor had written the missing parts and then printed those in different font so as to indicate that these parts were not written by Jaynes. Better still would have been if the editor had invited researchers that are intimately familiar with Jaynes' work and the topic of each of the missing pieces to submit text for the missing pieces. The editor could then have chosen from these to provide a "best guess" for what Jaynes might have written.

Finally, there is the issue of Jaynes' writing style. This is of course largely a matter of taste. I personally like his writing style very much because it is clear, and not as stifly formal as most science texts. However, some readers may find his style too belligerent and polemic.

41 of 41 people found the following review helpful:

4.0 out of 5 stars usually insightful, sometimes annoying, always challenging, May 9, 2005

By 

Neal Alexander (Cali, Colombia) - See all my reviews
(REAL NAME)   

This review is from: Probability Theory: The Logic of Science (Hardcover)

From a few common sense requirements, the books starts by deriving basic results such as the product and sum rules, for probabilities defined not in terms of frequencies, but as degrees of plausibility. This was an eye-opener for me, having imbibed the common attitude that such probabilities are 'subjective' and, implicitly, lacking rigor and utility.

Jaynes' knowledge of the history and philosophy of statistics is far deeper than that of most statisticians (including myself). His trenchant style gives the book a narrative drive and cover-to-cover readability that, in my experience, is unique in the field. One such strand is the continual battle between his respect for RA Fisher's abilities, and his exasperation at how wrongheadedly he feels they were channelled. And he doesn't hesitate to take on philosophical heavyweights such as Hume in defending the possibility - - in fact, the necessity - - of inductive inference. However, this style also produces some more bitter fruit, such as the way the author repeatedly likens himself to historical victims of religious persecution.

The book weakens when it turns to applications. Regression with errors in both variables is said to be 'the most common problem of inference faced by experimental scientists' who have 'searched the statistical literature in vain for help on this'. Good points. So why don't the author and editor give us at least a reference for just one of the 'correct solutions' which 'adapt effortlessly' to scientists' needs? And Jaynes' argument that the null hypothesis procedure 'saws off its own limb' would also rule out mathematical proof by reductio ad absurdum.

When estimating periodicities, we're told that 'the eyeball is a more reliable indicator of an effect than an orthodox equal-tails test'. So why not show us the data of the example used, to let us use our eyes? In fact, there's only one graph of empirical data in all the book's 600+ pages.

Several convincing arguments are presented for the use of the Jeffreys (reciprocal) prior for scale parameters, including scale independence. However, just when I was ready to go and use it, there's a warning against the use of improper priors except as 'as a well-defined limit of a sequence of proper priors'. A few pages later a uniform prior is used for the mean of a Gaussian, with no such justification as a limit, which makes it far from clear what exactly is being recommended.

I could give a lot more space to the book's many other insights, and several other annoyances. Instead, I'll finish now by recommending it to anyone interested in the foundations and practice of statistical analysis.