Scientific Reasoning: The Bayesian Approach Paperback – April 10, 2005

This book is a little-known treasure in the philosophy of science that deserves a spot alongside the better known works of Popper, Kuhn, Lakatos, and Feyerabend, and is more practical than most of those. Herein lies the clearest, simplest, and most persuasive discussion I've ever seen on the limits of Karl Popper's view of science, along with a sound introduction to the Bayesian probability theory requiring no more than high school algebra and a little persistence.
Much of this book will strike students of classical probability theory and philosophy of science as very counter-intuitive at first, but it is so well argued and so clear that I think most readers will begin to warm up to the Bayesian view at least to some degree by the time they finish the book.
The book starts out introducing one version of the traditional "problem of induction": 'how can we be certain of a rule inferred from finite individual observations ?' We then quickly discover why the usual solutions offered don't quite work in actual theory construction in practice. Mainly, the usual solutions (generally based on the disconfirmation of hypotheses) don't address the way _auxilliary_ hypotheses help theories escape refutation, and how webs of evidence of different kinds often converge to help confirm theories.
It has been generally accepted by modern philosophers of science that useful scientific theories go well beyond the experimental data. Hence they can technically not be "proven" in a logical sense, only considered increasingly more likely as their testable predictions are validated.
The Bayesian view is not based so much on a negative attitude toward objective confirmation of theories, as on the observation that classical methods which are the guardians of total objectivity, in fact violate that ideal constantly and in arbitrary ways. The most objective methods, such as those of Fisher and Neyman and Pearson are credibly claimed to rely on personal judgement of likelihood at key points, rather than being the objective logical consequences generally assumed of them.
The Bayesian view starts off acknowledging that subjective assessment of likelihood is an important part of theory selection and construction, and makes it part of the philosophy of science. The central point is that we have degrees of belief in theories, and that these degrees of belief adhere to probability calculus.
The power of scientific reasoning then results not from some elusive objective logic of discovery but because our innate inference abilities lead observation of evidence to beliefs that follow probability calculus, and hence our sense of increasing credibility tends to reflect greater likelihood of a theory making accurate predictions. Although our inferences are not consistently Bayesian by any means, our own intuitions about what represents *correct* inductive reasoning _are_ Bayesian in nature. So when we take pains to correct our inferences based on our own standards of tenability, our subjective assessments lead us to increasingly better theories.
Our beliefs can be measured as probabilities, and probabilities can be used to confirm theories. Among other things, the Bayesian view uniquely predicts, in contrast to the classical view of Popper and statistician Fisher, that novel observations should have and do have special importance in theory construction. The authors not only introduce probability calculus in simple algebraic terms and discuss its application to philosophy of science, but they also devote considerable time to exploring specific weaknesses of alternate views, and considerable time persuasively addressing the strongest criticisms of the Bayesian approach, such as that it is "too subjective." But the Bayesian philosophy of science is actually built on a powerful theory of inference and is itself "unimpeachably objective" because of its strict rules of consistency, even though its subject matter is subjective degrees of belief.
If you've ever wondered exactly what the Bayesian approach to probability is, and what it is supposed to offer science, or you've ever been dissatisfied with the traditional answers to the problem of induction, this book will be your welcome friend for a number of evenings. It combines mathematical elegance and deftness with simple philosophical wisdom and deals convincingly with the controversial nature of its claims.

Now in its third edition, Scientific Reasoning: The Bayesian Approach, is a basic introduction to the philosophy that scientific reasoning is, and should be, conducted in accordance with the axioms of probability. Called the Bayesian view, after a theorem first proven by Thomas Bayes in the late eighteenth century, has recently gained increased standing as a valuable methodology for examining scientific evidence. Scientific Reasoning explains the elements of probability calculus that are relevant to Bayesian methods and argues that probability calculus should be understood as a form of logic. Accessibly written, even to readers who understand only the basics of probability or calculus, Scientific Reasoning is a solid explanation of how Bayesian theory offers a unified and highly satisfactory accounting of scientific procedure, and is an absolute "must-read" for all scientists and students of science.

If you have good command of elementary statistics, this is a good
first book for someone who is interested in practical uncertainty
quantification, that would like to learn about the Big Picture.
It is a book about thinking and working like a Bayesian, rather
than about techniques of Bayesian estimation.

This book contains lots of useful information for the budding Baysian. Excellent discussions on many topics. However, I have to give this only 3 stars, because on a cardinal point, the authors give very bad advice: they give the impression that Komogorov complexity-based methods are ill motivated. In fact, Kolmogorov complexity is one of the most fruitful new developments in Baysianism, and I have personally used it many times in industrial settings to solve otherwise intractible problems.

However, on most points the book is very useful. I recommend buying the first edition over the second, because the second edition doesn't really add that much useful info over the first. I also recommend buying in addition to this book Ming Li and Paul Vianyi's book on Kolmogorov complexity, for a comprehensive intro to a whole wonderland of Baysianism which Howson & Urbach have overlooked.