Statistical and Inductive Inference by Minimum Message Length (Information Science and Statistics) [Hardcover]

C.S. Wallace (Author)

Book Description

ISBN-10: 038723795X | ISBN-13: 978-0387237954 | Publication Date: May 26, 2005 | Edition: 1

The Minimum Message Length (MML) Principle is an information-theoretic approach to induction, hypothesis testing, model selection, and statistical inference. MML, which provides a formal specification for the implementation of Occam's Razor, asserts that the ‘best’ explanation of observed data is the shortest. Further, an explanation is acceptable (i.e. the induction is justified) only if the explanation is shorter than the original data. This book gives a sound introduction to the Minimum Message Length Principle and its applications, provides the theoretical arguments for the adoption of the principle, and shows the development of certain approximations that assist its practical application. MML appears also to provide both a normative and a descriptive basis for inductive reasoning generally, and scientific induction in particular. The book describes this basis and aims to show its relevance to the Philosophy of Science. Statistical and Inductive Inference by Minimum Message Length will be of special interest to graduate students and researchers in Machine Learning and Data Mining, scientists and analysts in various disciplines wishing to make use of computer techniques for hypothesis discovery, statisticians and econometricians interested in the underlying theory of their discipline, and persons interested in the Philosophy of Science. The book could also be used in a graduate-level course in Machine Learning and Estimation and Model-selection, Econometrics and Data Mining. "Any statistician interested in the foundations of the discipline, or the deeper philosophical issues of inference, will find this volume a rewarding read." Short Book Reviews of the International Statistical Institute,  December 2005

Editorial Reviews

Review

From the reviews: "The subject matter is highly technical, and the book is correspondingly detailed. The book is intended for graduate-level courses, and should be effective in that role if the instructor is sufficiently expert in the area. For researchers at the postdoctoral level, the book will provide a wealth of information about the field.… [T]he book is likely to remain the primary reference in the field for many years to come." (Donald RICHARDS, JASA, June 2009, Vol. 104, No. 486) "Any statistician interested in the foundations of the discipline, or the deeper philosophical issues of inference, will find this volume a rewarding read." (International Statistical Institute, December 2005) "This very significant monograph covers the topic of the Minimum Message Length (MML) principle, a new approach to induction, hypothesis testing, model selection, and statistical inference. … This valuable book covers the topics at a level suitable for professionals and graduate students in Statistics, Computer Science, Data Mining, Machine Learning, Estimation and Model-selection, Econometrics etc." (Jerzy Martyna, Zentralblatt MATH, Vol. 1085, 2006) "This book is around a simple idea: ‘The best explanation of the facts is the shortest’. … The book applies the above idea to statistical estimation in a Bayesian context. … I think it will be valuable for readers who have at the same time strong interest in Bayesian decision theory and in Shannon information theory." (Michael Kohler, Metrika, Vol. 64, 2006)

From the Back Cover

The Minimum Message Length (MML) Principle is an information-theoretic approach to induction, hypothesis testing, model selection, and statistical inference. MML, which provides a formal specification for the implementation of Occam's Razor, asserts that the ‘best’ explanation of observed data is the shortest. Further, an explanation is acceptable (i.e. the induction is justified) only if the explanation is shorter than the original data. This book gives a sound introduction to the Minimum Message Length Principle and its applications, provides the theoretical arguments for the adoption of the principle, and shows the development of certain approximations that assist its practical application. MML appears also to provide both a normative and a descriptive basis for inductive reasoning generally, and scientific induction in particular. The book describes this basis and aims to show its relevance to the Philosophy of Science. Statistical and Inductive Inference by Minimum Message Length will be of special interest to graduate students and researchers in Machine Learning and Data Mining, scientists and analysts in various disciplines wishing to make use of computer techniques for hypothesis discovery, statisticians and econometricians interested in the underlying theory of their discipline, and persons interested in the Philosophy of Science. The book could also be used in a graduate-level course in Machine Learning and Estimation and Model-selection, Econometrics and Data Mining. C.S. Wallace was appointed Foundation Chair of Computer Science at Monash University in 1968, at the age of 35, where he worked until his death in 2004. He received an ACM Fellowship in 1995, and was appointed Professor Emeritus in 1996. Professor Wallace made numerous significant contributions to diverse areas of Computer Science, such as Computer Architecture, Simulation and Machine Learning. His final research focused primarily on the Minimum Message Length Principle.

See all Editorial Reviews

Product Details

• Hardcover: 448 pages
• Publisher: Springer; 1 edition (May 26, 2005)
• Language: English
• ISBN-10: 038723795X
• ISBN-13: 978-0387237954
• Product Dimensions: 9.3 x 6.2 x 1 inches
• Shipping Weight: 1.6 pounds (View shipping rates and policies)
• Average Customer Review: 5.0 out of 5 stars  See all reviews (1 customer review)
• Amazon Best Sellers Rank: #536,087 in Books (See Top 100 in Books)
  • #65 in Books > Computers & Technology > Computer Science > Artificial Intelligence > Machine Learning

More About the Author

Discover books, learn about writers, read author blogs, and more.

Customer Reviews

Most Helpful Customer Reviews

9 of 10 people found the following review helpful:

5.0 out of 5 stars Table of Contents, October 4, 2005

By 

Charles R. Twardy (Charlottesville, VA) - See all my reviews
(REAL NAME)   

This review is from: Statistical and Inductive Inference by Minimum Message Length (Information Science and Statistics) (Hardcover)

I may never finish a proper review. So let me at least present the Table of Contents and a quick guide for Philosophers of Science. The non-technical chapters are rich with new (or uncommon) insights about Induction, Explanation, Theory Choice, and even the Arrow of Time.

Table of Contents
------------------------------------------
1. Inductive Inference
2. Information
3. Strict Minimum Message Length (SMML)
4. Approximations to SMML
5. MML: Quadratic Approximations to SMML
6. MML Details in Some Interesting Cases
7. Structural Models
8. The Feathers on the Arrow of Time
9. MML as a Descriptive Theory
10. Related Word

Philosophers of Science should read:
-------------------------------------
* Chapter 1, at least through 1.5 (18 pages).
- - MML as a precise rendering of Occam's Razor
- - Why the best explanation is the shortest
- - MML as induction (pure Bayesianism as deductive)
- - MML explanations, induction, unification, etc.
* Chapter 9 (12 pages)
- - The big picture: can you use MML to describe scientific revolutions?

Those interested in the Arrow of Time should read
---------------------------------------------------------
* Chapter 8 (38 pages)
- - A very careful account of reversibility and irreversibility.
- - Accurate simulations to convince people not already convinced of Boltzmann's claim that entropy will increase in both directions.
- - A novel account of asymmetry, suggesting that while we predict the future, we EXPLAIN the past, in the MML sense. That is, MML inference naturally picks out the past we remember, as it is the best explanation of the present.

Anyone wanting more details should begin with:
----------------------------------------------
* Chapter 2 (87 pages)
- - Shannon Information, coding, and entropy
- - Algorithmic Complexity
- - Information, Inference, and Explanation