Probability and Information: An Integrated Approach [Paperback]

David Applebaum (Author)

Book Description

Publication Date: September 1, 2008 | ISBN-10: 052172788X | ISBN-13: 978-0521727884 | Edition: 2

This new and updated textbook is an excellent way to introduce probability and information theory to students new to mathematics, computer science, engineering, statistics, economics, or business studies. Only requiring knowledge of basic calculus, it begins by building a clear and systematic foundation to probability and information. Classic topics covered include discrete and continuous random variables, entropy and mutual information, maximum entropy methods, the central limit theorem and the coding and transmission of information. Newly covered for this edition is modern material on Markov chains and their entropy. Examples and exercises are included to illustrate how to use the theory in a wide range of applications, with detailed solutions to most exercises available online for instructors.

Editorial Reviews

Review

"... the writing is authoritative, and well-tailored to the intended readership. Places where more advanced mathematics is required are indicated clearly, the illustrative material is well-chosen. Any lecturer seeking a text, at this level, to recommend to students, should give this book serious consideration."
John Haigh for Significance

Book Description

This updated, popular introduction to probability theory and information theory, only requiring knowledge of basic calculus, contains new material on Markov chains. Topics covered include statistical inference, random walks, statistical mechanics and communications modelling. Examples and exercises illustrate practical applications with detailed solutions to most exercises available online for instructors.

See all Editorial Reviews

Product Details

• Paperback: 250 pages
• Publisher: Cambridge University Press; 2 edition (September 1, 2008)
• Language: English
• ISBN-10: 052172788X
• ISBN-13: 978-0521727884
• Product Dimensions: 9.6 x 6.8 x 0.6 inches
• Shipping Weight: 1.3 pounds (View shipping rates and policies)
• Average Customer Review: 3.0 out of 5 stars  See all reviews (2 customer reviews)
• Amazon Best Sellers Rank: #1,745,554 in Books (See Top 100 in Books)

More About the Author

Discover books, learn about writers, read author blogs, and more.

Customer Reviews

Most Helpful Customer Reviews

2 of 2 people found the following review helpful:

4.0 out of 5 stars The middle way, July 16, 2010

By 

Professor J. Frink (California) - See all my reviews

This review is from: Probability and Information: An Integrated Approach (Paperback)

I bought/read the first edition of this book several years ago. And I quite enjoyed it! I was glad to see that it had been followed by a second edition, but surprised to see that it had only rated "two stars".

After reading the earlier review, I thought "It wasn't that bad, was it?". So I went back and looked at my copy, and sure enough I had pencil-corrected several errors (one couldn't call them typos, because they recurred). I already had a sufficient background in probability theory, so the errors were immediately recognizable. But I have to admit that a naive reader would have been seriously confused by the definition/notation of conditional probability and Bayes' Theorem, if only going by the slipshod presentation in section 4.3.

[The problem starts on p.43 (of the 1st edition): "...In light of our knowledge of B, the probability of A now changes to P_B(A), which we call the conditional probability of B given A. <Oops!> ... Note: Most textbooks write P_B(A) as P(A/B). <OK, here.> We shall not use this latter notation as it misleadingly suggests that conditional probability is the probability of some set A/B constructed from A and B." A noble intention that goes completely awry in the following several pages. In fact, it looks like somebody (maybe an editor, rather than the author) decided to "fix" the notation, but only converted half of the instances, leaving a confused mess. In this section, there are also two places where they wrote "continuous probability" rather than the intended "conditional probability". Somebody was having a bad day!]

Be that as it may, I still have a high opinion of the author's intentions and, for the most part, execution of the final product. At the time when I found this book, it was the best "middle way" for getting an introduction to Information Theory--more depth than found in Pierce, but not the full baptism of Cover & Thomas. I see that the 2nd edition of this book has added a tenth chapter, introducing stochastic processes and Markov chains. Judging from the 1st edition, I would expect this to be a valuable "middle way" through those topics, as well.

Any 2nd edition owners care to comment?

4 of 9 people found the following review helpful:

2.0 out of 5 stars Not very helpful, July 23, 2008

By 

Michael Johnson (New Brunswick, NJ) - See all my reviews
(REAL NAME)   

This review is from: Probability and Information: An Integrated Approach (Paperback)

I've only read half this book, and I don't think I'll finish it. I'm a philosophy grad student and I wanted to get more in-depth in my understanding of probability theory and information theory. But I'm finding this book unhelpful.

First, some of the topics are insufficiently covered. Despite the fact that Chapter 2 is devoted entirely to combinatorics, there's all of 1 page that introduces and explains 'nPr' and 'nCr', and the explanations are (a) merely examples or (b) difficult to understand mathematicianese. By 'mathematicianese' I mean saying things like 'clearly there are n ways to pick the first object' when everyone knows there's exactly one way to pick *the* F, which is to pick it, although there are many objects one could pick first (similarly, 'In how many different ways can a group of three ball bearings be selected from a bag containing eight' is a funny way of asking 'How many distinct groups of ball bearings can one select from a bag of eight?') I understood the equations in the book, but not the concepts, and had to find an online tutorial before I could proceed.

Second, if you're working independently, doing the problems isn't very helpful. This is because instead of solutions being formulae (as when the problem is 'if 2 balls are drawn from a container of n red and m blue balls, what's the probability that they're both red?') they're often rather numbers (as when the problem is 'if 2 balls are drawn from a container of 5 red and 3 blue balls, what's the probability that they're both red?'). This makes it difficult to see from the answers in the back of the book (a) what one did wrong, when one gets a problem wrong and (b) how to get the problem right.

Third, I just found a lot of the book's cutesyness annoying and absurd. For instance, the author takes sets to be lists of symbols. This is fine at first, but then he says things like 'the number 5 is in set S' as though numbers were symbols, and 'the set R is larger than the set N' as though there could be more than countably many symbols, and then uses the set comprehension notation in inconsistent ways, e.g. '{x: x in S}' as though 'x' ranged over symbols and '{x: x + 2}' as though 'x' ranged over numbers. Also on p47 the author gets Bayes's Theorem wrong (!!!) and he then "proves" his incorrect formulation. [For the record, the author's formulation is P(B/A) = P(A/B)P(A)/P(B). Exercise for the reader: prove a contradiction from this statement (Hint: first prove P(A) = P(B) for all A, B)]

I see that there's a second edition coming out, so I imagine it will correct the p47 error. But I think it just tells of general sloppiness in argumentation where the author makes leaps and the reader, who is trying to learn, can't really follow.