Customer Reviews

Probability and Information: An Integrated Approach

5 star:		(0)
4 star:		(1)
3 star:		(0)
2 star:		(1)
1 star:		(0)

 

 

I bought/read the first edition of this book several years ago. And I quite enjoyed it! I was glad to see that it had been followed by a second edition, but surprised to see that it had only rated "two stars".

After reading the earlier review, I thought "It wasn't that bad, was it?". So I went back and looked at my copy, and sure enough I had pencil-corrected several errors (one couldn't call them typos, because they recurred). I already had a sufficient background in probability theory, so the errors were immediately recognizable. But I have to admit that a naive reader would have been seriously confused by the definition/notation of conditional probability and Bayes' Theorem, if only going by the slipshod presentation in section 4.3.

[The problem starts on p.43 (of the 1st edition): "...In light of our knowledge of B, the probability of A now changes to P_B(A), which we call the conditional probability of B given A. <Oops!> ... Note: Most textbooks write P_B(A) as P(A/B). <OK, here.> We shall not use this latter notation as it misleadingly suggests that conditional probability is the probability of some set A/B constructed from A and B." A noble intention that goes completely awry in the following several pages. In fact, it looks like somebody (maybe an editor, rather than the author) decided to "fix" the notation, but only converted half of the instances, leaving a confused mess. In this section, there are also two places where they wrote "continuous probability" rather than the intended "conditional probability". Somebody was having a bad day!]

Be that as it may, I still have a high opinion of the author's intentions and, for the most part, execution of the final product. At the time when I found this book, it was the best "middle way" for getting an introduction to Information Theory--more depth than found in Pierce, but not the full baptism of Cover & Thomas. I see that the 2nd edition of this book has added a tenth chapter, introducing stochastic processes and Markov chains. Judging from the 1st edition, I would expect this to be a valuable "middle way" through those topics, as well.

Any 2nd edition owners care to comment?

I've only read half this book, and I don't think I'll finish it. I'm a philosophy grad student and I wanted to get more in-depth in my understanding of probability theory and information theory. But I'm finding this book unhelpful.

First, some of the topics are insufficiently covered. Despite the fact that Chapter 2 is devoted entirely to combinatorics, there's all of 1 page that introduces and explains 'nPr' and 'nCr', and the explanations are (a) merely examples or (b) difficult to understand mathematicianese. By 'mathematicianese' I mean saying things like 'clearly there are n ways to pick the first object' when everyone knows there's exactly one way to pick *the* F, which is to pick it, although there are many objects one could pick first (similarly, 'In how many different ways can a group of three ball bearings be selected from a bag containing eight' is a funny way of asking 'How many distinct groups of ball bearings can one select from a bag of eight?') I understood the equations in the book, but not the concepts, and had to find an online tutorial before I could proceed.

Second, if you're working independently, doing the problems isn't very helpful. This is because instead of solutions being formulae (as when the problem is 'if 2 balls are drawn from a container of n red and m blue balls, what's the probability that they're both red?') they're often rather numbers (as when the problem is 'if 2 balls are drawn from a container of 5 red and 3 blue balls, what's the probability that they're both red?'). This makes it difficult to see from the answers in the back of the book (a) what one did wrong, when one gets a problem wrong and (b) how to get the problem right.

Third, I just found a lot of the book's cutesyness annoying and absurd. For instance, the author takes sets to be lists of symbols. This is fine at first, but then he says things like 'the number 5 is in set S' as though numbers were symbols, and 'the set R is larger than the set N' as though there could be more than countably many symbols, and then uses the set comprehension notation in inconsistent ways, e.g. '{x: x in S}' as though 'x' ranged over symbols and '{x: x + 2}' as though 'x' ranged over numbers. Also on p47 the author gets Bayes's Theorem wrong (!!!) and he then "proves" his incorrect formulation. [For the record, the author's formulation is P(B/A) = P(A/B)P(A)/P(B). Exercise for the reader: prove a contradiction from this statement (Hint: first prove P(A) = P(B) for all A, B)]

I see that there's a second edition coming out, so I imagine it will correct the p47 error. But I think it just tells of general sloppiness in argumentation where the author makes leaps and the reader, who is trying to learn, can't really follow.