Customer Reviews

Information Theory (Dover Books on Mathematics)

5 star:		(5)
4 star:		(0)
3 star:		(2)
2 star:		(0)
1 star:		(0)

 

 

This is the best book for self-study of information theory which I have found, and I looked hard because I needed to learn the basics of information theory in grad school. As far as content, Ash covers all the major topics in information theory, from definitions of basic quantities like mutual information to the mathematical representation of continuous communication channels. One of the best aspects of the book is a set of problem sets at the end of each chapter, each with detailed solutions at the end of the book. They serve as very useful checks on one's understanding. As for structure, Ash manages to cover these topics in a way that is concise and illuminating yet without sacrificing mathematical rigour (note that the book assumes you know basic probability theory and calculus). If anyone wants to learn the mathematical theory of communication, I highly recommend using this book as your guide.

This book is highly similar to the Reza book, also published by Dover publications. The Ash book kind of continues where the Reza book leaves off. In truth, this book is very, very rigorous... not so much in terms of proofs (see the small Khinchin book for great proofs), but in terms of it involves mathematics and concepts which require a higher level of knowledge. Undergraduate students would have alot of trouble trying to understand both math and general concepts.
Even graduate students would find this book daunting, because after all, it probably is one of the best books written on information theory.

If your a beginner seeking a good book, this is not it at all.
Aside from being too rigorous, it covers many topics which are of completely no use to a beginner or even somebody with a fair amount of information theory knowledge. Also, the book is not very motivating from a practical aspect. That is, much like the Reza and Kitchkin book, it's written more from a dry mathematical perspective and not an "engineers" perspective.
It doesn't examine information theory from the perspective of electrical engineering and communications theory... which might make it hard for some people to relate to if they can't be told what the practical applications are (see Pierce's books and Cover and Thomas for very good "practical" books).

For beginners, I recommend the Pierce book, subtitled "Symbols, Signals and Noise" which is bar-none the best beginners book ever written (or some of Pierce's other books). Pierce is one of the finest authors of his era and he published several books on information theory; most of which are more "engineer friendly" and are more relavent to the study of electronic communications.

Summary, this book is NOT for beginners. It will be almost completely useless unless you have a decent degree of information theory knowledge to begin with. Sadly, this was the first book I ever purchased on that topic.. and boy was that a mistake!! I spent 2 years trying to figure heads or tails of half the chapters.. Then I went ahead and got some more appropriate books (Pierce, Reza, Cover and Thomas) and when I had sufficient knowledge... only then did this book make any sense.

This 1990 Dover publication of the original 1965 edition serves as a great introduction to "the statistical communication theory", otherwise known as Information Theory, a subject which concerns the theoretical underpinnings of a broad class of communication devices. The exposition here is based on the Shannon's (not Wiener's) formulation or model of the theory, having been initiated in his breakthrough 1948 paper. I purchased this book more than a couple of years ago as a beginning math grad student mainly interested to (quickly and affordably) learn some basics about the subject, without necessarily intending to specialize in it. The text in my opinion should also be accessible to any engineering student with a one or two semester background in real analysis, and a working knowledge of the theory of probability (also summarized at the beginning of the book). Topics discussed include: noiseless coding, discrete memoryless channels, error correcting codes, information sources, channels with memory, and continuous channels. There are some very illuminating historical notes + remarks, and also problem sets at the end of each chapter, with solutions included at the back of the book, making an ideal setting for self-study. Aside from being a great resource for learning the basics however, one sole setback of the book is that all the results and theorems presented therein date from the 50's and early 60's, so one will have to look elsewhere to find out about some of the more recent developments in the field.

I know what you're saying - Dover books have a reputation for publishing crap books, right? This book is just too cheap to be any good, right? Well, think again. This book is a no nonsense introduction to classical information theory. By no-nonsense I mean it does not have chapters like most books out there on "information and physics", "information and art", or all sorts of pseudo scientific popularizations of information theory. It does one thing: present with a minimum of hassle and with a maximum of details and examples the mathematical and conceptual framework of information theory, nothing more, nothing less. On the other hand, it manages to avoid the old "theorem-lemma-corollary" format of many other ultra-dense math books out there. This book actually makes an effort to explain where the math fits in conceptually. When introducing a new concept, it always accompanies the definition with an example. This is even true when proving a complicated theorem. Add to these virtues the interesting problems at the end of each chapter, each with its own detailed solution at the end of the book, and you've got a pedagogical gem.
It should be noted that the only prerequisite is a prior course in basic probability - conditional probability, Tchebychev's theorem, simple and basic stuff every 2nd-3rd year undergraduate should be familiar with.
If you're looking for the perfect introduction to information theory, look no further, this is it!

The book by Shannon and Weaver (1949) is the classic; Shannon almost *is* information theory. There is more to it: The present lovely little book appeared first in 1965, but is still very relevant. I think it is a good next book to read. At least the mathematical part of the subject stays more constant over the years, as do the fundamental principles;-- that is what Ash's book is about. I especially liked ch 4 on error correcting codes, and the mathematical appendix which is centered around the Karhunen-Loeve theorem;-- the latter having found recent exciting applications in wavelet theory.

Although this book is kind of old, it gave almost complete proof for every important theorem for information theory. It is much better than Shannon orginal book.

My interest in information theory comes from general physics and philosophy. Ash`s book is doubtless excellent for a communications specialist. What I had hoped for, for example, was something enlightening about the relation between thermodynamic entropy and communication entropy with an explanantion of the position of Boltzmann`s constant betweren the two. Thermal noise according to the Nyquist formula might provide a link. But Nyquist doesn`t get a mention.
In short the book was not what I needed. Probably not the author`s fault but if you are not already, or not anxious to be a dedicated communication specialist then beware.