From the Publisher
Following a brief introduction and overview, early chapters cover the basic algebraic relationships of entropy, relative entropy and mutual information, AEP, entropy rates of stochastics processes and data compression, duality of data compression and the growth rate of wealth. Later chapters explore Kolmogorov complexity, channel capacity, differential entropy, the capacity of the fundamental Gaussian channel, the relationship between information theory and statistics, rate distortion and network information theories. The final two chapters examine the stock market and inequalities in information theory. In many cases the authors actually describe the properties of the solutions before the presented problems.
From the Inside Flap
Elements of Information Theory is an up-to-date introduction to the field of information theory and its applications to communication theory, statistics, computer science, probability theory, and the theory of investment. Covering all the essential topics in information theory, this comprehensive work provides an accessible introduction to the field that blends theory and applications. In step-by-step detail, the authors introduce the basic quantities of entropy, relative entropy, and mutual information and show how they arise as natural answers to questions of data compression, channel capacity, rate distortion, hypothesis testing, information flow in networks, and gambling. In addition, Elements of Information Theory investigates a number of ideas never before covered at this level in a textbook, including:
* The relationship of the second law of thermodynamics to Markov chains
* Competitive optimality of Huffman codes
* The duality of data compression and gambling
* Lempel Ziv coding
* Kolmogorov complexity
* Portfolio theory
* Inequalities in information theory and their consequences in mathematics
Complete with numerous illustrations, original problems and historical notes, Elements of Information Theory is a practical reference for communications professionals and statisticians specializing in information theory. It also serves as an excellent introductory text for senior and graduate students taking courses in telecommunications, electrical engineering, statistics, computer science, and economics.