Markov Chains: Gibbs Fields, Monte Carlo Simulation, and Queues (Texts in Applied Mathematics) Hardcover – March 1, 2008

this was a book i quite liked when i was in grad school, enough for me to go back and buy it even though i had a mediocre PDF scan of the book. it is definitely aimed at advanced undergraduate to starting grad students with quick overview of probability in terms of sigma algebra and measure theory, and then it gets right into it with markov chains, linear algebra, blah blah. the main points i have for this book are
- it's not an intro probability book, make sure you are comfortable with probability to some extent and maybe some random processes. if you want an intro probability book, maybe the Bertsekas would be ok?
- the examples are relevant if a little dated, like not a lot of ppl now probably even remember ALOHA protocol being a thing
- it helps to know some linear algebra and stuff
- it goes over gibbs fields, MCMC, stuff like that in detail and why it works

Bremaud is a probabilist who mainly writes on theory. This is no exception. It is an advanced mathematical text on Markov chains and related stochastic processes. As with most Markov chain books these days the recent advances and importance of Markov Chain Monte Carlo methods, popularly named MCMC, lead that topic to be treated in the text. It is interesting that the other amazon reviewers emphasize the queueing applications. Queueing theory isn't really covered until Chapter 9. The book is rigorous and deep mathematically. It includes stochastic calculus in Chapter 9, a difficult topic even for probabilists.
For a more detailed chapter by chapter account of the book's contents see the review by Carlson.

This book gives a general overview of the more classical results and tools in queuing theory and Monte Carlo simulation. The author gives a review of probability theory in Chapter 1. Measure theory and real analysis are not used here nor in the rest of the book so it should be accessible to advanced undergraduates and graduate students. Random walks and stochastic automata are used as examples of discrete-time Markov chains, the topic of Chapter 2. The author's discussion is very clear and concise and easy to follow. The important topics of ergodicity and recurrence are discussed in Chapter 3, in which the author gives a nice definition of the invariant measure, a concept which is usually presented much too abstractly in modern texts. He gives a very nice elementary example of an irreducible, homogeneous Markov chain that has an invariant measure, but that is not recurrent.Network engineers will appreciate the example of instability in the slotted ALOHA protocol in this section. Physicists will like his discussion of the ergodic theorem in the last section of the chapter. The next chapter discusses the important topic of convergence to equilibrium, and he does a good job of doing this without using real analysis or measure theory. The definition of convergence relies on a concept of distance in variation between two probability distributions. A brief discussion of thermodynamic irreversibility is given in this chapter, and illustrated nicely with Newton's law of cooling. The concepts of absorption and renewal are treated in this chapter in a way that is very intutive and understandable. A cute example of absorption is given in terms of sister-brother mating in genetics. Chapter 5 covers martingales and Lyapunov functions, of current interest to financial engineers and network engineers. The author returns to the slotted ALOHA example of Chapter 3 and shows that altering the retransmission probability will give stability to the protocol. He discusses the practical limitations of this solution to instability by an analysis of collision-resolution protocols subsequently. Potential theory is introduced in this chapter and the author again manages to do it without real or complex analysis. The important Perron-Frobenius theorem is discussed in the next chapter on nonhomogeneous Markov chains. The genetics example introduced earlier is given as one where the eigenvalues of the transition matrix are computed. The author shows that the eigenvalue problem for transition matrices is tractable for the reversible case. Chapter 7 deals with Monte Carlo simulation and the author does a fine job of introducing the concept of a random fields. He does get into some topological notions here, by defining neighborhood systems on the elements of a set. This might have been a problem for the reader not versed in point set topology, but the author gives the reader good insight by relating these notions to vertices and edges of a graph. Gibbs distributions are introduced in a manner similar to what would find in an undergraduate book on statistical physics. The Ising model and neural networks are used to illustrate Gibbs models. The author discusses the Gibbs-Markov equivalence in a nice, detailed manner.Those interested in computer graphics will enjoy his discussion of image models and textures in the third section of this chapter, and again in the next section on Bayesian restoration of images. Section 5 is all physics, where magnetization and the Ising model dominate the discussion. The simulation of random fields, along with the all-important Markov chain Monte Carlo method are the topics of the next two sections. The discussion of MCMC is definitely the best part of the entire book. The Metropolis algorithm is discussed in detail. The last section of the chapter discusses simulated annealing and the discussion is again made very intuitive and avoids the usual multivariable calculus. Chapter 8 deals with continuous-time Markov models and the author does a good job, via examples and proofs, of making the relevant notions clear and understandable. Infinitesimal generators are introduced via stochastic matrices, and jump processes, so important to financial engineering, are discussed in this chapter. The author does discuss the Kolmogorov equation in this chapter, giving the reader a nice taste of stochastic differential equations. Queuing theory dominates the last chapter of the book, via Poisson systems. Readers get their first taste of stochastic discrete-event dynamical systems, and network modeling engineers will definitely want to pay close attention to this chapter. The mathematical manipulations could be viewed as a warm-up maybe to later advanced reading on Ito/Stratonovich calculus, and so financial engineers will be interested in this discussion. All the standard queuing theory of networks is discussed in the last sections of the book. Students of network engineering, physics, financial engineering, and mathematics will want to take a look at this book. Its price is reasonable and it is packed with information and insight. Definitely worth reading.

Markov chains are an important topic in statistics, with numerous applications in computing and engineering. If you have never dealt with these, and desire an advanced text, then Bremaud's book can be of interest.

It explains key ideas, like a transition matrix. Which is usually defined for a discrete time Markov process, but can in fact be generalised to an infinite dimensional state space. From the matrix approach, we get a transition graph and cycles. Then, ergodic behaviour is studied, with invariant measures being found to characterise a given chain.

Bremaud also covers applications of Markov chains. He treats phase transitions and spontaneous magnetisation. Then there are queuing problems and Monte Carlo simulations. The latter can be used in simulated annealing; which in turn can be put to a wide range of problems.

From the context of a network engineer with basic mathematics training, this is a useful book but a little tough going at first. However, with a little help with a more introductory book on Markov chains and stochastic processes I quickly started actually understanding and using the concepts in the book. It gives a very good insight into the operating characteristics of queues, especially with regards to Slotted Aloha. I would definitely recommend this book to anyone with a more than casual interest in queueing theory, but be prepared for a little hard work if you have limited training in probability theory concepts.

This book was saved from 1 star by its many examples - including modern ones like stability analysis of the ALOHA protocol. Other than that, there is nothing in this book that is not better presented elsewhere. Instead of developing the readers intuition this book immediately assaults the reader with rigor and notation. Key concepts are completely unmotivated or so hopelessly muddled that the reader is left wondering where a particular line of reasoning is going or why it is important. There are numerous typographical errors and the typesetting is difficult to tolerate, especially for the long periods of time it will invariably take to get through this tome. Much of the material in this book is better covered in the standard random processes book of Karlin and the second volume of Feller.