Markov Chains (Cambridge Series in Statistical and Probabilistic Mathematics) [Paperback]

J. R. Norris (Author)

Book Description

ISBN-10: 0521633966 | ISBN-13: 978-0521633963 | Publication Date: July 28, 1998

In this rigorous account the author studies both discrete-time and continuous-time chains. A distinguishing feature is an introduction to more advanced topics such as martingales and potentials, in the established context of Markov chains. There are applications to simulation, economics, optimal control, genetics, queues and many other topics, and a careful selection of exercises and examples drawn both from theory and practice. This is an ideal text for seminars on random processes or for those that are more oriented towards applications, for advanced undergraduates or graduate students with some background in basic probability theory.

Editorial Reviews

Review

"...impressive ....I heartily recommend this book....this is the best book available summarizing the theory of Markov Chains....Norris achieves for Markov Chains what Kingman has so elegantly achieved for Poisson processes....Such creative tinkering will be a pleasure to many teachers." Bulletin of Mathematical Biology

Book Description

Markov chains are central to the understanding of random processes. This textbook, aimed at advanced undergraduate or MSc students with some background in basic probability theory, focuses on Markov chains and develops quickly a coherent and rigorous theory whilst showing also how actually to apply it. There are applications to simulation, economics, optimal control, genetics, queues and many other topics, and a careful selection of exercises and examples drawn both from theory and practice.

Product Details

• Paperback: 254 pages
• Publisher: Cambridge University Press (July 28, 1998)
• Language: English
• ISBN-10: 0521633966
• ISBN-13: 978-0521633963
• Product Dimensions: 10 x 7 x 0.7 inches
• Shipping Weight: 15.2 ounces (View shipping rates and policies)
• Average Customer Review: 3.9 out of 5 stars  See all reviews (8 customer reviews)
• Amazon Best Sellers Rank: #219,904 in Books (See Top 100 in Books)

More About the Author

Discover books, learn about writers, read author blogs, and more.

Customer Reviews

Most Helpful Customer Reviews

21 of 22 people found the following review helpful:

3.0 out of 5 stars Good introductory treatment, but has some issues, May 6, 2005

By 

Paul Thurston (New York, NY USA) - See all my reviews
(VINE VOICE)    (REAL NAME)   

Amazon Verified Purchase

This review is from: Markov Chains (Cambridge Series in Statistical and Probabilistic Mathematics) (Paperback)

This book has two principal aims. In the first half of the book, the aim is the study of discrete time and continuous time Markov chains. The first part of the text is very well written and easily accessible to the advanced undergraduate engineering or mathematics student.

My only complaint in the first half of the text regards the definition of continuous time Markov chains. The definition is introduced using the technical concepts of jump chain/holding time properties. This doesn't tie out well with the treatment of the discrete time case and may seem counter-intuitive to readers initially. However, the author does establish the equivalence of the jump chain/holding time definition to the usual transition probability definition towards the end of Chapter 2.

The second half of the text deals with the relationship of Markov chains to other aspects of stochastic analysis and the application of Markov chains to applied settings.

In Chapter 4, the material takes a serious jump (explosion?) in sophistication level. In this chapter, the author introduces filtrations, martingales, optional sampling/optional stopping and Brownian motion. This is entirely too ambitious a reading list to squeeze into the 40 or so pages allocated for all of this, in the opinion of this reviewer. The author places some prerequisite material in the appendix chapter.

Chapter 5 is a much more down-to-earth treatment of genuine applications of Markov chains. Birth/Death processes in biology, queuing networks in information theory, inventory management in operations research, and Markov decision processes are introduced via a series of very nice toy examples. This chapter wraps up with a nice discussion of simulation and the method of Markov chain Monte Carlo.

If the next edition of this book removes chapter 4 and replaces it with treatment of an actual real-world problem (or two) using genuine data sets, this reviewer would be happy to rate that edition 5 stars.

11 of 12 people found the following review helpful:

3.0 out of 5 stars Unique but poorly written, June 6, 2009

By 

Diff D (York, PA United States) - See all my reviews

This review is from: Markov Chains (Cambridge Series in Statistical and Probabilistic Mathematics) (Paperback)

This is a unique book that bridges the gap between undergraduate and graduate treatment but only in the first three chapters. It does require a number of preparatory courses, from multivariate calculus, linear algebra, differential equations to solid understanding of at least undergraduate level of probability, preferable something like Williams' "Probability with Martingales" which Norris seems to refer to. In the United States this is a preparation that is available only in elite schools as many do not even require differential equations for mathematics major while probability is indeed a rare feat to see in mostly pathetic US undergraduate math curricula.

Unfortunately the book is not well written and that is the main reason why it is not more popular than it is. The text beyond first three chapters is largely useless and hard to sort out, with very little care about readability.

There are moments in the text when the author assumes his reader is quite telepatic as some proofs are rather sketchy, and some are even erroneous. Given the first edition that is forgivable although fairly annoying on occasion. The book also contains quite a few misprints adding to confusion. In its scope, the Chapter 1 on discrete Markov Chains is charming, rigorous, and accessible. I have not seen elementary Markov Chains treatment with such a solid level of rigor. Intuition is paired with precise proofs. Chapter 2 on the other hand is somewhat strange and too lax but with important treatment of special cases of Markov Processes to motivate the theory with the main result of definition of continuous Markov Chain. Prior to that the author has chosen "holding times - jump process" presentation which is intuitivelly easier to understand although the definition itself is more intricate. Unfortunately, the author failed to prove the equivalence of the definition with the standard "transition probabilities" definition since the only place he proves "Strong Markov Property" is in the appendix where he made a whole sorry mess with erroneous proofs of lemmas leading to the theorem. The chapter 2 is supposed to strengthen the basic understanding and, in my view, it does that well. Proofs are sometimes sketchy and require considerable work to decipher through. For example Theorem 2.8.6 is appealing to Lemma 2.8.5 while it actually uses different fact that is never proven. It looks like Norris is not aware of that in his sloppiness. I've found myself hard to believe in that proof so me and my colleague have proven theorem using a different deduction. Only at the very end of the chapter the author returns to appropriate precision. Continuous treatment of main results in Chapter 3 is somewhat lacking precision as well, plenty of messy proofs there, in particular conditional probabilities are treated too intuitive. For example stopping time sigma algebra is never properly introduced even though it was used in statements of few propositions with reference to stopped variables (in Chapter 2) but without any precision.

I would assume this book is beneficial for the introductory graduate course on stochastic processes and Markov Chains although not more than that. That is for the first 3 Chapters. As short as it is, it is a good alternative for one semester course.

I am not so sure what is the purpose of the 4th Chapter as it is insufficient to be serious enough. It is rather poorly written appealing on intuition and lacking precision. The mortal flaw of the whole chapter is that the author uses terms that are never defined, proofs that are sketchy appealing on intuition, and to top that off the senseless notation is just relentless, the same continues in Chapter 5. The end of the section on Brownian motion is a classic example of a largely useless text that is only readable by the author himself. The whole chapter is probably some of the worst math writing I have seen in probability texts though there is no shortage of rather bad publications. Any reader would be served the best to avoid the whole Chapter 4 as it requires disproportionate time to read it given meager benefits of learning anything from it. Chapter 5 on applications suffers again from casual and sketchy writing with little care about presentation. Even though it presents the read with valuable examples, for example very nice applications in Queuing Theory, it is probably not worth the effort to read. It is full of tough misprints that make it very hard to read, for example upside down fraction in description of Hastings Algorithm. Since it is largely irrelevant for those going into any specialized direction given how sketchy it is, it can be easily discarded until Norris makes a more consistent presentation, perhaps in the second edition.

Make no mistake, this BOOK REQUIRES GRADUATE LEVEL OF MATURITY as far as most of math majors in US go. I do not see any undergraduate beside exceptionally gifted ones to be able to read this text. The reader is expected to fill in many gaps in proofs. Take a look at the theorems 3.5.3 and 3.5.6. Everything is correct there but sloppiness of presentation creates a mess pretty hard to read. In the proof of 3.5.3 there is a reference on Fubini while in fact the proof goes by conditioning expectation. In 3.5.6 bunch of (correct) facts are thrown at the reader with a showel and without any regard for clarity of presentation. One would not expect that from a textbook.

Also on the negative side, measure theoretic aspect is sketchy/ambiguous/insufficient, the short appendix is not a great help for the same reasons including few confusing errors that should be embarrassing to the author. The editors have not done a good job of removing misprints. There are plenty of those, this is already a staple of poor editing in Cambridge Press publications, some are obvious but some are difficult to spot making already sketchy presentation by the author hard to decipher. Case in point - try to find a misprint on the 4th line of page 187.

This book needs to be rewritten and author ought to chose what level of rigor/intuition he wants throughout the whole text instead of changing the approach from chapter to chapter. If he choses to omit important deductions then he should have 0 misprints. Nevertheless this is a charming and serious book though it could have been written significantly better than it is. Given that nowadays we, the readers that is, are putting up with all sorts of rather slopy textbooks this is not so bad given the first edition.

7 of 8 people found the following review helpful:

4.0 out of 5 stars Very useful if you have a good mathematical background!, May 23, 2001

By 

"alxgsaucedo" (Mexico City) - See all my reviews

This review is from: Markov Chains (Cambridge Series in Statistical and Probabilistic Mathematics) (Hardcover)

I found this book very useful for a better comprehension of Markov chains and for understanding the theory, but you need a very good knowledge of mathematics like calculus, matrix, differential equations and statistics in order to keep going in your reading. Don't expect full numerical examples, be ready for demostrations! I recommend it for students interested in stochastic processes or looking a good approach to the basis of Markov processes.