Customer Reviews

Numerical Analysis for Statisticians

5 star:		(2)
4 star:		(0)
3 star:		(0)
2 star:		(0)
1 star:		(1)

 

 

Ron Thisted's book on computing algorithms for statisticians was one of the most useful and clearly written texts on the topic. There have also been a few other good ones. Lange brings to the table a more current book that deals with the key new methods such as resampling, Markov chain Monte Carlo, Fourier series and wavelets,the EM algorithm and extensions of it. He also includes useful but uncommon results for power series, exponentiating matrices and continued fraction expansions.

The usual matrix algebra stuff for linear models is also there. You will also find a chapter on nonlinear equations and a chapter on splines. There are asymptotic expansions in Chapter 4 and Edgeworth expansions in Chapter 17. Almost everything that is important in statistical computing today is included.

This book can be used as for a graduate course in statistical computing and is a valuable reference for any statistical researcher.

The author states in the introduction "My focus on principles of
numerical analysis is intended to equip students to craft their own
software and to understand the advantages and disadvantages of different
numerical methods". Lets look at a few topics to see whether these
lofty goals were achieved.

Least-squares calculations: The chapter on linear regression is nine
pages. The largest section is on the sweep operator (the problems with
the sweep are not mentioned). Solving least squares is thru the normal
equations only (which numerical analysts agree is the least stable of
the "big three" methods for solving least squares problems). There is a
page on woodbury's formula for determinants. Who uses that!? So many
problems in statistics eventually boil down to a least-squares
calculation. This book has almost nothing useful to say about this
problem. How can students "craft their own software" after reading this
book? They simply can't. Look elsewhere.

Eigenvalues: The chapter on eigenvalues is eight pages and covers only
Jacobi's and the Rayleigh quotient, nothing on the QR, nothing on
bidiagonalization. The nine pages would have been better used for
soemthing else.
Bootstrap calculations: I decided to check out section 22.5,
"importance sampling". After a so-so 2-page inroduction we get an
example. Example 22.5.1 uses the "Hormone Patch Data" from Efron and
Tibshirani's Bootstrap book (a wonderful book, by the way). First, the
analysis is botched, the numerator and denominator variables were
interchanged (relative to Efron and Tibshirani). Now, the denominator
has postive probability of being zero, which is not a problem in of
itself. Then there is a graph based on 100,000 bootstrap samples. The
book says: "Clearly, importance sampling converges more quickly".
Figure 22.1 shows that it actually didn't converge at all!. Then do we
really need importance sampling for this problem? The whole exact
bootstrap distribution has 8^8=16.7 million points (at most). It took just one
minute to write and run a program that computed the exact tail
probability. Why the hell do I need 100,000 bootstrap samples to
approximate something I can compute exaclty with less work? What can
students actually learn from this?

I can go on and on and on, but I'll stop here. What is good about this
book? It does occasionally explain nicely the math behind certain
methods, but even then it really doesn't integrate ideas well enough for
a student.

Somehow, I had missed the first edition of this book and thus I started reading it this afternoon with a newcomer's eyes (obviously, I will not comment on the differences with the first edition, sketched by the author in the Preface). Past the initial surprise of discovering it was a mathematics book rather than an algorithmic book, I became engrossed into my reading and could not let it go! Numerical Analysis for Statisticians, by Kenneth Lange, is a wonderful book. It provides most of the necessary background in calculus and some algebra to conduct rigorous numerical analyses of statistical problems. This includes expansions, eigen-analysis, optimisation, integration, approximation theory, and simulation, in less than 600 pages. It may be due to the fact that I was reading the book in my garden, with the background noise of the wind in tree leaves, but I cannot find any solid fact to grumble about! Not even about the MCMC chapters! I simply enjoyed Numerical Analysis for Statisticians from beginning till end.

From the above, it may sound as if Numerical Analysis for Statisticians does not fulfill its purpose and is too much of a mathematical book. Be assured this is not the case: the contents are firmly grounded in calculus (analysis) but the (numerical) algorithms are only one code away. An illustration (among many) is found in Section 8.4: Finding a Single Eigenvalue, where Kenneth Lange shows how the Raleigh quotient algorithm of the previous section can be exploited to this aim, when supplemented with a good initial guess based on Gerschgorin's circle theorem. This is brilliantly executed in two pages and the code is just one keyboard away. The EM algorithm is immersed into a larger MM perspective. Problems are numerous and mostly of high standards, meaning one (including me) has to sit and think about them. References are kept to a minimum, they are mostly (highly recommended) books, plus a few papers primarily exploited in the problem sections.

While I am reacting so enthusiastically to the book (imagine, there is even a full chapter on continued fractions!), it may be that my French math background is biasing my evaluation and that graduate students over the World would find the book too hard. However, I do not think so: the style of Numerical Analysis for Statisticians is very fluid and the rigorous mathematics are mostly at the level of undergraduate calculus. The more advanced topics like wavelets, Fourier transforms and Hilbert spaces are very well-introduced and do not require prerequisites in complex calculus or functional analysis. (Although I take no joy in this, even measure theory does not appear to be a prerequisite!) On the other hand, there is a prerequisite for a good background in statistics. This book will clearly involve a lot of work from the reader, but the respect shown by Kenneth Lange to those readers will sufficiently motivate them to keep them going till assimilation of those essential notions. Numerical Analysis for Statisticians is also recommended for more senior researchers and not only for building one or two courses on the bases of statistical computing. It contains most of the math bases that we need, even if we do not know we need them! Truly an essential book.