Modern Multivariate Statistical Techniques: Regression, Classification, and Manifold Learning (Springer Texts in Statistics) [Hardcover]

Alan J. Izenman (Author)

Book Description

Publication Date: August 28, 2008 | ISBN-10: 0387781889 | ISBN-13: 978-0387781884 | Edition: 1

This is the first book on multivariate analysis to look at large data sets which describes the state of the art in analyzing such data. Material such as database management systems is included that has never appeared in statistics books before.

Editorial Reviews

Review

From the reviews: "This book will be enjoyed by those who wish to understand the current state of multivariate statistical analysis in an age of high-speed computation and large data sets. … persons interested in learning new trends of multivariate methods would find Izenman’s book very helpful. … The full-color graphics is quite impressive - well done! There are numerous real-data examples from many scientific disciplines so that not only statisticians may find this book useful and interesting." (Simo Puntanen, International Statistical Review, Vol. 76 (3), 2008) "The book describes how to manage data for maintaining and querying large databases. … I recommend this book for advanced students in statistics and related profiles as, computer science, artificial intelligence, cognitive sciences, bio-informatics, and the involved different branches of engineering. More than 60 data sets are used for working out as examples. More than 200 exercises are presented in the book." (J. A. Rouen, Revista Investigación Operacional, Vol. 30 (2), 2009) "For the first time in a book on multivariate analysis, nonlinear as well as linear methods are discussed in detail. … Another unique feature of this book is the discussion of database management systems. This book is appropiate for advanced undergraduate students, graduate students, and researchers in statistics, computer science, artificial intelligence, psychology, cognitive sciences, business, medicine, bioinformatics and engineering. … The book presents a carefully-integrated mixture of theory and applications, and of classical and modern multivariate statistical techniques, including Bayesian methods." (T. Postelnicu, Zentralblatt MATH, Vol. 1155, 2009) “This monograph provides a comprehensive account of the development of multivariate statistical analysis powered by the explosion in the capability and speed of computers during the last four decades. It is written by an expert in the field. The book is suitable for very advanced undergraduate students and graduate students in statistics, but can also be used in a host of other areas … where statistics plays a major role. … Any researcher in multivariate statistical analysis should have this book in his personal library.” (Steen Arne Andersson, Mathematical Reviews, Issue 2010 b) “…Exemplifies the transition of statistical science as a scientific discipline focused on testing to one focused on information and knowledge discovery. …Acknowledges in a novel way the link between statistical science and computer science, artificial intelligence, and machine learning theory…This book implements an overhaul for teaching multivariate analysis…” (The American Statistician, February 2010, Vol. 64 No.1) “The author of this well-written, encyclopaedic text of roughly 730 pages highlights data mining using huge data sets and aims to blend ‘classical’ multivariate topics (such as regression, principal components and linear discriminant analysis, clustering, multi-dimensional scaling and correspondence analysis) with more recent advances from the field of computational statistics (such as classification and regression trees, neural networks, support vector machines or topics around committee machines—bagging, boosting and random forests). It is noteworthy that some of the more classical methods are derived as special cases of a common theoretical framework: reduced rank regression, a field to which Professor Izenman already has contributed with his doctoral thesis back in 1972. …Furthermore it is worth noting as well that the first chapter after the introductory overview deals with data, databases and database management—indicating the author’s seriousness about data analysis in the presence of permanently growing magnitudes of data sets to analyse. …Most chapters end with sections on software packages, and all chapters end with bibliographical notes and exercises; the final list of references contains 552 entries. …Personally, I felt the book to be heavy, yet rewarding, reading. It seems to have full potential to become a second standard reference next to Hastie et al. (2009).” (Journal of the Royal Statistical Society) “In Modern Multivariate Statistical Techniques, Alan Izenman attempts to synthesize multivariate methods developed across the various literatures into a comprehensive framework. The goal is to present the current state of the art  in multivariate analysis methods while attempting to place them on a firm statistical basis. …This book would be a fantastic reference for researchers interested in learning about multivariate and machine learning methods. …The first half of the book would be suitable for an advanced undergraduate or graduate multivariate analysis course. The second half of the book would be a great reference for a machine-learning course. I definitely enjoyed reading the book.”  (Biometrics, Summer 2009, 65, 990–991)

From the Back Cover

Remarkable advances in computation and data storage and the ready availability of huge data sets have been the keys to the growth of the new disciplines of data mining and machine learning, while the enormous success of the Human Genome Project has opened up the field of bioinformatics. These exciting developments, which led to the introduction of many innovative statistical tools for high-dimensional data analysis, are described here in detail. The author takes a broad perspective; for the first time in a book on multivariate analysis, nonlinear methods are discussed in detail as well as linear methods. Techniques covered range from traditional multivariate methods, such as multiple regression, principal components, canonical variates, linear discriminant analysis, factor analysis, clustering, multidimensional scaling, and correspondence analysis, to the newer methods of density estimation, projection pursuit, neural networks, multivariate reduced-rank regression, nonlinear manifold learning, bagging, boosting, random forests, independent component analysis, support vector machines, and classification and regression trees. Another unique feature of this book is the discussion of database management systems. This book is appropriate for advanced undergraduate students, graduate students, and researchers in statistics, computer science, artificial intelligence, psychology, cognitive sciences, business, medicine, bioinformatics, and engineering. Familiarity with multivariable calculus, linear algebra, and probability and statistics is required. The book presents a carefully-integrated mixture of theory and applications, and of classical and modern multivariate statistical techniques, including Bayesian methods. There are over 60 interesting data sets used as examples in the book, over 200 exercises, and many color illustrations and photographs. Alan J. Izenman is Professor of Statistics and Director of the Center for Statistical and Information Science at Temple University. He has also been on the faculties of Tel-Aviv University and Colorado State University, and has held visiting appointments at the University of Chicago, the University of Minnesota, Stanford University, and the University of Edinburgh. He served as Program Director of Statistics and Probability at the National Science Foundation and was Program Chair of the 2007 Interface Symposium on Computer Science and Statistics with conference theme of Systems Biology. He is a Fellow of the American Statistical Association.

Product Details

• Hardcover: 760 pages
• Publisher: Springer; 1 edition (August 28, 2008)
• Language: English
• ISBN-10: 0387781889
• ISBN-13: 978-0387781884
• Product Dimensions: 9.5 x 6.3 x 1.5 inches
• Shipping Weight: 2.6 pounds (View shipping rates and policies)
• Average Customer Review: 4.5 out of 5 stars  See all reviews (6 customer reviews)
• Amazon Best Sellers Rank: #259,490 in Books (See Top 100 in Books)
  • #39 in Books > Computers & Technology > Computer Science > Artificial Intelligence > Machine Learning
  • #96 in Books > Computers & Technology > Databases > Data Mining

More About the Author

Discover books, learn about writers, read author blogs, and more.

Customer Reviews

Most Helpful Customer Reviews

18 of 18 people found the following review helpful:

5.0 out of 5 stars A great step forward in the way we look at multivariate data, February 6, 2009

By 

Robert S. Newman (Grapevine, TX USA) - See all my reviews
(REAL NAME)   

Amazon Verified Purchase

This review is from: Modern Multivariate Statistical Techniques: Regression, Classification, and Manifold Learning (Springer Texts in Statistics) (Hardcover)

This book surprised me. I was expecting a book filled with a discussion of mostly traditional multivariate techniques supplemented by a few chapters of more recent developments. Instead, I found a completely new and refreshing approach to statistics and data exploration that framed the classical regression approach to most issues as a special, limiting case of a broader view of data exploration and analysis.

Sections on random vectors and matrices, nonparametric density estimation, tree methods, ANI, support vector machines, random forests, bagging and boosting, latent variables, manifold learning, and other topics are discussed and explored in adequate depth for an introductory text. The book assumes you know matrix algebra and have had some exposure to probability distributions, and common multivariate methods, but it extends the discussion in areas that are usually only covered in separate advanced texts and research papers.

The book is a little light on Bayesian methods but some compromises had to be made considering the bulk of the range of new material discussed. I especially liked the broad array of examples from genetics, medicine, physics, and other application areas and the nice color graphs where needed. The references to Matlab, R, S-Plus and other standard math packages was much appreciated although I would have liked Mathematica to have been included as well.

Overall, this is a wonderful survey of a wide range of multivariate techniques and methods. I hope it gets incorporated in college grad and undergrad courses.

7 of 7 people found the following review helpful:

5.0 out of 5 stars a truly modern treatment of multivariate analysis, October 12, 2010

By 

Michael R. Chernick "statman31147" (Holland PA) - See all my reviews
(REAL NAME)   

This review is from: Modern Multivariate Statistical Techniques: Regression, Classification, and Manifold Learning (Springer Texts in Statistics) (Hardcover)

Traditional graduate level texts such as Ted Anderson's focus on the multivariate normal distribution and its statistical properties. So out of that we get MANOVA, Hotelling's T square, linear and quadratic discriminant analysis, principal component analysis, Wishart distributions and canonical correlations. As other reviewers have said this book is quite different. You don't see those topics as chapters in this book. In fact most of these topics are avoided. Izenman finds that with large dimensional data sets that come up in practice these classical techniques do not work very well. So he takes a more modern and "nonparametric' approach. Color adds to the attractiveness of the book although often not essential to the graphical description of the data.

The book begins with exploratory data analysis and extends it to the realm of data mining. The ability to do analysis like this on large data sets comes from the amazing advances in computer speed. Several important concepts are introduced in intuitive ways including pattern recognition and machine learning, prediction error, cross-validation and bootstrap, and overfitting of models.

Chapter is again aimed at the practical by emphaiszing data structure and data bases and by introducing data quality issues including data inconsistencies, outlying observations (which becomes more complicated in multivariate analysis as many directions in a multivariate space can be considered extreme), missing data, and common to today's research data containing many variables but only a few observations such as gene expression on microarrays and satellite images.

But great ideas are not always modern. Izenman points to the curse of dimensionality, a concept coined by Richard Belman back in 1961. Chapter 3 on random vectors and matrices is the one place where the multivariate normal theory is explicitly covered.

Chapter 4 is truly nonparametric and covers multivariate density estimation with instructional examples sprinkled throughout the chapter. Chapter 5 deals with multiple regression a very important and common technique that is described in many texts. Izenman starts with some historical perspective going back to work on least squares by Gauss, Laplace and Legendre where the determination of planetary orbits were modeled circa 1800 and the work of Galton on heredity and regression to the mean in the 1880s and 90s. He gets to all the classical work but also discussion prediction error, the bias of the apparent error rate for a model estimate and the use of cross-validation and the bootstrap as ways to remove large biases in estimates. Again teh techniques are demonstrated with real examples. He discusses some reasonable biased regression approaches including ridge regression, principal components regression and partial least squares regression. Some of these techniques are new even to me (an aging statistician). Since practical problems often involve many potential variables of which some may be unimportant or highly correlated with others, practical regression analysis often use variable selection techniques. Izenman explains the methods and the associated controversies with them. He then introduces some modern approaches not seen outside the research literature including regularized regression (Friedman's general penalized least square approach and the Tibshirani's lasso and Brieman's garotte). He also devotes a whole section to least angle regression developed by Efron, Hastie, Johnstone and Tibshirani.

Chapter 6 generalizes to multivariate regression which includes MANOVA and MANCOVA. Chapter 7 deals with linear dimensionality reduction which includes the classical principal component analysis, canonical variables and canonical correlation and generalizations and then moves to the not so commonly treated topic of project pursuit. In several of the chapters including chapter 7 software packages are listed that can be used to implement the techniques described in the chapter.

Chapter 8 introduces the classification problem with the classical approach of linear discriminant analysis which leads to the nonparametric approach in Chapter 9 sometimes called recursive partitioning but because of the fundamental book Classification and Regression Trees by Brieman, Olshen, Friedman and Stone the more common and popular term is tree-based methods. In Chapter 9 Izenman also includes extensions to these methods which include survival trees and Friedman's multivariate adaptive regression splines. Other approaches coming from the disciplines of artificial intelligence and computer science are the subjects of Chapters 10 and 11, neural netowrks and support vector machines respectively.

Chapter 12 covers unsupervised learning through techniques called cluster analysis methods. Chapter 13 covers multidimensional scaling (here color plays a useful role). Chapter 14 is called Committee Machines. This incorporated the great breakthroughs to improving classification algorithms; bootstrap aggregating which Breiman called "bagging" and boosting algorithms of Schapire and Freund in the early 1990s. Also random forests which introducing a randomization component to bagging also due to Brieman is also discussed.

Later chapters include nonlinear dimensionality reduction, exploratory factor analysis and ending up with a multivariate technique called correspondence analysis that got a lot of attention by the french school of statisticians but was largely ignored in the US for many years.

Aside from the many unique and modern topics discussed in this book what really sets it apart is the academic thoroughness from including a large bibliography of over 550 references, with bibliographic notes at the end of each chapter, illustrative and relevant examples expertly placed throughout the chapters, numerous homework exercises starting with Chapter 2 and a list of software tools for implementing the methods where applicable (every chapter from 7 through 17). As a statistician with interest in bootstrap methods I was particularly pleased with the heavy emphasis on the use of the bootstrap where it has been most successful and gratified that in addition to referencing the commonly referenced bootstrap texts Efron and Tibshirani(1993), Davison and Hinkley(1997) and Hall(1992), he also mentions Chernick (1999). Very little was left out on modern methods. The only things I can think of that are not included are the use of influence functions to detect multivariate outliers as described in Gnandesikan's text and the work of Pesarin and his colleagues on multivariate permutation tests.

8 of 10 people found the following review helpful:

5.0 out of 5 stars Good book - For Statistics majors, September 5, 2009

By 

Statistixian (USA) - See all my reviews

This review is from: Modern Multivariate Statistical Techniques: Regression, Classification, and Manifold Learning (Springer Texts in Statistics) (Hardcover)

This is not Johnson and Wichern or TW Anderson - Think Bishop (PRML) or Hastie, Tibshirani and Friedman (EoSL). We used this for a course last year and this is a great book - as opposed to Bishop which treats things form a Com. Sci. perspective or HTF which assumes a much higher level. One warning though - don't be turned off by the multivariate notation (Duh... Look at the title, of course), but once you master the early chapter on matrix theory and analysis, everything else is very readable.