The Elements of Statistical Learning [Hardcover]

T. Hastie (Author), R. Tibshirani (Author), J. H. Friedman (Author)

Book Description

ISBN-10: 0387952845 | ISBN-13: 978-0387952840 | Publication Date: July 30, 2003 | Edition: Corrected

During the past decade there has been an explosion in computation and information technology. With it have come vast amounts of data in a variety of fields such as medicine, biology, finance, and marketing. The challenge of understanding these data has led to the development of new tools in the field of statistics, and spawned new areas such as data mining, machine learning, and bioinformatics. Many of these tools have common underpinnings but are often expressed with different terminology. This book describes the important ideas in these areas in a common conceptual framework. While the approach is statistical, the emphasis is on concepts rather than mathematics. Many examples are given, with a liberal use of color graphics. It should be a valuable resource for statisticians and anyone interested in data mining in science or industry. The book's coverage is broad, from supervised learning (prediction) to unsupervised learning. The many topics include neural networks, support vector machines, classification trees and boosting---the first comprehensive treatment of this topic in any book.

This major new edition features many topics not covered in the original, including graphical models, random forests, ensemble methods, least angle regression & path algorithms for the lasso, non-negative matrix factorization, and spectral clustering. There is also a chapter on methods for ``wide'' data (p bigger than n), including multiple testing and false discovery rates.

Trevor Hastie, Robert Tibshirani, and Jerome Friedman are professors of statistics at Stanford University. They are prominent researchers in this area: Hastie and Tibshirani developed generalized additive models and wrote a popular book of that title. Hastie co-developed much of the statistical modeling software and environment in R/S-PLUS and invented principal curves and surfaces. Tibshirani proposed the lasso and is co-author of the very successful An Introduction to the Bootstrap. Friedman is the co-inventor of many data-mining tools including CART, MARS, projection pursuit and gradient boosting.

Editorial Reviews

Review

From the reviews:

SIAM REVIEW

"The book is very well written and color is used throughout. Color adds a dimension that can be used to help the reader visualize high-dimensional data, and it is also very useful to help the eye see patterns and clusters more easily. This makes color effective in the book and not just a pleasing gimmick. This is the first book of its kind to treat data mining from a statistical perspective that is comprehensive and up-to-date on the statistical methods…I found the book to be both innovative and fresh. It provides an important contribution to data mining and statistical pattern recognition. It should become a classic…It is especially good for statisticians interested in high-dimensional and high-volume data such as can be found in telephone records, satellite images, and genetic microarrays. It can be used for an advanced special topics course in statistics for graduate students."

TECHNOMETRICS

"[This] is a vast and complex book. Generally, it concentrates on explaining why and how the methods work, rather than how to use them. Examples and especially the visualizations are principle features…As a source for the methods of statistical learning…it will probably be a long time before there is a competitor to this book."

SHORT BOOK REVIEWS

"This book describes modern tools for data analysis. With the exception of the last chapter, it is concerned with "supervised" methods - those methods in which a sample of cases is available, including values of an outcome variable, and on which one can build a model allowing one to predict the value of the outcome variable for new cases. The authors are amongst the leaders in this area, having developed many of the modern tools. Such methods have seen extraordinary development in recent decades, primarily because of progress in computer technology, but also because of the huge range of applications. Furthermore, the practical development of these modeling and inferential tools has resulted in a deeper theoretical understanding of the modeling process... The book includes many special cases and examples, which give insights into the ideas and methods. It explains very clearly the relationships between the methods, and covers both standard statistical staples, such as linear and logistic regression, as well as modern tools. It is not overburdened with unnecessary mathematics but uses only what is necessary for the practical application of the methods...The book has been beautifully produced. It was a pleasure to read. I strongly recommend it."

MATHEMATICAL REVIEWS

"The book provides a comprehensive and up-to-date introduction to the field of statistical pattern recognition, now commonly referred to as statistical learning…Browsing through the book, one is immediately attracted to the skillful use of color plots to stress the different behaviors of algorithms on real-world datasets. This tells a lot about the books style: intuition about a learning technique is built by looking at the behavior on the data, then the statistical analysis follows. However, even in its most technical parts, the presentation flows very smoothly, avoiding the definition-theorem-proof writing style…this is a very complete and up-to-date work covering all the most important learning techniques, which are presented in a rigorous but accessible statistical framework."

JOURNAL OF CLASSIFICATION, JUNE 2004

"This is a great book. All three authors have track records for clear exposition and are famously gifted for finding intuitive explanations that illuminate technical results…In particular, we admire the book for its:

-outstanding use of real data examples to motivate problems and methods;

-unified treatment of flexible inferential procedures in terms of maximization of an objective function subject to a complexity penalty;

-lucid explanation of the amazing performance of the AdaBoost algorithm in improving classification accuracy for almost any rule;

-clear account of support vector machines in terms of traditional statistical paradigms;

-regular introduction of some new insight, such as describing self-organizing maps as constrained k-means clustering.

…No modern statistician or computer scientist should be without this book."

JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION, JUNE 2004

"In the words of the authors, the goal of this book was to ‘bring together many of the important new ideas in learning, and explain them in a statistical framework.’ The authors have been quite successful in achieving this objective, and their work is a welcome addition to the statistics and learning literatures…A strength of the book is the attempt to organize a plethora of methods into a coherent whole. The relationships among the methods are emphasized. I know of no other book that covers so much ground."

"The book by Hastie et al. covers a wider number of topics such as supervised learning based on linear models, nearest neighbor methods, decision theory, function approximations, roughness, and kernels … . The charts and graphs are done in color to distinguish different patterns. The book has both challenging and easier exercise problems in each chapter. The book is suitable for a graduate level data mining course. I learned a lot from this well written book and recommend it highly." (Ramalingam Shanmugam, Journal of Statistical Computation & Simulation, Vol. 74 (4), 2004)

"One of the great features of the book is that it really contains more or less all modern methods for statistical learning, so it gives the reader a very good overview of this important field. … The author worked very hard on presentation of the material, in particular they illustrated the material by many colored graphics. … I think this book is valuable for anyone interested in statistical learning and its application, and I am happy to have it on my desk." (Michael Kohler, Metrika, February, 2003)

"For anyone who … wants to learn the new terminology and to understand what the ‘competition’ is doing, this is the book to buy. … the thinking is still very much statistical. This makes the book very easy to digest and pleasant to read for people with a statistical background. The many superb graphs add to this pleasure. … The book is important because it shows that interaction between statistics and machine learning can be profitable for both fields." (Hans C. van Houwelingen, Statistics in Medicine, Vol. 23, 2004)

"This is a great book. … We have taught a large graduate course (for statisticians and computer scientists) in data mining from this book. In developing this course we spoke to many other faculty members at a range of institutions, and we found no one who did not enjoy reading and teaching from this text. … there is no other book worth considering for such a course. … The book has beautiful graphics … . No modern statistician or computer scientist should be without this book." (David Banks and Feng Liang, Journal of Classification, Vol. 21 (1), 2004)

"The book provides a long-sought link between Statistics and Data Mining. It provides an excellent reference for researchers in information sciences … . Written by well-known specialists in applied statistics, the book provides a good practical orientation, with related theoretical issues coming out quite clearly. … this is the first book to address different aspects of data mining, inference and prediction in a coherently interdisciplinary context. … this book will always be remembered for laying the foundation of that scientific pyramid." (Kassim Said Mwitondi, Journal of Applied Statistics, Vol. 30 (1), 2003)

"The emphasis is on concepts rather than mathematics, and several examples are given as illustration. … This book is designed for researchers and students in a broad variety of fields such as statistics, artificial intelligence, engineering and finance. It should be a valuable resource for those who are interested in data mining in science or industry. I believe that it will be a very useful addition to any scholarly library." (Theofanis Sapatinas, Journal of the Royal Statistical Society Series A: Statistics in Society, Vol. 157 (1), 2004)

"A mere glance at the table of contents gives an idea of the breadth and depth of coverage of this remarkable book. … The style of this beautifully presented book is friendly and intuitive, and at the same time clear and rigorous. All the techniques dealt with are presented and compared through examples with real and simulated data. … The book can be used as a basis for courses of different levels, from the purely practical to the thoroughly theoretical. … a wonderful book!" (Ricardo Maronna, Statistical Papers, Vol. 44 (3), 2003)

"The book covers two topics: 12 chapters discuss statistical methods of supervised learning, the final chapter is on unsupervised learning. … The getup of the book is outstanding … . The book is an excellent and comprehensive treatment of the topics for which the authors are well known … . The book may well serve as a textbook for an advanced course; the illustrating examples and the discussion of computational aspects make the book useful for those who want to apply the methods." (Peter Hackl, Statistical Papers, Vol. 43 (3), 2002)

"This book describes modern tools for data analysis. … The book includes many special cases and examples, which give insights into the ideas and methods. It explains very clearly the relationship between the methods, and covers both standard statistical staples, such as linear and logistic regression, as well as modern tools. It is not overburdened with unnecessary mathematics but uses only what is necessary for the practical application … . The book has been beautifully produced. It was a pleasure to read. I strongly recommend it." (D. J. Hand, Short Book Reviews, Vol. 22 (1), 2002)

"This book is designed for researchers and students in the fields of statistics, artificial, intelligence, engineering, finance, and oth...

From the Back Cover

During the past decade there has been an explosion in computation and information technology. With it have come vast amounts of data in a variety of fields such as medicine, biology, finance, and marketing. The challenge of understanding these data has led to the development of new tools in the field of statistics, and spawned new areas such as data mining, machine learning, and bioinformatics. Many of these tools have common underpinnings but are often expressed with different terminology. This book describes the important ideas in these areas in a common conceptual framework. While the approach is statistical, the emphasis is on concepts rather than mathematics. Many examples are given, with a liberal use of color graphics. It is a valuable resource for statisticians and anyone interested in data mining in science or industry. The book's coverage is broad, from supervised learning (prediction) to unsupervised learning. The many topics include neural networks, support vector machines, classification trees and boosting---the first comprehensive treatment of this topic in any book. This major new edition features many topics not covered in the original, including graphical models, random forests, ensemble methods, least angle regression and path algorithms for the lasso, non-negative matrix factorization, and spectral clustering. There is also a chapter on methods for ``wide'' data (p bigger than n), including multiple testing and false discovery rates. Trevor Hastie, Robert Tibshirani, and Jerome Friedman are professors of statistics at Stanford University. They are prominent researchers in this area: Hastie and Tibshirani developed generalized additive models and wrote a popular book of that title. Hastie co-developed much of the statistical modeling software and environment in R/S-PLUS and invented principal curves and surfaces. Tibshirani proposed the lasso and is co-author of the very successful An Introduction to the Bootstrap. Friedman is the co-inventor of many data-mining tools including CART, MARS, projection pursuit and gradient boosting. --This text refers to an alternate Hardcover edition.

See all Editorial Reviews

Product Details

• Hardcover: 552 pages
• Publisher: Springer; Corrected edition (July 30, 2003)
• Language: English
• ISBN-10: 0387952845
• ISBN-13: 978-0387952840
• Product Dimensions: 9.4 x 6.1 x 1.2 inches
• Shipping Weight: 2.3 pounds
• Average Customer Review: 4.0 out of 5 stars  See all reviews (45 customer reviews)
• Amazon Best Sellers Rank: #623,318 in Books (See Top 100 in Books)
  • #72 in Books > Computers & Technology > Computer Science > Artificial Intelligence > Machine Learning
  • #86 in Books > Computers & Technology > Computer Science > Artificial Intelligence > Neural Networks
  • #93 in Books > Professional & Technical > Professional Science > Biological Sciences > Bioinformatics

Customer Reviews

Most Helpful Customer Reviews

66 of 67 people found the following review helpful:

4.0 out of 5 stars The Elements of Statistical Learning, November 12, 2004

By 

R. Krause (Tempe, AZ) - See all my reviews
(REAL NAME)   

This review is from: The Elements of Statistical Learning (Hardcover)

The book is written by some of the biggest names currently in the field, and thus is written at a certain level, this isn't a fault of the book or the authers, but rather it was written for a specific audience. However I did find it odd when they would occassionally explain basic readily known notation, but later on assume the reader is familiar with what I would regard as advanced notation, or leave out quite a few steps in their mathematics assuming the reader understands what they did. This book covers a wide range of techniques ranging from the more traditional to the current, and for each topic presents an overview of the technique and provides adequate references for further exploration.

The reader should have a good underlying understanding of linear algebra, statistics and probability theory and also be familiar with the techniques presented here. This book was used in a graduate engineering data mining class, and most of us struggled greatly with the book. This book probably would have been more appropriate if this was a book to augment another text, or if this had not been the first time we had seen topics such as those presented, this being the book to explain neural networks, support vector machines and whatnot when you've never seen them before makes for a very bewildering experience, but once you find a few journal articles the techniques actually are fairly easy to understand.

The book does not explain how to implement using software any of the techniques, this is a topic left up to other books, such as Modern Applied Statistics with S by Ripley and Venerables, and only in their discussion about apriori for association rules did I see that they state a software package. It would have been nice if they would have given some insight into how they created some of the great graphics that punctuate the book, perhaps as additional material on the website.

A book that is more down to earth for engineers, albeit different in scope, would be Duda and Hart's Pattern Classification, which I believe are electrical engineers and written more from an engineering standpoint. In addition the Duda and Hard book gives a lot of applications-based problems and has an associated MATLAB handbook to walk readers through building many types of learners, while this book the end-of-chapter excercises are almost exclusively proofs and theoretical excercises. Not a fault of the book, but rather just a difference and depends on what the reader wants to get out of it.

Ultimately, even though it did prove to be a rather confusing book, I have learned a lot from it and will continue to go through it to learn even more from it as it does tend to become more lucid the more I go through it.

100 of 106 people found the following review helpful:

5.0 out of 5 stars Useful book on data mining, February 6, 2002

By 

frank lindemann - See all my reviews
(REAL NAME)   

Amazon Verified Purchase

This review is from: The Elements of Statistical Learning (Hardcover)

I use data mining tools in my financial engineering and financial modeling work and I have found this book to be very useful. This book provides two crucial types of information. First, it provides enough theory to allow a potential user to understand the essential insights that motivate specific techniques and to evaluate the situations in which those technique are appropriate. Second, the book gives the exact algorithms to implement the various techniques.
While no book I have seen covers every data mining methodology available, this one has the strongest coverage I have seen in additive models, non-linear regression, and CART/MART (regression/classification trees). It also has very strong coverage in many other areas. I highly recommend it.

53 of 56 people found the following review helpful:

4.0 out of 5 stars The Elements of Statistical Learning, December 18, 2001

By A Customer

This review is from: The Elements of Statistical Learning (Hardcover)

The book by Hastie, Tibshirani and Friedman is a welcome
addition to the quickly growing area of machine learning
and data mining. This is a well written book, laid out
nicely with excellent examples by 3 well established
researchers in the field. It will be helpful to those
who are interested in learning about this field, as well
as experts who want to know more

My only complaint is that although the authors do
make an honest attempt to clearly highlight methods
that are based on their own research,
often this distinction becomes cloudy and the reader
is left with the impression that the methods
advocated are often the best and represent
the standard in the industry. In fact many of
their ideas are only heuristic and it is more than
conceivable that these will eventually be superseeded
with better methods.

A good book, which gets you up to speed in the literature
but it will only be relevant for a few years.