51 of 55 people found the following review helpful
on April 12, 2011
This review is written from the perspective of a programmer who has sometimes had the chance to choose, hire, and work with algorithms and the mathematician/statisticians that love them in order to get things done for startup companies. I don't know if this review will be as helpful to professional mathematicians, statisticians, or computer scientists.
The good news is, this is pretty much the most important book you are going to read in the space. It will tie everything together for you in a way that I haven't seen any other book attempt. The bad news is you're going to have to work for it. If you just need to use a tool for a single task this book won't be worth it; think of it as a way to train yourself in the fundamentals of the space, but don't expect a recipe book. Get something in the "using R" series for that.
When it came out in 2001 my sense of machine learning was of a jumbled set of recipes that tended to work in some cases. This book showed me how the statistical concepts of bias, variance, smoothing and complexity cut across both fields of traditional statistics and inference and the machine learning algorithms made possible by cheaper cpus. Chapters 2-5 are worth the price of the book by themselves for their overview of learning, linear methods, and how those methods can be adopted for non-linear basis functions.
The hard parts:
First, don't bother reading this book if you aren't willing to learn at least the basics of linear algebra first. Skim the second and third chapters to get a sense for how rusty
your linear algebra is and then come back when you're ready.
Second, you really really want to use the SQRRR technique with this book. Having that glimpse of where you are going really helps guide you're understanding when you dig in for real.
Third, I wish I had known of R when I first read this; I recommend using it along with some sample data sets to follow along with the text so the concepts become skills not just
abstract relationships to forget. It would probably be worth the extra time, and I wish I had known to do that then.
Fourth, if you are reading this on your own time while making a living, don't expect to finish the book in a month or two.
52 of 58 people found the following review helpful
on April 4, 2009
This is one of the best books in a difficult field to survey and summarize. Like 'Pattern Recognition', 'Statistical Learning' is an umbrella term for a broad range of techniques of varying complexity, rigor and acceptance by practitioners in the field. The audience for such a text ranges from the user requiring a code library to the mathematician seeking proof of every statement. I sit somewhere in the middle, but more towards the mathematical end. I subscribe to the traditional statistician's view of Machine Learning. It is a term invented in order to avoid having to prove theorems and dodge the rigors of 'real' statistics. However, I strongly support such a course of action. There is an immense need for Machine Learning algorithms, whether they have actual properties or not, and an equal need for books to introduce these topics to people like myself who have a strong mathematical background, but have not been exposed to these techniques.
Hastie & Tibshirani has the most post-it's of any book on my shelf. When my company built an custom multivariate statistical library for our targeted product, we largely followed Hastie & Tibshirani's taxonomy. Their overview of support vector machines is excellent, and I found little of value to me in dedicated volumes like Cristianini & Shawe-Taylor that wasn't covered in Hastie & Tibshirani. Hastie & Tibshirani is another book with excellent visual aides. In addition to some great 2-D representations of complex multidimensional spaces, I thought the 'car going up hill' icon was a very useful cue that the level was going up a notch.
Having praised this book, I can't argue with any of the negative reviews. There is no right answer of where to start or what to cover. This book will be too mathematical for some, insufficiently rigorous for others, but was just right for me. It will offer too much of a hodge-podge of techniques, miss someone's favorite, or offer just the right balance. In the end, it was the best one for me, so if you're like me (someone with a very solid math base, not a mathematician, who appreciates rigor, but isn't married to it, and who is looking to self-start on this topic.) you'll like it.
17 of 17 people found the following review helpful
on May 17, 2014
I have been using The Elements of Statistical Learning for years, so it is finally time to try and review it.
The Elements of Statistical Learning is a comprehensive mathematical treatment of machine learning from a statistical perspective. This means you get good derivations of popular methods such as support vector machines, random forests, and graphical models; but each is developed only after the appropriate (and wrongly considered less sexy) statistical framework has already been derived (linear models, kernel smoothing, ensembles, and so on).
In addition to having excellent and correct mathematical derivations of important algorithms The Elements of Statistical Learning is fairly unique in that it actually uses the math to accomplish big things. My favorite examples come from Chapter 3 "Linear Methods for Regression." The standard treatments of these methods depend heavily on respectful memorization of regurgitation of original iterative procedure definitions of the various regression methods. In such a standard formulation two regression methods are different if they have superficially different steps or if different citation/priority histories. The Elements of Statistical Learning instead derives the stopping conditions of each method and considers methods the same if they generate the same solution (regardless of how they claim they do it) and compares consequences and results of different methods. This hard use of isomorphism allows amazing results such as Figure 3.15 (which shows how Least Angle Regression differs from Lasso regression, not just in algorithm description or history: but by picking different models from the same data) and section 3.5.2 (which can separate Partial Least Squares' design CLAIM of fixing the x-dominance found in principle components analysis from how effective it actually is as fixing such problems).
The biggest issue is who is the book for? This is a mathy book emphasizing deep understanding over mere implementation. Unlike some lesser machine learning books the math is not there for appearances or mere intimidating typesetting: it is there to allow the authors to organize many methods into a smaller number of consistent themes. So I would say the book is for researchers and machine algorithm developers. If you have a specific issue that is making inference difficult you may find the solution in this book. This is good for researchers but probably off-putting for tinkers (as this book likely has methods superior to their current favorite new idea). The interested student will also benefit from this book, the derivations are done well so you learn a lot by working through them.
Finally- don't buy the kindle version, but the print book. This book is satisfying deep reading and you will want the advantages of the printed page (and Amazon's issues in conversion are certainly not the authors' fault).
105 of 132 people found the following review helpful
on February 16, 2010
I have three texts in machine learning (Duda et. al, Bishop, and this one), and I can unequivocally say that, in my judgement, if you're looking to learn the key concepts of machine learning, this one is by far the worst of the three. Quite simply, it reads almost as a research monologue, only with less explanation and far less coherence. There's little/no attempt to demystify concepts to the newcomer, and the exposition is all over the map. There simply isn't a clear, coherent path that the authors set out to go on in writing a given chapter of this text; it's as if they tried to squeeze every bit of information of the most recent results into the chapter, with little regard to what such a decision might do to the overall readability of the text and the newcomer's understanding. To people who might disagree with me on this point, I'd recommend reading a chapter in Bishop's text and comparing it to similar content in this one, and I think you'll at least better appreciate my viewpoint, if not agree with it.
So you might be wondering, why do I even own the text given my opinion? Well, two reasons: (1) it cost 25 dollars through Springer and a contract they have with my university (definitely look into this before buying on Amazon!), and (2) if you actually already know the concepts, it is quite useful as a summary of what's out there. So to those who understand the basics of machine learning, and also have exposure to greedy algorithms, convex optimization, wavelets, and some other often-utilized methods in the text, this makes for a pretty good reference.
The authors are definitely very well-known researchers in the field, who in particular have written some good papers on a variety of machine learning topics (l1-norm penalized regression, analysis of boosting, to name just two), and thus this book naturally will attract some buzz. It may be very useful to someone like myself who is already familiar with much of what's in the book, or someone who is an expert in the field and just uses it as a quick reference. As a pedagogical tool, however, I think it's pretty much a disaster, and feel compelled to write this as to prevent the typical buyer -- who undoubtedly is buying it to learn and not to use as a reference -- from wasting a lot of money on the wrong text.
15 of 17 people found the following review helpful
on March 22, 2009
This is a quite interesting, and extremely useful book, but it is wearing to read in large chunks. The problem, if you want to call it that, is that it is essentially a 700 page catalogue of clever hacks in statistical learning. From a technical point of view it is well-ehough structured, but there is not the slightest trace of an overarching philosophy. And if you don't actually have a philosophical perspective in place before you start, the read you face might well be an even harder grind. Be warned.
Some of the reviews here complain that there is too much math. I don't think that is an issue. If you have decent intuitions in geometry, linear algebra, probability and information theory, then you should be able to cruise through and/or browse in a fairly relaxed way. If you don't have those intuitions, then you are attempting to read the wrong book.
There were a couple of things that I expected (things I happen to know a bit about), but that were missing. On the unsupervised learning side, the discussion of Gaussian mixture clustering was, I thought, a bit short and superficial, and did not bring out the combination of theoretical and practical power that the method offers. On the supervised learning side, I was surprised that a book that dedicates so much time to linear regression finds no room for a discussion of Gaussian process regression as far as I could see (the nearest point of approach is the use of Gaussian radial basis functions [oops: having written that, I immediately came across a brief discussion (S5.8.1) of, essentially, GP regression - though with no reference to standard literature]).
10 of 11 people found the following review helpful
on January 22, 2010
This should certainly not be the first statistics book you read, or even the second or third book, but when you are ready for it then you should absolutely read it. But be prepared to read it very slowly and digest each page. Its greatest strength is that it shows how much of modern statistics comes down to a few fundamental issues: bias, variance, model complexity, and the curse of dimensionality. There is no free lunch in statistics, methods that claim to avoid these tradeoffs only do so by adding more assumptions about the structure of your data. If your data match the assumptions of such methods, you gain statistical power, but if your data don't match the assumptions then you lose.
By looking closely at the assumptions, the book shows how many contemporary methods that look different are fundamentally similar under the hood.
And in my own work I have adopted their use of open circles for the points in scatterplots. These circles are easier to see than tiny solid dots, but overlapping symbols don't cover each other the way large filled symbols do.
9 of 10 people found the following review helpful
on December 15, 2009
"The Elements of Statistical Learning: Data Mining, Inference and Prediction," 2nd edition by Trevor Hastie, Robert Tibshirani and Jerome Friedman is the classic reference for the recent developments in machine learning statistical methods that have been developed at Stanford and other leading edge universities. Their book covers a broad range of topics and is filled with applications. Much new material has been added since the first edition was published in 2001. Since most of these procedures have been implemented in the open-source program R, this book provides a basic and needed reference for their application. Important estimation procedures discussed include MARS, GAM, Projection Pursuit, Exploratory Projection Pursuit, Random Forest, General Linear Models, Ridge Models and Lasso Models etc. There is an discussion of bagging and boosting and how these techniques can be used. There is an extensive index and the many of the datasets discussed are available from the web page of the book or from other sources on the web. Each chapter has a number of problems that test mastery of the material. I have used material from this book in a number of graduate classes at the University of Illinois in Chicago and have implemented a number of the techniques in my software system B34S. While the 1969 book by Box and Jenkins set the stage for time series analysis using ARIMA and Transfer Function Models, Hastie, Tibshirani and Friedman have produced the classic reference for a wide range of new and important techniques in the area of Machine Learning. For anyone interested in Data Mining this is a must own book.
7 of 7 people found the following review helpful
on August 20, 2014
Good book. Can be downloaded as a PDF for free. No need to buy from Amazon.
5 of 5 people found the following review helpful
on January 30, 2014
This is a graduate level overview of key statistical learning techniques. I bought it as I am currently a student in Hastie's graduate level stats course at Stanford. The book is fairly deep given it's breadth, which is why it's so long. It's a very good book and an amazing reference but I do not recommend it unless you have strong mathematical maturity. If you do not know linear algebra / probability well, you will be lost. If you're like me, you'll also have to do some googling around for some of the stats stuff he talks about (confidence intervals, chi squared distribution...). I personally don't mind this. The coverage of this book is truly amazing. Trevor talks about everything from the lasso and ridge regression to knn, SVM's, ensemble methods, and random forests. It's fun to just flip to a random page and just start reading. If you want to be an expert in this area, you need to have this book. On the other hand if you just want to know basic techniques and then apply them, this book might be overkill and you may be better served by a more elementary text.
5 of 6 people found the following review helpful
on December 12, 2009
The book is excellent if you want to use it as a reference and study machine learning by yourself. It's quite comprehensive and deep in areas in which authors are most familiar & famous (frequentist approach, ensemble techniques, maximum likelihood and its variations, lasso). I would recommend you Bishop's machine learning book as an alternative if you want to gain a deeper understanding of Bayesian techniques--that one is more readable as well. Hastie et al's book is just ok from a didactic point of view. The real world examples are complicated to follow (would prefer simpler synthetic data sets). Some descriptions & explanations are too terse--a price to pay for comprehensiveness in a small volume. Overall, a great effort and useful contribution. You'll most likely need to check out other sources to gain a deeper understanding of some of the topics discussed. See MIT machine learning lecture notes (available online).