Customer Reviews

Ensemble Methods in Data Mining: Improving Accuracy Through Combining Predictions (Synthesis Lectures on Data Min)

5 star:		(5)
4 star:		(1)
3 star:		(0)
2 star:		(0)
1 star:		(0)

 

 

During my 10 plus years of modeling experience, I have always paid most of my attention on variable selection, predictive power, effectiveness and efficiency of a single model form such as logistic model, ordinary regression, tree, etc. From time to time, I also segment my sample space into pieces and then apply different modeling techniques. Never really aware of the concept of 'model selection' or 'model combination'. That classical approach has served me well. But I always suspected that there was a better approach to combine different methods to get better predictions.
Ensemble methods detailed in this books gave me the 'ah ha'. It gave a nicely balanced flavor of easy implementation and difficult concepts. I really enjoyed the book. I was able to finish the book quick and would save it for reference.
If there is anything that I would want to see in more detail, it is the treatment of evaluation of model prediction. It is a bit light on how to tell if the final product is really working. Given that the book is an intro, then it is not really a mis-treatment.
overall, awesome small book.

This book is an accessible introduction to the theory and practice of ensemble methods in machine learning. It is a quick read, has sufficient detail for a novice to begin experimenting, and copious references for those who are interested in digging deeper. The authors also provide a nice discussion of cross-validation, and their section on regularization techniques is much more straightforward, in my opinion, than the equivalent sections in The Elements of Statistical Learning (Elements is a wonderful, necessary book, but a hard read).

The heart of the text is the chapter on Importance Sampling. The authors frame the classic ensemble methods (bagging, boosting, and random forests) as special cases of the Importance Sampling methodology. This not only clarifies the explanations of each approach, but also provides a principled basis for finding improvements to the original algorithms. They have one of the clearest descriptions of AdaBoost that I've ever read.

The penultimate chapter is on "Rule Ensembles": an attempt at a more interpretable ensemble learner. They also discuss measures for variable importance and interaction strength. The last chapter discusses Generalized Degrees of Freedom as an alternative complexity measure; it is probably of more interest to researchers and mathematicians than to practitioners.

Overall, I found the book clear and concise, with good attention to practical details. I appreciated the snippets of R code and the references to relevant R packages. One minor nitpick: this book has also been published digitally, presumably with color figures. Because the print version is grayscale, some of the color-coded graphs are now illegible. Usually the major points of the figure are clear from the context in the text; still, the color to grayscale conversion is something for future authors in this series to keep in mind.

Recommended.

For once, "Product Description" is specific and hype-free. (Apart from the claim regarding importance sampling - dealt with on a single page). This is a concise, to-the-point and accessible introduction to the subject, discussing bagging, random-forest and boosting methods, in classification context. Once these methods are explained, the authors move on to measures of variable importance and model complexity, which may be of less interest to practitioners. R snippets, leveraging rpart and gbm packages, are a plus, but the programming is fairly simple.

PS. Morgan Claypool sell the book's PDF for $20, or $0 for those affiliated with the publisher's institutional subscribers.

This is a really great (short) book in my opinion. It contains the best "need to know" information found in the Elements in Statistical Learning, and other good books on data mining. The included R code is a big bonus. I am enjoying reading it so far, and I highly recommend it. The only thing that frustrates me is that the online version on the publishers website is in color, while the print version is not. This is the only reason I did not give it 5 stars. I saw the online version first, and thought that the print version would be in color as well. I am sadly mistaken. There are many graphics in this book that reference different colors and it just looks really crappy in grayscale. If you are familiar with the Elements of Statistical Learnining, imagine printing that out in grayscale and you will know what I mean.

The definitive reference on ensembles, which are a central, profound aspect of predictive modeling best practices. A solid, core, important reference with great coverage and tutorial value.

"Ensemble Methods in Data Mining" (EMIDM) is an excellent introduction and reference to ensemble methods, from several perspectives:

- good mathematical descriptions of the algorithms;
- intuitive explanations of the concepts involved;
- several illustrative examples (including R code); and
- a great structured guide to the vast literature on ensemble methods

The only other book I know that covers ensemble methods is the well-known "The Elements of Statistical Learning" (TEOSL), which can be quite a dense read at times. For example, TEOSL covers Importance Sampling Learning Ensembles (ISLE) and Rule Ensembles (RE) in a couple of pages each, whereas EMIDM dedicates a chapter to each (personally, I had overlooked the significance of those 2 methods until I read the more developed narrative in EMIDM).

In any event, you should own a copy of TEOSL (which can be freely downloaded off the authors website), but if you want to master ensemble methods (currently one of the hottest areas in data mining and machine learning) and confidently be able to apply them in practice, then EMIDM is a wise investment.