Customer Reviews

Machine Learning: An Algorithmic Perspective (Chapman & Hall/Crc Machine Learning & Pattern Recognition)

5 star:		(11)
4 star:		(1)
3 star:		(1)
2 star:		(1)
1 star:		(2)

 

 

This is an good book on machine learning for students at the advanced
undergraduate or Masters level, or for self study, particularly if
some of the background math (eigenvectors, probability theory, etc)
is not already second nature.

Although I am now familiar with much of the math in this area and consider
myself to have intermediate knowledge of machine learning, I can still recall
my first attempts to learn some mathematical topics. At that time my approach
was to implement the ideas as computer programs and plot the results. This
book takes exactly that approach, with each topic being presented both
mathematically and in Python code using the new Numpy and Scipy libraries.
Numpy resembles Matlab and is sufficiently high level that the book code
examples read like pseudocode.

(Another thing I recall when I was first learning was the mistaken
belief that books are free from mistakes. I've since learned to
expect that every first edition is going to have some, and doubly so
for books with math and code examples. However the fact that many of the examples
in this book produce plots is reassuring.)

As mentioned I have only intermediate knowledge of machine learning, and
have no experience with some techniques. I learned regression trees
and ensemble learning from this book -- and then implemented an ensemble
tree classifier that has been quite successful at our company.

Some other strong books are the two Bishop books (Neural Networks for Pattern
Recognition; Pattern Recognition and Machine Learning),
Friedman/Hastie/Tibshirani (Elements of Statistical Learning) and
Duda/Hart/Stork (Pattern Classification). Of these, I think the first Bishop
book is the only other text suitable for a beginner, but it doesn't have the
explanation-by-programming approach and is also now a bit dated (Marsland
includes modern topics such as manifold learning, ensemble learning, and a bit
of graphical models). Friedman et al. is a good collection of algorithms,
including ones that are not presented in Marsland; it is a bit dry however.
The new Bishop is probably the deepest and best current text, but it is
probably most suited for PhD students. Duda et al would be a good book at a
Masters level though its coverage of modern techniques is more limited. Of
course these are just my impressions. Machine learning is a broad subject and
anyone using these algorithms will eventually want to refer to several of these books.
For example, the first Bishop covers the normalized flavor of radial basis
functions (a favorite technique for me), and each of the mentioned books has
their own strengths.

I first saw this book on a colleague's bookshelf; i picked it up and briefly looked through it. The simple diagrams and the relative lack of equations (compared to e.g., Bishop) might suggest to you that it's a 'beginner' text--and by that i mean that the textbook is only an introduction to ML and doesn't teach you enough so that you can begin writing ML code to solve real classification/regression problems. That's what i though at first, and i was wrong. This is an introductory text, but only in the sense that it's accessible to more or less anyone, but this book's explanation/theory and the practical examples (in python) are brilliantly integrated--the explanation (often summarizing two or three pages of terse equations found in other textbooks, in a single paragraph) helped me grok the code, and the code reinforced the theory behind the algorithm.

I don't think there's another ML book like this--it's aimed right at the blind spot framed by applied math reference-type books such as Bishop on one end, and books like 'Programming Collective Intelligence' which are dense with working ML code, but light on theory.

I also like this book because the code is written in NumPy, rather than in the Python standard library code. NumPy is what you would use 'in the real world' to code an ML algorithm, and if you understand the matrix-driven syntax, then the code is far more concise (e.g., no triply nested recursive loops) than the same algorithms coded using just the Python standard library.

In sum, an excellent book.

There are two or three things that I really like about this book.

First, so many books of this type seem to leave off the first 20 or so pages that should tell you what it is that they are trying to do. Instead of assuming that you know what Machine Learning is all about, this book has an initial chapter that explains in simple terms what we are trying to do here.

Second, instead us using some kind of psuedocode, the examples are written in a standard language, Python. Python is a free language in the open source community so students can get/use it without incurring the costs associated with some other languages. It is also intended to be very readable which makes the demonstration programs easier to understand. There is also a chapter on programming in Python.

Machine learning usually is put into the computer science department in universities, and as a result is usually taught to computer science students. In fact, machine learning also requires more mathematical background and more engineering background than most computer science students have. The approach used in this book is to discuss algorithms used in machine learning, but to do so by stressing how and why they work.

The author says that the book is suitable for undergraduate use. Yes, it is, but for the rather advanced undergraduate or even early graduate level student

This book is a superb introduction to machine learning. It is ideally suited for under-graduates, but anyone will find it immensely useful to get their feet wet with machine learning!

The author has excellent style of communication and his examples are simple and clear. The Math is explained in painstaking detail (even matrix multiplication). Advanced readers can easily skip these explanations, but novices will find them really useful.

Added to that, the author has taken the extra effort to code up everything in python and has provided the code to play around with! (This book could also be a fun introduction to python, which is a lovely language to learn.) Many times you read some algorithm and try to implement the pseudo-code, only to find that it takes hours for you to make it work. Something as simple as linear regression could turn out to be a pain, if you have incorrect step sizes! Many books simply say "use a small value" for the step size, and you are wondering whether 2 is small or 0.001 is small! But this book goes one step further, and says, what value of step size to use for the example, so you first get something to work before you experiment with it.

If you are teaching an under-grad CS course/introductory CS grad course on machine learning, then this is the book you need to use. Your students will love you :-).

I've only had this book for a few days and my judgement is not definitive, but it looks very promising indeed. The author has an unassuming casual and helpful style.
It's certainly a lot more detailed that Tom Mitchell's book, covers a lot of material you wont find in Norvig and Russel and is much more appealing than Bishop's latest book.
I thought you might be interested in the table of contents since there is no "see inside" for this book. Here it is by OCR:

Prologue xv

1 Introduction 1
1.1 If Data Had Mass, the Earth Would Be a Black Hole 2
1.2 Learning 4
1.2.1 Machine Learning 5
1.3 Types of Machine Learning 6
1.4 Supervised Learning 7
1.4.1 Regression 8
1.4.2 Classification 9
1.5 The Brain and the Neuron 11
1.5.1 Hebb's Rule 12
1.5.2 McCulloch and Pitts Neurons 13
1.5.3 Limitations of the McCulloch and Pitt Neuronal Model 15
Further Reading 16

2 Linear Discriminants 17
2.1 Preliminaries 18
2.2 The Perceptron 19
2.2.1 The Learning Rate j 21
2.2.2 The Bias Input 22
2.2.3 The Perceptron Learning Algorithm 23
2.2.4 An Example of Perceptron Learning 24
2.2.5 Implementation 26
2.2.6 Testing the Network 31
2.3 Linear Separability 32
2.3.1 The Exclusive Or (XOR) Function 34
2.3.2 A Useful Insight 36
2.3.3 Another Example: The Pima Indian Dataset 37
2.4 Linear Regression 41
2.4.1 Linear Regression Examples 43
Further Reading 44
Practice Questions 45

3 The Multi-Layer Perceptron 47
3.1 Going Forwards 49
3.1.1 Biases 50
3.2 Going Backwards: Back-Propagation of Error 50
3.2.1 The Multi-Layer Perceptron Algorithm 54
3.2.2 Initialising the Weights 57
3.2.3 Different Output Activation Functions 58
3.2.4 Sequential and Batch Training 59
3.2.5 Local Minima 60
3.2.6 Picking Up Momentum 61
3.2.7 Other Improvements 62
3.3 The Multi-Layer Perceptron in Practice 63
3.3.1 Data Preparation 63
3.3.2 Amount of Training Data 63
3.3.3 Number of Hidden Layers 64
3.3.4 Generalisation and Overfitting 66
3.3.5 Training, Testing, and Validation 66
3.3.6 When to Stop Learning 68
3.3.7 Computing and Evaluating the Results 69
3.4 Examples of Using the MLP 70
3.4.1 A Regression Problem 70
3.4.2 Classification with the MLP 74
3.4.3 A Classification Example 75
3.4.4 Time-Series Prediction 77
3.4.5 Data Compression: The Auto-Associative Network . 80
3.5 Overview 83
3.6 Deriving Back-Propagation 84
3.6.1 The Network Output and the Error 84
3.6.2 The Error of the Network 85
3.6.3 A Suitable Activation Function 87
3.6.4 Back-Propagation of Error 88
Further Reading 90
Practice Questions 91
4 Radial Basis Functions and Splines 95
4.1 Concepts 95
4.1.1 Weight Space 95
4.1.2 Receptive Fields 97
4.2 The Radial Basis Function (RBF) Network 100
4.2.1 Training the RBF Network 103
4.3 The Curse of Dimensionality 106
4.4 Interpolation and Basis Functions 108
4.4.1 Bases and Basis Expansion 108
4.4.2 The Cubic Spline 112
4.4.3 Fitting the Spline to the Data 112

4.4.4 Smoothing Splines 113
4.4.5 Higher Dimensions 114
4.4.6 -Beyond the Bounds 116
Further Reading 116
Practice Questions 117
5 Support Vector Machines 119
5.1 Optimal Separation 120
5.2 Kernels 125
5.2.1 Example: XOR 128
5.2.2 Extensions to the Support Vector Machine 128
Further Reading 130
Practice Questions 131
6 Learning with Trees 133
6.1 Using Decision Trees 133
6.2 Constructing Decision Trees 134
6.2.1 Quick Aside: Entropy in Information Theory 135
6.2.2 ID3 136
6.2.3 Implementing Trees and Graphs in Python 139
6.2.4 Implementation of the Decision Tree 140
6.2.5 Dealing with Continuous Variables 143
6.2.6 Computational Complexity 143
6.3 Classification and Regression Trees (CART) 145
6.3.1 Gini Impurity 146
6.3.2 Regression in Trees 147
6.4 Classification Example 147
Further Reading 150
Practice Questions 151
7 Decision by Committee: Ensemble Learning 153
7.1 Boosting 154
7.1.1 AdaBoost 155
7.1.2 Stumping 160
7.2 Bagging 160
7.2.1 Subagging 162
7.3 Different Ways to Combine Classifiers 162
Further Reading 164
Practice Questions 165
8 Probability and Learning 167
8.1 Turning Data into Probabilities 167
8.1.1 Minimising Risk 171
8.1.2 The Naïve Bayes' Classifier 171
8.2 Some Basic Statistics 173

8.2.1 Averages 173
8.2.2 Variance and Covariance 174
8.2.3 The Gaussian 176
8.2.4 The Bias-Variance Tradeoff 177
8.3 Gaussian Mixture Models 178
8.3.1 The Expectation-Maximisation (EM) Algorithm . 179
8.4 Nearest Neighbour Methods 183
8.4.1 Nearest Neighbour Smoothing 185
8.4.2 Efficient Distance Computations: the KD-Tree . . 186
8.4.3 Distance Measures 190
Further Reading 192
Practice Questions 193
9 Unsupervised Learning 195
9.1 The k-Means Algorithm 196
9.1.1 Dealing with Noise 200
9.1.2 The k-Means Neural Network 200
9.1.3 Normalisation 202
9.1.4 A Better Weight Update Rule 203
9.1.5 Example: The Iris Dataset Again 204
9.1.6 Using Competitive Learning for Clustering 205
9.2 Vector Quantisation 206
9.3 The Self-Organising Feature Map 207
9.3.1 The SOM Algorithm 210
9.3.2 Neighbourhood Connections 211
9.3.3 Self-Organisation 214
9.3.4 Network Dimensionality and Boundary Conditions 214
9.3.5 Examples of Using the SOM 215
Further Reading 218
Practice Questions 220
10 Dimensionality Reduction 221
10.1 Linear Discriminant Analysis (LDA) 223
10.2 Principal Components Analysis (PCA) 226
10.2.1 Relation with the Multi-Layer Perceptron 231
10.2.2 Kernel PCA 232
10.3 Factor Analysis 234
10.4 Independent Components Analysis (ICA) 237
10.5 Locally Linear Embedding 239
10.6 Isomap 242
10.6.1 Multi-Dimensional Scaling (MDS) 242
Further Reading 245
Practice Questions 246

11 Optimisation and Search 247
11.1 Going Downhill 248
11.2 Least-Squares Optimisation 251
11.2.1 Taylor Expansion 251
11.2.2 The Levenberg-Marquardt Algorithm 252
11.3 Conjugate Gradients 257
11.3.1 Conjugate Gradients Example 260
11.4 Search: Three Basic Approaches 261
11.4.1 Exhaustive Search 261
11.4.2 Greedy Search 262
11.4.3 Hill Climbing 262
11.5 Exploitation and Exploration 264
11.6 Simulated Annealing 265
11.6.1 Comparison 266
Further Reading 267
Practice Questions 267
12 Evolutionary Learning 269
12.1 The Genetic Algorithm (GA) 270
12.1.1 String Representation 271
12.1.2 Evaluating Fitness 272
12.1.3 Population 273
12.1.4 Generating Offspring: Parent Selection 273
12.2 Generating Offspring: Genetic Operators 275
12.2.1 Crossover 275
12.2.2 Mutation 277
12.2.3 Elitism, Tournaments, and Niching 277
12.3 Using Genetic Algorithms 279
12.3.1 Map Colouring 279
12.3.2 Punctuated Equilibrium 281
12.3.3 Example: The Knapsack Problem 281
12.3.4 Example: The Four Peaks Problem 282
12.3.5 Limitations of the GA 284
12.3.6 Training Neural Networks with Genetic Algorithms . 285
12.4 Genetic Programming 285
12.5 Combining Sampling with Evolutionary Learning 286
Further Reading 289
Practice Questions 290
13 Reinforcement Learning 293
13.1 Overview 294
13.2 Example: Getting Lost 296
13.2.1 State and Action Spaces 298
13.2.2 Carrots and Sticks: the Reward Function 299
13.2.3 Discounting 300

13.2.4 Action Selection 301
13.2.5 Policy 302
13.3 Markov Decision Processes 302
13.3.1 The Markov Property 302
13.3.2 Probabilities in Markov Decision Processes 303
13.4 Values 305
13.5 Back on Holiday: Using Reinforcement Learning 309
13.6 The Difference between Sarsa and Q-Learning 310
13.7 Uses of Reinforcement Learning 311
Further Reading 312
Practice Questions 312
14 Markov Chain Monte Carlo (MCMC) Methods 315
14.1 Sampling 315
14.1.1 Random Numbers 316
14.1.2 Gaussian Random Numbers 317
14.2 Monte Carlo or Bust 319
14.3 The Proposal Distribution 320
14.4 Markov Chain Monte Carlo 325
14.4.1 Markov Chains 325
14.4.2 The Metropolis-Hastings Algorithm 326
14.4.3 Simulated Annealing (Again) 327
14.4.4 Gibbs Sampling 328
Further Reading 331
Practice Questions 332
15 Graphical Models 333
15.1 Bayesian Networks 335
15.1.1 Example: Exam Panic 335
15.1.2 Approximate Inference 339
15.1.3 Making Bayesian Networks 342
15.2 Markov Random Fields 344
15.3 Hidden Markov Models (HMMs) 347
15.3.1 The Forward Algorithm 349
15.3.2 The Viterbi Algorithm 352
15.3.3 The Baum-Welch or Forward-Backward Algorithm 353
15.4 Tracking Methods 356
15.4.1 The Kalman Filter 357
15.4.2 The Particle Filter 360
Further Reading 361
Practice Questions 362

16 Python 365
16.1 Installing Python and Other Packages 365
16.2 Getting Started 365
16.2.1 Python for MATLAB and R users 370
16.3 Code Basics 370
16.3.1 Writing and Importing Code 370
16.3.2 Control Flow 371
16.3.3 Functions 372
16.3.4 The doc String 373
16.3.5 map and lambda 373
16.3.6 Exceptions 374
16.3.7 Classes 374
16.4 Using NumPy and Matplotlib 375
16.4.1 Arrays 375
16.4.2 Random Numbers 379
16.4.3 Linear Algebra 379
16.4.4 Plotting 380
Further Reading 381
Practice Questions 382


Update:
OK, I've had this book for quite a while now and I confirm it is a very good introductory text. It's very important to read the first couple of chapters quite carefully and then you have a bit more freedom when reading the rest.
It may annoy people who are mathematically inclined, and if you read French there is a far more "formal" introduction to artificial learning by Cornuejols and Miclet which seems to treat a wider range of topics. It doesnt appear as if that 780 page book has been translated.

Modern Machine Learning is deeply statistical and mathematical in nature, and as others have said, this book aims to trade off some rigor in favor of a more intuitive approach. That in itself is not a bad idea; there should be room for a book that gives the reader a working knowledge of the more important techniques, even if they don't necessarily understand how they work at a deep level. Unfortunately, this book stumbles quite badly in many respects.

We chose the book for an introductory course in Machine Learning at my university, as our students often don't have the level of mathematical background necessary for treatments like the Bishop book. However, I and my colleague often had to resort to essentially replacing entire chapters of material due to serious flaws in the text. In some cases, the author gives definitions for commonly available terms that are simply factually incorrect. For example, the chapter on Evolutionary Algorithms repeatedly confuses parent selection and elitism, states that crossover cannot be defined for non-binary representations, and other similar mistakes. In the chapter on Reinforcement Learning, most of the material is at least correct, but does not progress in any meaningful manner. For instance, the author introduces TD(') methods using execution traces, gives the formula for updating the trace, and then simply stops. There is no indication of what you should do with this value once it's been calculated. There are quite a number of these sort of issues in the book.

I would also concur with the other reviewers who felt that the idea of stripping away unnecessary mathematical formalism has simply been taken too far here. It sits in an awkward middle ground: it doesn't provide enough formalism to tell you how the algorithms really work, but it simultaneously wastes so much space in trying to hand-wave its way around the issue that the code is relegated to fragments that don't give a very complete picture. I think the book would be more successful if it either accepted that at least some mathematical sophistication is necessary and dispensed with some of the cutesy hand-holding, or else dispensed with the pretense of explaining the material in an academically interesting way and simply focused on implementations of the algorithms along with explanations of the code.

On the whole, I think the book does have some value. It's one of the few books covering modern topics in machine learning that can be used by someone without a pretty solid grounding in statistics, and if the reader is motivated to augment it heavily with other sources, it can be a decent starting point for looking into some technique. But that's about it really. I think it's flawed enough that as a single source of information, it's likely to not meet its intended purpose with a great deal of success.

At my job I was asked to jump into the field of machine learning for multiple reasons, and having had a moderate background in mathematics with an extensive background in software engineering I have been in search of the "perfect book" to combine these two levels of competency. With this book, I have found it.

The author doesn't completely weigh you down with mathematical details, but gently introduces you to the topics in a very digestible and comfortable manner. The author knows you will not be able to understand all of the most intricate details in the subject from one chapter, and so very clearly states, (I'm paraphrasing) - "We will cover this in greater detail in chapter XX, but for now just understand these are the important things to note....". Most authors in this field will just dump on you page after page of advanced theory, assuming you pretty much understand it all to begin with; but not this author.

I have recommended this book to all my coworkers who deal with the same tasks as I, and recommend it to any reader out there who wishes to become more familiar with the single-most fascinating branch of computing known to man.

I chose to use this book for a course on machine-learning for computer scientists that I taught in the spring of 2011.
The main reason for selecting it involved its use of the python language, and a more overall programming-oriented
approach to machine learning. I do believe the author has the right idea, assuming the targeted audience is cs majors who
need some basic introductory exposure to the subject. Yes, the author does walk a fine line between trying to provide some
degree of mathematical rigor, and yet not overwhelming the student with too many equations. In doing so, he trips and stumbles
several times throughout the text, but, in the end, I do believe that the average undergraduate would benefit from this book
more so than from a more traditional math-oriented ML book. So it does fill a niche in the ML literature. To improve the book,
say, for the next edition, I would suggest toning down the "cs students need to have their hands held when doing math"
attitude that is purveyed throughout the text, and adding a bit more rigor where needed.

Also, I found much of the python matrix code very slick and often hard to follow. For this reason, the final chapter ought
to be the first read (for those students with no python background) and more examples and exercises should be given
that reinforce the matrix manipulations. I also suggest that the next edition include more problems at the end of each chapter,
and simple exercises throughout the reading,
especially ones that help students practice both the mathematics and the programming. Having lots of exercises can help smooth
out some of the discontinuities that are found throughout the text. By "discontinuities", I mean at times taking large leaps from
intuitive wordy explanations, followed by jumping into the mathematical models.

A great example of a textbook that succeeds in walking the above tight rope is "Theory of Computation", by Michael Sipser.
Every future author who wants to balance accessibility and rigor ought to first read this book for inspiration. Another
great success story is Mitzenmacher's "Probability and Computing". So it can be done! And I hope to give the second edition of
this book 5 stars!

Stephen has done a great job and making a complicated topic accessible. He has mastered the art of knowing when to provide extra treatment of the formulas in the book. The use of code alongside the concepts also helps clarify what is going on. Actually, the code alone is worth the price of the book. It really does make all the difference. I think anyone with some mathematical sophistication (a calculus course, linear algebra, and discrete math) will have no problem reading the book and working through the material. In other words, you don't need a PhD in math to learn from Stephen's great mastery of the topic.

This really is a great book for self study. You will be glad to have this book to guide explorations of machine learning. The author covers both some theory and some practical guidelines to get started with the exercises and the chapter ends with classic references to delve deeper into the literature. This book is a rare find.