Customer Reviews

Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks

5 star:		(8)
4 star:		(0)
3 star:		(0)
2 star:		(0)
1 star:		(0)

 

 

It was a pleasure to have the opportunity to read the Russell Reed, Robert Marks book "Neural Smithing". I have been an engineer at Boeing for 20 years involved in computing, CAD 3D design, and related applications. My current assignment includes supporting a sophisticated Neural Network based design retrieval system. I am also completing my Ph.D. dissertation based on Neural Network research.

To begin with, it seems reasonable to characterize "Neural Smithing" in broad terms. The book is not just a stuffy, hypothetical, academic treatment of Neural Networks loaded with formulas and references. It does contain these elements but they are encapsulated in a larger presentation which leads the reader on an adventure of exploration and an ultimately satisfying journey of discovery. The book is certainly well grounded in theory and motivated by classical approach but the overall message is: you too can make neural networks from scratch by following the principles, guidelines, suggestions, and hints presented in this handbook. What's more, your network will probably perform correctly, or at least you'll understand the reason why not.

"Neural Smithing" guides the reader through various channels and pathways, around pitfalls, and ultimately to an understanding of neural networks on a personal level. The reader comes away from the first reading with a feeling of intimate knowledge and intuitive understanding of neural networks. After this, the book transforms from a required travel guide into a trusty reference book. With over 380 references, it is a veritable who's who in neural network technology and a "must have" for any serious experimenters workbench.

The underlying assumption the authors seem to be working from is that the reader will in fact "Roll up their sleeves, sit down at a computer, and get involved with neural networks at a working level". Ultimately, they will arrive at the intersections of discovery marked by the various chapters of the book. Here, the information contained in the book stands ready to guide the "user" through these intersection in an interactive way where the reader participates in the choices. The reader is made to consider their particular application and the implication of a variety of choices on the results they are interested in achieving.

The authors bring a unique understanding and hands-on experience to their discussions. While the book is liberally referenced, the originality of the book is recognized at a higher level as the authors use their experience to facilitate overall integration into a coherent, comprehensive presentation. This book gets high marks for presentation, particularly in the areas of line-of-reasoning and chain-of-development. The style will be instantly recognizable to readers versed in formal, axiomatic development and presentation.

"Neural Smithing" is a sophisticated work replete with Hypercubes, Conjugate Gradient Descents, Correlation and Variance, Voronoi tessellation, and Genetic Algorithms. Yet least the reader begin to feel that they are in peril of being lost it conceptual hyperspace, have no fear. The authors have a down to earth (3-dimensional) way of pulling you back. "Just think of error optimization as a marble rolling around on hills and valleys," encourage the authors. Well, when you put it that way, how can you miss. Other euphemisms include "pin the hyperplanes to the data", and "leap-frog weights". The bottom line is that the authors have a way of bringing you with them even when the going gets thick.

Finally, I believe this work to be comprehensive. It is well paced from beginning to end and covers all the bases in a very logical format. First, the basics are presented in excellent depth. (I believe other books miss this opportunity in many cases). Next, the heart of neural network theory and operation is presented and classically motivated. Finally, more exotic topics are covered (in equivalent depth I might add) such as Genetic Algorithms as they apply to neural networks, and generalization heuristics. The discussions are extremely well documented with references, formulas, and very relevant figures. Also, the appendixes provided are excellent and pertinent.

"Neural Smithing" is a handbook, a reference book, a cookbook complete with recipes for getting the job done, and finally a documentary on the exploration of an exciting field which is clearly setting the foundations for eventual understanding of the human mind. I would strongly recommend this book to someone who feels a compelling need to have access to the tools of this realm. This book is a vehicle for exploration.

Frank S. Holman III, Boeing Commercial Airplane Company, Seattle, WA. 98124-0346

Many textbooks can help me to understand the different concepts of neural network, but not the practical tips needed to optimize neural network anlysis and implementation.

The topics covered are reminicent to those discussed in part 2 and 3 of the Neural Network FAQ. In chapter 6, the relationships between learning rate, momontum, trainig time and learning modes are presented graphically. With this, it helps me to rule out and avoid learning parameters that are unlikely to improve the NN performance. This is especially important if the dataset is large and the NN program is implemented in Java.

If the aim is to develop a NN solution that will give you the best results, I find both chapter 7 (heuristics for weights initialization) and 16 (heuristics for improving generation) are esential and saves me a lot of time from reading many journals.

In summary, this book has helped me to develop the art of NN optimization. It shows me how to visualize decision surface and the various graphical relationships between learning paramters and various components of NN topology. I think you will find this book very useful after your NN program is up and running and you are looking for ideas and explaination on how to improve the NN performance further.

The book is a five star effort. Here is a review circulated in popular neural network newsgroup:

Newsgroups: comp.ai.neural-nets From: saswss@hotellng.unx.sas.com (Warren Sarle) Subject: Neural Smithing Message-ID: <F8wrMs.6Dp@unx.sas.com> Organization: SAS Institute Inc.

I have added a new book to the list of "The best elementary textbooks on practical use of NNs" in the NN FAQ (it may not show up on the server for a few days):

Reed, R.D., and Marks, R.J, II (1999), Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks, Cambridge, MA: The MIT Press, ISBN 0-262-18190-8.

After you have read Smith (1993) or Weiss and Kulikowski (1991), Reed and Marks provide an excellent source of practical details for training MLPs. They cover both backprop and conventional optimization algorithms. Their coverage of initialization methods, constructive networks, pruning, and regularization methods is unusually thorough. Unlike the vast majority of

books on NNs, this one has lots of really informative graphs. The chapter on generalization assessment is a little weak, which is why you should read Smith (1993) or Weiss and Kulikowski (1991) first. There is a little elementary calculus, but not enough that it should scare off anybody.

One minor complaint: "smith" is not a verb!

Warren S. Sarle SAS Institute Inc. The opinions expressed here saswss@unx.sas.com SAS Campus Drive are mine and not necessarily (919) 677-8000 Cary, NC 27513, USA those of SAS Institute.

Some books just have the right feel about them and this is one of them. The author clearly knows neural networks and also knows how to communicate to others in a no-nonsense fashion. With so many books being published, you rarely find a technical book that is as good as this. The emphasis here is on conveying the insights that the experts in this area would know. Importantly, concepts are explained equally in words, graphics and mathematics, maximising the uptake of knowledge from the book. Tufte would be impressed by the quality of the line graphics in this book, and the information that they convey, not to mention the overall presentation. I suspect that this book would make an ideal textbook for a course in neural networks. Overall, I've enjoyed reading it very much.

Book is excellent. Covers the theory very well, such that you can make the computer code yourself. They also provide puedocode. You will be able to learn it better than other books that just give you the code. I find that once you understand the theory, writing the code is easy.

Everybody who tries to use NNets for real goes through these steps.
First, there is the Delta rule.
Then, there is overfitting, local minima, generalization problems and frustration.

The complexity of NN is not in it's math; the difficulty is in the construction of a NN. This book is excellent in providing rules-of-thumb for NN construction, while at the same time providing the theoretical backing.

Hey I am not making money reviewing this book, it's just really good.

After owning this book for many years and reaching for it many times, I decided that I at least owed the book a five star review.

Early in my graduate career I began working with neural networks and discovered this book in a electronic bookshelf available at my university. After printing chapter after chapter to read on subway rides home I ended up buying it for convenience. It gave me the background I needed to code up a basic artificial neural network in C++ and to then extend it to fit my needs.

The style of the writing is the perfect balance of enough detail to understand a concept or method without unnecessary wordiness. Each chapter covers an important aspect of neural network development and application - for exmaple, internode weight initilaization techniques - and acts a sort of mini-review of the most popular methods with a clear explanation of the pros and cons of each.

This is an excellent bookshelf addition for anyone who works with neural networks.

I bought this 9 or 10 years ago when I became interested in AI, but my math skills were pretty weak at the time. I've since completed a CS & math degree, and I find this book easy to read, thoroughly interesting, and insightful.

Many of the questions I found myself asking while I read were soon answered as I read later sections of the book.

For those considering the purchase and unsure whether they can handle it, for much of the book a decent exposure to calculus will suffice. For a few chapters some exposure to ordinary differential equations would be wise. Numerical Analysis is probably a good idea as well.

An exposure to probability & statistics (not the freshman version) would help as well. The section on initialization techniques talks about various probability distributions when determining methods for initializing weights. If you don't care about the why's, it can be used as a reference for coming up with a scheme for weight initialization, but I find it handy to know why my code is doing something so I better know how to tweak it.

-Brian