Modeling Brain Function: The World of Attractor Neural Networks [Hardcover]

Daniel J. Amit (Author)

Book Description

ISBN-10: 0521361001 | ISBN-13: 978-0521361002 | Publication Date: September 29, 1989 | Edition: First edition.

Exploring one of the most exciting and potentially rewarding areas of scientific research, the study of the principles and mechanisms underlying brain function, this book introduces and explains the techniques brought from physics to the study of neural networks and the insights they have stimulated. Substantial progress in understanding memory, the learning process, and self-organization by studying the properties of models of neural networks have resulted in discoveries of important parallels between the properties of statistical, nonlinear cooperative systems in physics and neural networks. The author presents a coherent and clear, nontechnical view of all the basic ideas and results. More technical aspects are restricted to special sections and appendices in each chapter.

Editorial Reviews

Review

"...of interest to those following the neural net field...takes off from discoveries that link areas of physics with the emerging neural network paradigm." Intelligence Monthly

"...regard this book as an opening of a discussion--undoubtedly a very qualified one." Journal of Mathematical Psychology

Book Description

Substantial progress in understanding memory, the learning process, and self-organization by studying the properties of models of neural networks have resulted in discoveries of important parallels between the properties of statistical, nonlinear cooperative systems in physics and neural networks.

Product Details

• Hardcover: 528 pages
• Publisher: Cambridge University Press; First edition. edition (September 29, 1989)
• Language: English
• ISBN-10: 0521361001
• ISBN-13: 978-0521361002
• Product Dimensions: 9 x 6.2 x 1.3 inches
• Shipping Weight: 1.6 pounds
• Average Customer Review: 4.0 out of 5 stars  See all reviews (1 customer review)
• Amazon Best Sellers Rank: #1,226,757 in Books (See Top 100 in Books)
  • #37 in Books > Computers & Technology > Computer Science > Artificial Intelligence > Artificial Life

More About the Author

Discover books, learn about writers, read author blogs, and more.

Customer Reviews

Most Helpful Customer Reviews

7 of 7 people found the following review helpful:

4.0 out of 5 stars Of historical importance, October 3, 2005

By 

Dr. Lee D. Carlson (Baltimore, Maryland USA) - See all my reviews
(VINE VOICE)    (HALL OF FAME REVIEWER)    (REAL NAME)   

Amazon Verified Purchase

This review is from: Modeling Brain Function: The World of Attractor Neural Networks (Hardcover)

The study of the physics of the brain from the standpoint of dynamical systems was very popular during the 1980's. The theory of chaotic dynamical systems, and the accompanying concepts of strange attractors, horseshoe maps, and fractal basins of attraction was the subject of intense research at that time. It was inevitable perhaps that these theories would be applied to the understanding of the brain, given the dynamical nature of the neuronal synapse. This book, published in 1989, gives a good overview of what was known at the time. It could be read by anyone with a background in dynamical systems and some elementary knowledge of brain biology. The mathematics is also straightforward in that the author does not bring in any of the heavy tools from differential topology or measure theory, which is normally done in discussions of dynamical systems.

There are some points made in the book that must be understood by the reader because the author feels that they are needed to build a successful model of the brain. For example, he discusses the notion of an `input system', which is a system that, for each input, produces and output with the same "status." Cognitive discrimination must be used at the input level, if one is to avoid the use of the `homunculus' (the little external observer), for distinguishing between "good" and "bad" outputs. The major task in the author's view is to produce "exceptional" input-output relations, i.e. relations that correspond to intuitions about cognitive processes. A successful brain model, i.e. one that is able to incorporate memory, should be able to distinguish between stimuli that are familiar from those that are to be submitted to the brain for processing or learning. Thus the model must avoid the use of what the author calls `spontaneous computations', which require an external observer (the homunculus again) to interpret the relation between the input and the output. The author gives an example of a system that performs only spontaneous computations early on in the book. Hence the author proposes the use of artificial neural networks (ANNs) to avoid the occurrence of spontaneous computations. An ANN organizes stimuli in association classes represented by an attractor, and all the stimuli in a particular class are associated with the attractor to which they flow. The author feels that ANNs are more adept at respecting the requirement that for mental computations, which are essentially operations on temporal sequences of data, some record of the initial input sequence must be carried along on a parallel channel, in order to provide the outcome with specific "meaning" and a correspondence to the assigned task.

These considerations on the dependence of the processing on the initial input motivate the author to discuss the role of ergodicity in the dynamics of the neural systems of the brain. As the author shows, any generic system subjected to noise will be ergodic, so that eventually the system will access each of its possible states in a manner that is completely independent of the initial state. The author points out two ways in which ergodicity can be avoided: one is to assume that the network is noiseless, and thus only certain moves are allowed from each vertex; the other is to assume that `cooperative phenomena' is present. Since the first possibility is rather exceptional, the author chooses the second, and gives detailed discussion on how cooperative behavior can arise in ANNs. One interesting, and ubiquitous example that he discusses for cooperativity as an emergent property is the Ising model. Mathematically, the breaking of ergodicity involves the taking of the thermodynamic limit, and a necessary condition for emergence is this context is the asymptotic degeneracy of the eigenvalues. To illustrate how this is done, the author uses the solution of a master equation that characterizes the probabilities of making transitions from one state to another in the system.

In order to build a credible model of the neuronal processes of the brain, the author is aware that such a model has to be able to deal with input in the form of temporal sequences, and not just single patterns. He devotes an entire chapter to this in the book, motivating his discussion with the notion of a `central pattern generator' (CPG). The simplicity of CPGs is a concern and the author is aware that such simplicity does not exist in models of cognitive processes. Nevertheless the modeling of CPGs using neural networks can add credence to the program to model general brain processes in terms of neural networks, complex as they can be.

One of course must be able to deal with both the storage and the retrieval of temporal sequences. After discussing some of the early research dealing with these needs, the author then reviews a strategy for dealing with temporal sequences that involves the notion of a `quasi-attractor', which is a network state that acts like an attractor for a short period of time. Quasi-attractors are used to delay the transfer of information out of the attractor. Thus the transitions are governed by synapses that have a time delay. The influence of a pre-synaptic neuron through these synapses will arrive later than the influence coming through a `stabilizing' synapse. The latter type of synapse arises because of the `stabilizing' term in the network model that ensures that if the network is in a state that is identical to a stored pattern then the network will remain there. The author shows how the network can use these delayed transitions to deal with temporal sequences in a manner that is acceptable, i.e. in a way that the `cognition time' is of the order of magnitude of the delay. The author discusses an example dealing with the counting of chimes, in order to give credence to his constructions. In this example it is seen that the network resides in each of the quasi-attractors for a long enough time so as to allow the output neurons to identify the cognitive event.