Our Final Invention: Artificial Intelligence and the End of the Human Era Hardcover – October 1, 2013

James Barrat, as a filmmaker and journalist, has layman's style that makes the topic accessible to non-experts. Some reviewers have commented that his lack of subject matter expertise is reflected in the weakness of his arguments. I disagree, Barrat went to great pains to interview many of the big thinkers on AI and its potential impact on society and humanity - including optimistic boosters, skeptics, and doomsday alarmists. While he ends up closer to the alarmists, I feel he got there by the strength of the arguments he heard along the way. If a machine can attain human level intelligence and is able to recursively improve its own code and therefore the rate at which it learns and becomes more intelligent, there are reasons to believe such a machine could exceed our intelligence many times over in a very short period of time. Well just unplug it, you say? The AI already thought of that and stored itself all over the place. It is not a machine anymore, it is an unbounded sentience. If we had it contained on one system completely cut-off from all networks, what strategies might it devise to convince us to connect it to the wider world? Promise a cure for cancer? The solution to clean cheap energy? Reversing climate change? if only it had a little more computing power.... Also consider this - what would a super-intelligence want, what would be its ambition? Just survival? Expansion? As Barrat cleverly puts it, maybe it has different uses for our human molecules than we do. And finally, consider that probably the most aggressive investments in AI are happening largely in the dark, by DARPA and defense contractors in the US and by God knows who in other countries. So why would we assume that the first sentient human-level AI will be friendly? These are the kinds of questions Barrat grapples with, and it is a heck of a ride.

The author is a film documentarian venturing into speculation about potential impacts of artificial intelligence from research to implementation. Specifically he evaluates likelihood and threats of developing AGI (Artificial General Intelligence) and eventually ASI (Artificial Strong Intelligence). His observations are based on extensive interviews including those with Kurzweil, Yudkowsky, Omohundro, Vinge, and Dyson among others.
My initial reaction to this book was skepticism because not a scientific technologist. I expected that he may miss more subtle but important technical steps being taken on this road to artificial intelligence (AI). The further I read the more it became clear he is providing some pointed observations derivative of his experience as interviewer for documentaries. In general his conclusion is that AGI and ASI constitute existential threats as a function of the rapidity and manner in which they are developed. The process of development is not clearly established because of a diversity of technical opinion regarding both feasibility and impact.
The range of opinion is very broad and nuanced. At one extreme is Ray Kurzweil whose many books on technology generally are most optimistic as among a group of those researchers with knowledge and experiences in this technological future. Though most optimistic he is also highly qualified not only as an analyst of tech trends but also developer of tech tools that, before his time, were regarded as difficult if not impossible. Among these is the optical character reader and some preliminary work leading to SIRI. He topped up his views with the most recent book “How to Create a Mind”. Though a summary of technical concepts it possesses many realistic elements in the work of such as Jurgen Schmidhueber and others working with neural nets.
If Kurzweil is at one extreme Yudkowsky and Vinge are probably at the other. Both express sceptism AGI or ASI development will prove benign venturing opinions that work toward artificial intelligence should be severely curtailed to the extent of stopping short of artificial strong intelligence (ASI) specifically.
In between these two extremes there are examples of opinions falling over a fairly wide range of future possibilities - increasingly probablities. The algorithmic avenue is already demonstrating some of the potential of AI. There are probably few finance and investment firms without one variation or the other of algorithmic high speed stock analysis and trading systems. These evince many elementary ingredients one may expect to see in future AI. So technically thorough as a matter of fact they operate relatively free of human interaction in producing recommendations for investments, effectively making ‘intelligent’, i.e. statistically valid, ‘decisions’.
In meantime the advances continue unrelenting toward a distant ASI/AGI future. The time frames, for example, between IBM Big Blue and Watson are shorter than forecast, and end products as powerful as planned and then some. Still neither of these developments is more than steps on a road to AI while also being quickly followed by other developments such as recently announced SYNAPSE development by IBM. All closer steps to technological ingredients on the AI road to human future.
There is some movement among AI researchers that a congress should be convened of the sort genetic researchers held in Asilomar California. That is, a convention to establish ground rules and limits on directions of AI research.
One of the cautions about development progress of AI-like tools is based on the important role played by DARPA (Defense Intelligence Research Projects Agency) as it provides a large percentage of funding for various projects underway including an annual robotics competition to observe advances approximating many human qualities of movement. Clearly this agency has a mission antithetical to a purely humane result of AGI/ASI. After all DARPA is in the business of developing ‘weapons’ for military use – a not altogether benign mission in technology except perhaps as seen from point of view men at arms.
The author mentions impact ASI and AGI will have on employment. His pessimism is mirrored in an Oxford University study concluding advancing tech developments pose an explicit threat to an estimated 47% of the 702 employment categories of the US Department of Commerce. While this report is an estimate it nonetheless raises the same sort of questions about computers in general, ASI and AGI in particular, and their impact on society. The report has recently been augmented with estimates of tech influence on employment in many other countries of the world.
Another Oxford author is John Bostrom who outlines in great detail a road from our present to some future of AGI/ASI. A more recent development centers around Musk and Tegmark motivated by concern to fund and form an institute for evaluating threats and benefits. There is a persistent sense of threat from computers, automation and robotics dating from decades before the present. More recently this sense of threat seems to be accelerating concern about our human future with highly developed robotic associates. Barratt is a lucid presentation of the issues from a non-technical point of view.

I bought this book after sampling about six others because this book doesn't pretend to be or lapse into pedantic, overly written, domineering, bull pucky. This book is readable and revelatory.

If you don't read this book, then you simply are not in the actual reality loop. James Barrat has made an exhaustive study of and deep commitment to this topic. He has reached deep inside his creative soul to try and harness all he learned to extrapolate and make his best educated guess at what the future of AI may hold for machines and for human beings.

Also, as a aside, it was clear to me that the writers of the extraordinary film Ex Machina, MUST have studied this exact book in their writing of the very script for the film itself. There are sections that clearly fortell what the Intelligence (AGI-ASI) in the film sets out to accomplish---or must accomplish---in order to survive and fulfill its prime directive.

I recommend this revelatory work for anyone interested in the topic, whether just in passing, or in dead seriousness.