Customer Reviews

Useless Arithmetic: Why Environmental Scientists Can't Predict the Future

5 star:		(5)
4 star:		(5)
3 star:		(3)
2 star:		(1)
1 star:		(8)

 

 

The first author is a retired professor of geology and a particular expert on beaches. He's a scientist's scientist, and clearly an opinionated and occasionally irascible guy. This book is a bit of a tirade in places but it's full of real examples, good data, and thought provoking stories. I enjoyed it a lot. The main theme is that the natural world is too complicated a place for quantitative models to work well, and that when politics is involved they can lead to really bad decisions. The majority of examples are drawn from cases where earth sciences meet human activities - sea level rise, beach erosion and "nourishment", hydrology of abandoned pit mines, storage of nuclear waste. Closely related are discussions of fishery management and invasive species. For the most part the book is well researched. The writing is clear - the book is an easy read and never boring.
Quantitative models are decried throughout the book, and the suggestion is made that what is reasonable is "qualitative" modelling. The distinction isn't really developed until the last chapter where some good examples are to be found. Still, the distinction isn't as crisp as I'd like - perhaps it is a qualitative difference and not a quantitative one! Another positive suggestion is that incrementalism is a generally better approach to interacting with the complexities of nature than the brittle approaches that arise from an overly numerate engineering mentality. In other words, instead of using quantitative models to plan enormous, long-term projects, try something on a small scale, observe the results, and go from there.
I came away with considerably more knowledge of the topics discussed. I was already a convert to the basic themes - that we tend to overestimate what we know, to trust numbers more than we should, that political processes often interact with science in ways that are inimical to both good decisions and greater knowledge. Several times I thought of Eisenhower's dictum that plans are generally useless but planning is essential. Perhaps that captures best the distinction Pilkey is trying to make about qualitative models.
Unlike some of the other reviewers, I was not offended by the political implications of anything Pilkey asserts. I didn't see it as either pro or anti global warming in any political sense. No hidden agendas here, it's really about modelling. Recommended.

At first glance, it is odd to single out environmental scientists for being unable to predict the future. However, like stock pickers, some preachers and electoral pollsters, environmental scientists do make a business of predicting the future.

Not one of these groups has had any success, but it is arguable that the environmental scientists have done the most damage to other people by being wrong.

Orrin Pilkey, a well-known if not always well-liked specialist on coastal processes at Duke, began a seminar to examine why the predictions of coastal engineers seemed so often to lead to projects that didn't work. (I have known the North Carolina beaches where Pilkey does much of his work for more than 50 years. The Outer Bankers hate what Pilkey says about their beaches, but he's right.)

The investigation led to a wider examination of numerical models of all sorts of natural processes. Among those examined -- all failures -- were managing the Grand Banks fishery, predicting the lifetime of nourished beaches, predicting toxicity of lakes in abandoned pit mines and predicting how fast sea level will rise. Predictably, more attention is paid to coastal processes than anything else, but other topics get fair treatment, even one as far from the coast as the Yucca Mountain atomic waste dump.

What the Pilkeys found was no surprise to me as a newspaper reporter, and will be even less a surprise to scientists. People forget, but reporters keep clipping files. Mine contain many reminders about predictions made but unfulfilled. The Pilkeys conclude that it is impossible to write quantitative numerical models of any complex process on the surface of the earth. This is unlike numerical models of tractable systems, like the orbits of comets or the fracture strength of a steel-frame building.

For most people, the most familiar such models are the global circulation models used by the Intergovernmental Panel on Climate Change to predict catastrophic global warming. The Pilkeys barely mention GCMs in passing, and this is probably wise. If their goal is to get non-technical people (or even technical people with open minds about modeling) to rethink the enthusiasm for them, then injecting global warming into the mix would no doubt distract from their greater goal.

However, the deduction is clear and inescapable -- forget the GCMs. They offer no guide to anything.

The reasons all these models fail are many. One the Pilkeys emphasize more than any other is the "ordering problem." Even if you knew all the parameters that count (seldom or never the case), and even if you knew how they interact with each other), you can never, ever know in which order the variable ones will present themselves. If current changes affect beaches and hurricanes affect beaches, and both are variable over time, you cannot know in advance whether the storm will come first, or something will alter the current first.

This is not an original thought. Although he did not call it an ordering problem, the immunologist and philosopher of science Peter Medawar identified it as the reason that it will always be impossible to conduct meaningful experiments to disentangle the effects of nature and nurture on human behavior. (See his essay, "Further Comments on Psychoanalysis" in "The Strange Case of the Spotted Mice" and my review of that book.)

Other critics, such as Vaclav Smil and Naomi Oreskes, have also demolished the concept of numerical modeling of complex systems, and the Pilkeys give them credit for their work. Even outside the natural sciences, the empirical failure of attempts to predict (or even to quantify) complex systems have been demonstrated conclusively, for instance by lawyer Michael Wheeler in "Lies, Damn Lies and Statistics: the Manipulation of Public Opinion in America." That book was published more than 30 years ago; there really can be no excuse at this late date for people to question the Pilkeys' conclusion.

People do. Orrin Pilkey has been called a neo-Luddite by defenders of quantitative modeling.

The beaches, however, are not there, just as he said they would not be.

As an alternative, the Pilkeys advocate qualitative modeling. It gives, at best, trends, it outlines areas that might require remediation as things develop. Its asset is flexibility. That's something you never get with a seawall. Those are inflexible.

Another approach, about which the Pilkeys take no firm stand, is the Dutch method of dealing with retreating beaches: dump and run. In other words, build up the beach without expectations of how soon it will have to be done again. The Dutch, with their heritage of centuries of battling the sea, probably think in terms of processes rather than "solutions."

This short but powerful volume is explicitly aimed at non-technical readers. Although quantitative models are mathematical, it is not necessary to know math to understand how they work, and the Pilkeys relegate what little math is in the book to an appendix. Like Orrin Pilkey's lectures to public audiences on coastal erosion, he can make complex processes understandable, if not predictable.

I only wish the Pilkeys had included a chapter on wellhead protection zones, which are going to become an increasingly hot topic in many areas. These attempt to predict how far you have to keep pollutant sources away from the places you find your drinking water. The approach used so far has been numerical modeling. "Useless Arithmetic" does not encourage confidence in the protection policies being adopted.

I wanted to give this book three ratings: Five stars for the basic tenets, three stars for supporting their tenets and one star for violating every valid point they make when the discussion turns to man made global warming. The book is relatively short, but should probably be condensed to a chapter. Bottom line: I didn't find much in the book that wasn't already in one of the reviews.

The Good: The authors advance the idea that mathematical (computer) modeling of complex systems is often misused. A combination of not understanding all of the important physical processes, making inappropriate assumptions about initial conditions and improper handling of chaotic events make the predictions inaccurate. The process is further corrupted by politics, money, bias and hubris. This is all too true.

The subject material is presented virtually without mathematics and can be understood by just about anyone. It will also arm the lay person to ask questions that are likely to be embarrassing to most models. Those questions range from "what were the assumptions?" to "has the model successfully predicted events or does it need to be constantly fudged to match the real world?". Multiple examples are given of assumptions and processes that violated basic common sense.

I was particularly interested in the chapter on modeling the nuclear waste repository at Yucca Mountain. I made some of the permeability measurements of basalt and granite that were used in the models. I shared office and lab space with people that generated a lot of the measurements on salt that went into the modeling effort. I can't offer independent confirmation of every statement the author's made. However, their comments about the models making the ludicrous assumption that cracks and fissures weren't important was spot on. My colleagues and I also published work that attempted to measure the characteristics of cracks and fissures and evaluated fluid flow through these cracks and fissures. The modelers made a conscious choice to ignore the latter data.

The bad: The authors are apparently unable to apply their lessons outside of their own field of expertise. After successfully demolishing a number of models that had been accepted without question, they turned to global warming. They spent the chapter praising the climate modelers for admitting that the models didn't accurately predict sea levels.

However, they completely accepted the idea that man made carbon dioxide was an important contribution to global warming. In doing so, they violated every statement they made about blindly accepting models:

1) They were apparently unaware the greenhouse effect is a theory and that the primary proof of the theory is an unverified mathematical model.

2) They make the case that we should believe this model because every credible scientist endorses it. They had previously made the point that other widely accepted models have been wrong. Further, this statement is a lie unless the test for credibility is agreement with the model.

3) The climate models predict that the greenhouse effect will warm the lower atmosphere (energy that would otherwise be radiated to space is captured by the lower atmosphere) and then be transferred to the Earth. Measurements have not shown the predicted temperature increases in the lower atmosphere. Further, the models predict that the effect would be stronger at the poles than at the equator. Measurements have shown the temperate zones getting warmer and the poles getting colder. This has not caused the authors or the proponents of man made global warming to reconsider the fundamental assumption.

4) Political bias has resulted in the IPCC report being modified to remove statements about the limitations of the model that the scientific community had included. The authors do point out that global warming research is a $4B business that will seek to perpetuate itself and then praise the global warming work while attributing an economic bias to those that question it.

5) The models have not successfully predicted anything. Further, the modelers have not been able to tweak the models to generate some important physical phenomenon such as the South Pacific heat vent (NASA says that the vent is triggered by a small rise in the ocean temperature and radiates about the same amount of heat as is purportedly generated by man made greenhouse gasses).

For a lucid and comprehensive discussion of the opposing views, I recommend "Unstoppable Global Warming Every 1,500 Years" by Singer and Avery.

The ugly. Not being content to ignore their own advice re. questioning the climate models, they stoop to characterizing those who do question the model with words such as "profoundly unenlightened", "rabid", "clumsy and disingenuous", "not objective", "high and mighty", "motivated entirely by economic consideration", "lack of scientific integrity". This displays a dismaying lack of intellectual integrity on the part of the authors.

As a systems engineer, I have practical experience in creating, testing, critiquing, and evaluating models that attempt to explain, predict, or illustrate system processes. Any engineer learns early on that regardless of what the model says - Reality Always Wins. Thus I was very interested in this book because of its evident intent to discuss the limitations of modeling as applied to natural processes.

Unfortunately, the authors exhibit a level of bias against any model they don't approve that is so over the top that I was constantly wondering what cheese would be served with the "whine". And then they cap it off by blindly accepting an entire range of dire global warming predictions, which are entirely derived from - you guessed it - models of complex natural processes. I guess if you like the model's answers then it is magically a good model.

I have a hard time accepting what appears to be intellectual dishonesty, so although the book makes some good points, I really can't recommend it. The authors also appear to be particularly upset with certain individuals and organizations in the coastal engineering community, because the animus comes through loud and clear.

If you really want a good book on the limitations of mathematical modeling as applied to the real world, there is a two-volume set called "Reality Rules" that is much better. However, the Reality Rules books are not aimed at the layperson, so be prepared for some real math in these books.

A repetitive and droning treatment on the topic of the over-reliance of the scientific community on computational models. Of course computational models can be taken as clear and incontrovertible predictions, which generally occurs in the context of scientific consulting in the case of an environmental problem. What the authors overlook is the standing opinion within the scientific community of computational methods as tools to further explore relationships that may not be measured or predicted easily due to constraints of time or technology. The misinterpretation of models as deterministic predictors of the environment is rapidly becoming outmoded among scientists, as environmental modelers (in the case of those I interact with, hydrologists) recognize and develop tools to express the uncertainty that exists in those predictions. That some among us choose to ignore the existence of that uncertainty, and the policymakers and business executives who expect results and get the results they expect, does not reduce the importance of such models to the point of being useless.

Some sciences, such as climate science are non-experimental which means their conclusions cannot be empirically demonstrated. They instead rely on the logic of a computer model. Models of natural systems are useful tools but because they cannot incorporate all of the variables or the sequences in which those variables operate, they must omit, generalize and assume. Accordingly it is impossible for them to predict a definite future with any useful degree of certainty. They are, however, handy for qualitative and order of magnitude projections.
Unfortunately too many folks are seeing models as oracles. This book examines seven natural system models and points out their uselessness as accurate quantitative predictors. They do not throw the work of climate science into this bin but recognize that there are loons out there who are attributing too much certainty to those models.
Well written, with touches of humor, this book is core for living with these darn models.

"Usless Arithmetic" was, both, good and bad. The authors seem to know the subjects upon which they are writing. Mostly, they write that quantitative computational models of complicated natural processes have severe limitations and tend to give misleading answers. They seem to believe that a better approach is to form stronger qualitative notions as to what is happening, before attempting quantitative models. Their evidence is the damages that can happen and which have happened. So much for the good.

However, by dispensing with the math, the authors force the reader to depend on the abilities of the authors and on their honesty. Unfortunately, the authors appear to choose to believe in global warming and disregard their own advice. After criticising models that don't work well, the authors grasp onto global warming, while mentioning some of the oddities of the warming models. Very strange.

My problem with the above situation is that the criticism of complex quantitative models has extremely wide applicablity in today's world. Apparently, one of the reasons for the housing-finance meltdown was a flawed model that did not, properly, consider the cyclic nature of financial markets, nor consider the possibility of a 'bubble' seriously enough. The result was that about one trillion American dollars got flushed down the drain. Ouch! Worse, the problem in financial banks added to a couple of other problem areas and the stock market retreated. The result of that is a few trillion dollars vanished and the Congress is about to throw another trillion dollars at the problem. So, yes, quantitative models can cause very serious, very wide-spread toubles, when they go wrong.

Which leads me back to global warming. The authors may accept global warming, but why do they not apply their own reasoning to that situation? They should have said: "The quantitative models for global warming need to be improved. Thus, a consideration of the qualitative factors shows that ..." That would lead to enough room to contemplate things like the cyclic nature of solar storm activity and its effects on Earth's climate.

Also, I do wish the authors had gone into more detail on the fallacies involved in accepting the predictions of the far future, without showing the degree of success or failure in predicting the near and the middle future, first. Interpolation far beyond the available data is, usually, a very bad idea.

Finally, I have one last grump. The authors seem to be all or nothing folks. If something is somewhat in error, it is in error. Thus, problems in models have lead to their title: "Useless Arithmetic". My own experience is that one can take a failed model and improve it. So the fact that the original model is inadequate is only another step in the direction of an acceptable model.

As many reviewers have said, this book gives an excellent account of some deeply flawed modeling practice. Several of the one-star reviewers repeat tired arguments defending the models that are refuted in the text. This makes the book an excellent series of object lessons for modelers, which can also be read as cautionary tales for people who take actions based on models.

In all the cases cited, however, the model was either dishonest or incompetent or both. That is not always true of the researchers working on the models, but the end result of the process they participate in is fraudulent. If you don't want to get to the truth, if you don't attempt aggressively to falsify your model, if you ignore anomalies and criticism, if you hide your data and analysis and obfuscate your model, if you don't backtest rigorously; you will get erroneous results whether you quantify or not.

Quantitative and qualitative modelers do the same sorts of things. Quantitative modelers do them more systematically and insist on consistency. Those two things come at a cost, however, and there are many situations in which a qualitative prediction is better. The authors claim that outside of physics and engineering, quantitative modeling cannot work because the world is too complicated. I think that's nonsense. They also claim that quantitative models are more susceptible to the problems listed above. That appears to be true in their experience. In mine, while I have met good and bad practitioners of both, I find slightly more reliability in the quantitative camp. However, these are quantitative models that can be and are explained in transparent, non-technical terms, with openness about data and methods. I would agree with the authors that complex and obscure quantitative models are very rarely good.

One point that is not emphasized is the examples all concern government work, or people attempting to get government permissions. I think this is a more important common denominator than use of mathematics. While there have been huge modeling failures among private individuals, they pale in frequency and size and damage and idiocy to government actions. I'm not an anarchist, I don't think all government action is bad, but when bureaucrats play with dangerous things from weapons of mass destruction to environmental models, I prefer to be far away.

The one exception to the above is some completely unsubstantiated swipes and financial models based on superficial reading of secondary sources. There is nothing like the detail given in environmental models. The authors clearly despise finance, they assert it has no social worth and lament that practitioners receive Nobel prizes; and they claim without evidence that positive results are impossible. They are entitled to their opinions, of course, but these brief passages do not add support to their position on modeling.

There is discussion of climate change that some reviewers think contradicts the argument of the book. I disagree, I think it sets it off nicely. Climate change models have demonstrated three things pretty conclusively: that the climate is changing; that the amount of carbon emitted by humans, depending on feedback effects, could significantly alter the climate; and that lags in the process are such that by the time a trend is obvious it can be difficult or impossible to reverse. These are valuable things to know, and it required quantitative models to establish them.

The remaining argument is political. Some people feel that it's safer not to emit than to emit so we should take strong action to limit carbon emissions. Other people feel that the results of that action is unpredictable. It might not actually reduce carbon emissions, and it may cause increased emissions of things that turn out to be more dangerous or other bad things. Climate change might make things worse or better, and human carbon emissions could help or hurt. What is predictable is the actions will increase government power and expense, will lead to corruption, will slow economic growth and will reduce freedom. Better to get more people out of poverty faster so we have more educated people working on how to solve problems, and build up more economic and technical resources to tackle the problems that do arise whether warming or anything else.

No amount of modeling will solve that basic issue. Moreover, we are almost certainly going to come to some kind of compromise with moderate discouragement of all kinds of emissions and research on both how to reduce human impact on the planet and how to engineer climate and adapt to changes. Pessimists and optimists will each get something. The fight between fear-mongering opportunists who insist they know the precise effect of cap-and-trade legislation on average global temperatures a century from now, and greedy polluters who demand mathematical certainty before taking sensible precautions, is not a fight about modeling, quantitative or qualitative.

When you take the politics out of it, I don't think the issues are too hard. We'll always have people who feel different ways about the same scientific facts, models won't make that go away. We'll also always have people who abuse models to get their way, and people who fight legitimate use of models for the same reason. I think the book would have been clearer separating the reasonable political disagreements from the fraudulent abuse or nonuse of models.

This book claims that environmental scientists can't predict
the future, and it offers to tell us why they can't. It provides
lots of evidence that that quantitative models in many realms
have been grossly inaccurate. Various chapters cover Yucca Mountain,
sea level rising, beach erosion and rebuilding, fishery yield
management, and effects of open pit mining, and invasive plants.
The authors claim that all environmental processes are too complex
to ever be modeled quantitatively. They are probably too
pessimistic in the long term, but they are definitely right so far.

The real problem is how the models are used, or rather misused.
Many seem like first guesses of the process, good enough to design
experiments or begin data gathering. Sometimes the creator of the
model knows it limitations, but usually does nothing to prevent
misuse. More often, the parameters are adjusted until the desired
answer is produced. They claim none of the model predictions are
ever compared to reality. The horror stories about bad predictions
are the best part of the book.

Several reviewers have complained that the authors support the AGW
alarmists. While they dismiss the climate models, they seem to
believe all the predictions from them. Perhaps this is an example
of the model misuse they complain about. Anyway, AGW alarmists
need not avoid the book for fear of weakening their faith. Skeptics
need not avoid the book since most of it deals with other issues.
I heard of the book from a very skeptical web site.

some of the attacks in the book seem to go beyond attacking the
ideas in some of the models, and attack those that do not agree
with their evaluation.

This is not a technical read. The only equations are in the appendix,
and they are provided only to show why they are wrong. Even
chemicals are explained by their properties, rather than being
blamed for their existence. This is an easy read, but Aynsley J.
Kellow's "Science and Public Policy: The Virtuous Corruption of
Virtual Environmental Science" is much better.

The authors of this book do not provide useful criticisms of environmental modeling and only highlight their own ignorance. While models have been used problematically (some examples are aptly employed by the authors), this should not deter us from using models to seek insights into environmental systems. The authors essentially point out that that non-scientists who employ the models are uniformed and use them inappropriately. If they were to spend some time learning a little more than what is provided in an introductory modeling course they would be better equipped to argue their point, however such knowledge would likely change their opinion. It is inappropriate and little more than a slur tactic to criticize methods that the authors clearly do not understand.
Note that no actual quantitative scientists are consulted for this book, however MIchael Crichton and C.S. Lewis make the cut.
Please do not read this book if you're interested in modeling, but don't know much about it. A nice alternative is David Quammen's 'Song of the Dodo', which describes how some simple models have been useful in solving ecological problems. Its a much more compelling story and is actually well written.