Uncontrolled and over one million other books are available for Amazon Kindle. Learn more



or
Sign in to turn on 1-Click ordering
More Buying Choices
Have one to sell? Sell yours here
Start reading Uncontrolled on your Kindle in under a minute.

Don't have a Kindle? Get your Kindle here, or download a FREE Kindle Reading App.
Sorry, this item is not available in
Image not available for
Color:
Image not available

To view this video download Flash Player

 

Uncontrolled: The Surprising Payoff of Trial-and-Error for Business, Politics, and Society [Hardcover]

Jim Manzi
4.7 out of 5 stars  See all reviews (13 customer reviews)

List Price: $28.99
Price: $17.99 & FREE Shipping on orders over $25. Details
You Save: $11.00 (38%)
o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o o
In Stock.
Ships from and sold by Amazon.com. Gift-wrap available.
Want it Wednesday, May 29? Choose One-Day Shipping at checkout. Details

Formats

Amazon Price New from Used from
Kindle Edition $14.99  
Hardcover $17.99  
Shop the Money & Markets Store
Are you a finance, investing, economics or accounting professional? Find books, read blog posts, and discover new authors and thought-leaders in Money & Markets, a new home for finance industry professionals on Amazon.com. > Shop now

Book Description

May 1, 2012
How do we know which social and economic policies work, which should be continued, and which should be changed? Jim Manzi argues that throughout history, various methods have been attempted—except for controlled experimentation. Experiments provide the feedback loop that allows us, in certain limited ways, to identify error in our beliefs as a first step to correcting them. Over the course of the first half of the twentieth century, scientists invented a methodology for executing controlled experiments to evaluate certain kinds of proposed social interventions. This technique goes by many names in different contexts (randomized control trials, randomized field experiments, clinical trials, etc.). Over the past ten to twenty years this has been increasingly deployed in a wide variety of contexts, but it remains the red-haired step child of modern social science. This is starting to change, and this change should be encouraged and accelerated, even though the staggering complexity of human society creates severe limits to what social science could be realistically expected to achieve. Randomized trials have shown, for example, that work requirements for welfare recipients have succeeded like nothing else in encouraging employment, that charter school vouchers have been successful in increasing educational attainment for underprivileged children, and that community policing has worked to reduce crime, but also that programs like Head Start and Job Corps, which might be politically attractive, fail to attain their intended objectives. Business leaders can also use experiments to test decisions in a controlled, low-risk environment before investing precious resources in large-scale changes – the philosophy behind Manzi’s own successful software company.

In a powerful and masterfully-argued book, Manzi shows us how the methods of science can be applied to social and economic policy in order to ensure progress and prosperity.


Frequently Bought Together

Uncontrolled: The Surprising Payoff of Trial-and-Error for Business, Politics, and Society + The Signal and the Noise: Why So Many Predictions Fail — but Some Don't + Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die
Price for all three: $52.16

Buy the selected items together


Editorial Reviews

Review

David Brooks, New York Times
“[Manzi’s] tour through the history of government learning is sobering, suggesting there may be a growing policy gap. The world is changing fast, producing enormous benefits and problems. Our ability to understand these problems is slow. Social policies designed to address them usually fail and almost always produce limited results. Most problems have too many interlocking causes to be explicable through modeling. Still, things don’t have to be this bad. The first step to wisdom is admitting how little we know and constructing a trial-and-error process on the basis of our own ignorance. Inject controlled experiments throughout government. Feel your way forward. Fail less badly every day.”

Wall Street Journal
“[O]ffers much to digest…. Uncontrolled is at its most provocative…when Mr. Manzi considers the largely unmet potential of controlled experimentation to improve outcomes in social science and government policy…. A vigorous book, pulsing with ideas.”
 
Arnold Kling, National Review
“The ideas in this book are important…. This is a provocative book for people who are interested in how social science relates to public policy.”
 
Forbes
“One of Hayek’s “old truths” is that individual freedom is an indispensible means to both human flourishing and material progress and that it is threatened by misguided government bureaucracy. We are fortunate to have it restated extraordinarily well in today’s language in…Jim Manzi’s Uncontrolled…His observations offer genuinely original insights into longstanding political and social problems.”
 
Tyler Cowen, Marginal Revolution
“This is a truly stimulating book, about how methods of controlled experimentation will bring a new wave of business and social innovation.”
 
The American
“This book is one of the most powerful challenges to progressive political impulses I’ve read in a while.”
 
The New Republic
“In the first two thirds of his book, Manzi describes the historical development of the RFT [randomized field trial] and its philosophical basis, and includes a digression on the philosophy of science. The argument will be familiar to empiricists and philosophers, but it may interest a popular audience, and is well done and readable…. A more ambitious argument emerges in the last part of the book. Manzi argues that the RFT – or more precisely, the overall approach to empirical investigation that the RFT exemplifies – provides a way of thinking about public policy. This is the most imaginative and interesting part of Manzi’s book.”
 
Andrew Sullivan, The Daily Beast / The Dish
“It’s a fresh, dense and fascinating exploration of what the policy implications of a true ‘conservatism of doubt’ would mean. I hope it can jumpstart a conservative intellectual renaissance.”
 
Kirkus Reviews
“A thoroughly argued, powerful study based on principles independent of the author's own conservative-libertarian views.”
 
Library Journal
“If social scientists entrusted with informing policymakers utilize more experiments, Manzi argues, the policies they create will be more effective over the long term. Simply put, adopting a trial-and-error methodology can help businesses, policymakers, and society as a whole. Backed by numerous pertinent examples, Manzi’s arguments are convincing. Recommended for anyone interested in policymaking or in how businesses make decisions.”
 
Booklist
“This challenging book highlights the astounding advances in science and technology that have started to be used in social-program evaluations.”
 
Conor Friedersdorf, The Atlantic
“If Uncontrolled were merely a restatement of the need for epistemic humility among wonks and legislators, interest in it might be confined to the right. The book is of broader interest, and may turn out to be important, because its author makes a compelling argument for an ideologically neutral method for improving policy, one that left and right might both plausibly embrace, even as it challenges both sides to rethink some of their reflexes…. [Uncontrolled is] the rare political book that goes out of its way to raise the most powerful objections to its arguments and to point out the limits of the reform program that it recommends.”
 
Kenneth Silber, The Daily Beast
“Jim Manzi’s Uncontrolled is an intriguing investigation of the power, limits, and varieties of empirical knowledge…. [A] substantial part of Uncontrolled’s value is in its sharp thinking about how various disciplines seek reliable knowledge…. Uncontrolled offers useful advice for navigating a hard-to-know world.”
 
Arnold Kling, National Review
“The ideas in this book are important…. This is a provocative book for people who are interested in how social science relates to public policy.”

The American Conservative
“[A]s Jim Manzi persuasively argues in his insightful and well-written new book, Uncontrolled, humanity is terrible at foresight, and trial-and-error is the chief way humans develop reliable knowledge…. In Uncontrolled, Manzi provides an incisive and highly readable account of how trial-and-error experimentation in science and free markets lessens human ignorance, uproots bias, and produces progress.”

Steve Sailer, Taki’s Magazine
“In his impressive first book, Uncontrolled: The Surprising Payoff of Trial-and-Error for Business, Politics, and Society, entrepreneur/intellectual Jim Manzi has the makings of an airport best seller in the genre of Steven Levitt’s Freakonomics and Malcolm Gladwell’s Blink. Indeed, Uncontrolled is far more reliable than those two sometimes-dubious tomes…. Uncontrolled offers one of the most lucid and sensible historical overviews of the philosophy of science I’ve ever read.”

Reihan Salam, National Review Online
“[T]he most important book of 2012 (read it now so you can be ahead of the curve).”

David McKenzie, World Bank, Development Impact blog
“[I]nteresting reading…. The lesson of trial-and-error with thousands of relatively low-cost experiments designed to make marginal improvements is one that could be useful in many government bureaucracies (and indeed in our own bureaucracy).”

Gary Gutting, The New York Times, Opinionator, The Stone
“[Jim Manzi] offers a careful and informed survey of the problems of research in the social sciences.”

About the Author

Jim Manzi is the founder and chairman of Applied Predictive Technologies (APT), an applied artificial intelligence software company. Prior to that he was a vice-president at Mercer Management Consulting. He is currently a contributing editor at National Review, where he writes about science, technology, business and economics, a senior fellow at the Manhattan Institute, and serves on a number of other corporate and non-profit boards. He has also written articles for a variety of political publications including the New York Post, the Weekly Standard, the Atlantic, and Slate. His work is regularly covered widely in the blogosphere, and his articles on why Republicans should acknowledge global warming and “Keeping America’s Edge” have become much-debated “must reads.” He lives in Paris.

Product Details

  • Hardcover: 320 pages
  • Publisher: Basic Books (May 1, 2012)
  • Language: English
  • ISBN-10: 046502324X
  • ISBN-13: 978-0465023240
  • Product Dimensions: 6.4 x 1.2 x 9.3 inches
  • Shipping Weight: 1.1 pounds (View shipping rates and policies)
  • Average Customer Review: 4.7 out of 5 stars  See all reviews (13 customer reviews)
  • Amazon Best Sellers Rank: #55,431 in Books (See Top 100 in Books)

More About the Author

Discover books, learn about writers, read author blogs, and more.

Customer Reviews

4.7 out of 5 stars
(13)
4.7 out of 5 stars
3 star
0
2 star
0
1 star
0
Most Helpful Customer Reviews
13 of 14 people found the following review helpful
Format:Hardcover
This ambitious book takes an interdisciplinary approach to discuss: (1) the scientific method; (2) why the scientific method cannot be mindlessly applied to any discipline or subject matter; (3) how the scientific method has been frequently invoked, but not effectively applied in the social sciences; (4) the limits of applying the scientific method to develop, test and improve business practices and strategies; (5) why the development and testing of public policy cannot be done within the constraints of the scientific method; and (6) how the inability to apply the scientific method to various subjects does not preclude using non-experimental methods of analysis to study, develop, and test business practices and public policies.

The author's discussion and analysis reflect an intriguing mix of: (a) scepticism about the general applicability of scientific methods to subjects beyond traditional sciences; (b) enthusiasm for encouraging the trial-and-error use of non-experimental methods to tackle business, political, and social issues and problems not susceptible to scientific methods; (c) apparent ambivalence about how to promote a willingness to experiment, engage in trial-and-error projects, and take risks, but still maintain a realistic and humble attitude about the limits of various non-experimental methods; and (d) cautious optimism about the ability of businesses, government officials, and the general public to improve their decision-making and goal-setting efforts.

The author makes a strong and very credible presentation about the limits of the scientific method, and the limited applicability of the scientific method to matters beyond traditional science. Furthermore, the author does a good job of balancing enthusiasm for the potential benefits of trial-and-error (nonscientific) experimentation with humility about the limits of various techniques to deal with complex business, political, and public policy issues. The author's discussion of political philosophy and political science concepts in the last part of the book is interesting, but not as convincing as his technical discussion and analysis in the earlier portions of the book. But, even though I found some of the author's contentions and arguments to be not persuasive, I consider his overall presentation to be serious, insightful, thought-provoking, and worth serious consideration.

This book is too technical for casual reading. Readers will be better able to understand and follow the author's contentions, arguments, conclusions, and proposals if they have some knowledge or experience with the scientific method, social science research techniques, business research, and political philosophy.

Any reader interested in the difficulties associated with trying to apply the scientific method to subjects beyond traditional sciences should consider also taking a look at Jerome Kagan, Psychology's Ghosts: The Crisis in the Profession and the Way Back (Yale University Press, 2012).
Comment | 
Was this review helpful to you?
23 of 30 people found the following review helpful
4.0 out of 5 stars An Excellent Start - May 8, 2012
Format:Hardcover
Author Manzi tells us that too often we allow policies to be guided by inflexible ideology. His alternatives include software that incorporates the experimental method for guiding business decision-making, and now this book that argues the same methods could be applied to important social issues.

Much of Manzi's material within the book is taken up explaining why simply using historical data, surveys, etc. is inadequate at best, due to the substantial and unquantified impact of factors not included within the 'model.' Similarly, regarding multiple linear regression analyses, the seemingly favorite tool of everyone with access to a computer and SPSS software. (Added problems with regression analysis - the need to incorporate non-linear variables such as binary zero-one to account for the presence/absence of a specific attribute or policy; interactions, and factors better represented as exponential variables, etc.) Thus, Manzi's rationale for instead using controlled experiments.

He cites Capitol One (credit-card firm that reportedly conducts 60,000 experiments/year), Gary Loveman's management at Harrah's Entertainment, and Google's sophisticated programs for evaluating alternative ad wording, etc. Unfortunately, he failed to elaborate on how any of those three leaders have utilized randomized experimentation. 'Uncontrolled' would be stronger if he'd also pointed out that there's a danger in these approaches - simply that the firm/government entity becomes overly focused on incremental improvement instead of breakthroughs. Had Henry Ford only followed these approaches, the best he could have accomplished would have been to breed a faster horse; similarly, Toyota never would have developed its revolutionary Toyota Production System that, unfortunately, helped decimate Detroit while simultaneously and significantly improving quality, response times, and costs.

Manzi's major recommendation is that government mandate randomized experiments to evaluate social policy proposals - especially when waiver approvals are being considered. He envisions as many as 10,000 such experiments/year. An excellent suggestion! Another - that we prioritize science and technology. As for treating immigration as a recruiting opportunity - his elaboration seemed to be simply a ruse for legalizing illegal immigration, a major contributor to today's high unemployment rates.

Good as Manzi's book is, he's still guilty of significantly overselling his positions. For example, his writing makes it abundantly clear that he's on the conservative end of the spectrum - that shouldn't even be mentioned or discernible in a book that must be seen as non-partisan if it is to be adopted to social science use. Secondly, he too quickly dismisses the potential use of thousands of non-random experiments, especially in education. Objective summaries in those areas offer immediate and credible support for pulling the plug on the belief that more money improves education outcomes. More importantly, the availability of such data for at least 50-some years underscores an important problem with even Manzi's ideal of using randomized data - the entrenched simply ignore and/or denigrate the findings. On the conservative side of education, Manzi further errs when he drops in an plug for vouchers - at best, quality experiments and analyses on this topic have found little if any benefit to doing so.

Still another area replete with quasi-experimental data - American health care. We lead the world by far in expenditures (18% GDP, vs. #2 at 12%), but lag in important preventable patient outcomes. Manzi, I presume, would prescribe multiple doses of experimentation to find out why. This is not needed - the differences between the U.S. and other nations are already obvious. Every other developed nation utilizes strong government regulation to control providers' ability to take advantage of the strong inelasticity of demand within that sector. (Same problem exists in education - also largely uncontrolled in America.) On the other hand, when it comes to evaluating treatment efficacy, randomized clinical trials similar to Manzi's thinking are the gold standard for practitioner guidance, as well as defanging demagoguery ala 'Death Panel' rhetoric.

One more point - since Manzi referenced Premier Deng's aphorism about black cats/white cats in 'Uncontrolled,' I'd like to cite another early lesson from Deng. Deng frequently encountered fierce resistance to proposals for major economic reform. Rather than remain stalemated or risk violent confrontation, he proposed, like Manzi, experiments. Such an approach would limit any potential damage, and could be isolated to relatively unpopulated boundary areas with existing local champions. Though these experiments weren't even on the same scale of sophistication as Mr. Manzi's used to and proposes, the improvements were obvious and fast-coming. Six 'Special Economic Zones' were approved in 1980 to conduct major, but non-random, experiments that greatly transcended Maoist thought. My point - further evidence that complex analyses such as Mr. Manzi champions aren't always needed. The keys in this success story were pragmatism, reliance on data, and physical isolation.

Bottom-Line: Some good points, but needing more examples and flexibility, along with less conservative bias.
Was this review helpful to you?
1 of 1 people found the following review helpful
Format:Hardcover|Amazon Verified Purchase
This book is difficult to rate. It is well-written and well-argued, with a lot of useful and interesting material. But it also has fatal gaps. My recommendation is to read the book, but only in combination with three related books: Trust in Numbers, The Politics of Large Numbers and Exploratory Data Analysis. On its own, this book is likely to mislead more than inform.

The central contention of the author is that randomized, double-blind, controlled experiments are the "gold standard" to establish validity. He goes as far as to claim that controlled experiments are necessary for scientific progress, which is patently absurd. But my criticism is not that I disagree, it's that the book fails to mention the reason people do these kinds of experiments.

For example, suppose we want to know if a drug helps lower back pain. We might select 100 healthy patients with severe, persistent lower back pain that has no discernible physical cause. We give half of them 100 mg of the drug daily for a week, the other half get an identical-looking pill with no active ingredients. Neither doctor nor patient knows who is getting the drug. We then ask the patients whether their back improved or not.

Real experiments will be more complicated. We may stratify the sample, test different dosages and regimes, record other data (certainly we will at least record reported side effects) and so forth. But the idea is still to throw away almost all the data, only adding back the stuff we think is important, and then to torture the remaining data to force it to give a definitive answer to a question that may not have one, or that the data may not know.

In contrast, you might treat the 100 patients one at a time, giving the treatment you thought was best, and recording everything about them before, during and after the trial. Blinding, control and randomization are still important. You will enter a treatment, but the computer will sometimes randomly change it, sometimes telling you, sometimes not, sometimes telling the patient, sometimes not. This is the only way to disentangle the effect of your diagnosis from the actual drug administered, and the effect of the drug from what you think and what the patient thinks. But we use the three techniques for the opposite reason as the standard protocol: to increase the dimensionality of our data rather than to reduce it.

The results may be analyzed using exploratory data analysis, letting the data talk to you rather than forcing them into pre-selected simple statistical tests. Or you could use a search algorithm, like the genetic algorithms the author likes. You start will thousands of hypotheses and recombine them randomly, selecting for the ones that explain the data best and are simplest. At the end you will have a weighted population of explanations, whose aggregated predictions give you a reasonable range of answers to what you really want to know, should I take this drug for my back pain? This is continuous learning, not a strict separation between experiment and application. There are no clear, simple answers, just increasingly precise and accurate subjective probability ranges.

A randomized, double-blind, controlled experiment is used when the goal is to get a group of people to agree on a simple action rather than to learn the truth. It is an organizational tool, not a scientific one. The author quotes Francis Bacon for indirect support of the idea, but Bacon was writing at a time when science was under attack by scholasticism and religious dogmatism. The book skips about 350 years to get to Ronald Fisher and randomized field trials. For most of this period, science had no need to prove itself to outsiders, there was rapid progress without significant public opposition. Only when scientists started demanding control over public policy and requesting huge government subsidies was there a need for the kind of experiments the author praises.

The people who came up with these experimental designs had lots of other bad ideas, such as non-expert witnesses, like judges or generals, for experiments. Their movement was popular with fascists. The fact that Nazis liked the ideas doesn't mean they were bad ideas, but it should give some pause. The author mentions Ronald Fisher's dismissal of the claims that smoking is bad for health (but not the money he was paid to argue for the case); he omits Fisher's support for institutionalized racism and eugenics. He also omits John Tukey's famous attack on the Kinsey Report that did so much to bring rationality to public discussion of sexuality, the claim that a random sample of 3 was better than Kinsey's detailed interviews with 300 non-randomly selected people (I recommended a book Tukey wrote much later in life above, when I knew him in the 1970s his opinions had nearly reversed from the 1950s).

The reason this book is five-stars despite the essential omissions, is the core of the argument is based on the author's extensive personal experience. He is really arguing for experiments, skepticism, trial-and-error, limited expectations, controls, blinding, randomization, quantification, careful record-keeping and common sense; he just somehow got it tangled up with a bad theory. This is valuable material you won't find other places, certainly not in as clear and lively form. His recommendations form the same mess of incisive sense mixed with tangled reversals of sense.

If you think for yourself, you will learn a lot from this book. If you don't, you should stay away.
Comment | 
Was this review helpful to you?
Most Recent Customer Reviews
5.0 out of 5 stars Good Stuff
I don't agree with everything the author says, but he has thought and researched very carefully about the scientific process in the social sciences and its flaws. Read more
Published 16 days ago by derwin
5.0 out of 5 stars Using Experiments in Social Science and Public Policy
Jim Manzi tries to make a case for the use of randomized field trials as a basis for public policy making. His book starts with a good introduction to the scientific method. Read more
Published 2 months ago by ivybeans
5.0 out of 5 stars Incredible book
This book covers everything from statistics to philosophy of science to public policy and does it all in an easy to understand, well organized, and well thought-out presentation. Read more
Published 6 months ago by reviewer
5.0 out of 5 stars Before launching expensive large-scale social programs, experiment...
In the social sphere, all governments can do is transfer wealth. The underlying question is whether a particular transfer of wealth is a good idea. Read more
Published 8 months ago by Harry D. Lewis
4.0 out of 5 stars Critical Things Most of our Leaders Don't Know or Don't Care About and...
Numerous references for the author's company that came across as advertising which was a negative for me. Read more
Published 8 months ago by Jeff Bennett
5.0 out of 5 stars Experimental methods applied to society
Manzi applies experimentation to three areas: business, society and political science. He starts with a well considered history of the scientific method going back to Roger Bacon. Read more
Published 8 months ago by Gderf
5.0 out of 5 stars Fantastic
I consider myself a conservative Republican, and Jim Manzi is quickly becoming one of my favorite right-of-center thinkers. Read more
Published 8 months ago by BSmith
4.0 out of 5 stars Great Idea, dense reading
The topic is probably my favorite of all topics to write about: The failure of planning to solve problems. His understanding of the topic is masterful. Read more
Published 9 months ago by Aretae
5.0 out of 5 stars The "missing link"!
Jim Manzi explains natural science in a way that can be understood by intelligent non-scientists. Moreover, he recommends "more freedom" to experiment in social science projects... Read more
Published 11 months ago by Jay Hanson
5.0 out of 5 stars Lies, all lies
What a fantastic reminder this book provides of the power of the scientific method. It also points out how common knowledge from "studies" is full of holes and assumptions that... Read more
Published 12 months ago by Michael N. Gualtieri
Search Customer Reviews
Only search this product's reviews

What Other Items Do Customers Buy After Viewing This Item?


Forums

There are no discussions about this product yet.
Be the first to discuss this product with the community.
Start a new discussion
Topic:
First post:
Prompts for sign-in
 



So You'd Like to...


Create a guide


Look for Similar Items by Category