Uncontrolled

The Surprising Payoff of Trial-and-Error for Business, Politics, and Society

Jim Manzi (View Bio)
Hardcover: Basic Books, 2012.

Entrepreneur and political commentator Jim Manzi argues for a radical new approach to our most pressing economic and social problems, using the scientific method—and its controlled experiments and skeptical mindset—to test what works in business and government. How do we know which social and economic policies work, which should be continued, and which should be changed? Jim Manzi argues that various methods have been attempted—except for controlled experimentation. Experiments provide the feedback loop that allows us, in certain limited ways, to identify error in our beliefs as a first step to correcting them. This technique goes by many names in different contexts (randomized control trials, etc). Randomized trials have shown, for example, that work requirements for welfare recipients have succeeded like nothing else in encouraging employment, that charter school vouchers have been successful in increasing educational attainment for underprivileged children, and that community policing has worked to reduce crime, but also that programmes which might be politically attractive, can often fail to attain their intended objectives. Business leaders can also use experiments to test decisions in a controlled, low-risk environment before investing precious resources in large-scale changes—the philosophy behind Manzi's own successful software company. In a masterfully-argued book, Manzie shows us how the methods of science can be applied to social and economic policy in order to ensure progress and prosperity.

"A vigorous book, pulsing with ideas." — Trevor Butterworth, The Wall Street Journal

"Jim Manzi has spent his career helping businesses learn from experience—first at AT&T Laboratories, then as a consultant with Strategic Planning Associates and then as founder of Applied Predictive Technologies, a successful software firm. In his new book, Uncontrolled, Manzi notes that many experts tackle policy problems by creating big pattern-finding models and then running simulations to see how proposals will work…. The problem is that no model can capture enough of the world’s complexity to yield definitive conclusions or make nonobvious predictions. A lot depends on what assumptions you build into them…. What you really need to achieve sustained learning, Manzi argues, is controlled experiments. Try something out. Compare the results against a control group. Build up an information feedback loop. This is how businesses learn…. These randomized tests actually do vindicate or disprove theories…. Businesses conduct hundreds of thousands of randomized trials each year. Pharmaceutical companies conduct thousands more. But government? Hardly any…. Manzi wants to infuse government with a culture of experimentation. His tour through the history of government learning is sobering, suggesting there may be a growing policy gap. The world is changing fast, producing enormous benefits and problems. Our ability to understand these problems is slow. Social policies designed to address them usually fail and almost always produce limited results. Most problems have too many interlocking causes to be explicable through modeling. Still, things don’t have to be this bad. The first step to wisdom is admitting how little we know and constructing a trial-and-error process on the basis of our own ignorance. Inject controlled experiments throughout government. Feel your way forward. Fail less badly every day." — David Brooks, The New York Times, April 27, 2012

"A thoroughly argued, powerful study." — Kirkus Reviews

"The thesis of software entrepreneur Manzi's first book is that the application of controlled experiments in social and economic spheres can ultimately lead to improved decision making…. Backed by numerous pertinent examples, Manzi's arguments are convincing. Recommended for anyone interested in policymaking or in how businesses make decisions." — Library Journal

"In the first two thirds of his book, Manzi describes the historical development of the randomized field trial (RFT) and its philosophical basis, and includes a digression on the philosophy of science. The argument will be familiar to empiricists and philosophers, but it may interest a popular audience, and is well done and readable. Manzi argues that RFT should be used in universities, businesses, and government more often than it currently is…. A more ambitious argument emerges in the last part of the book. Manzi argues that the RFT—or more precisely, the overall approach to empirical investigation that the RFT exemplifies—provides a way of thinking about public policy. This is the most imaginative and interesting part of Manzi’s book." — Eric A. Posner, The New Republic

"Manzi argues that social scientists have developed very little knowledge that is valid and useful in fields such as criminology, education, and other policy areas. I think Manzi makes a profound, compelling, and insightful case, so that Uncontrolled is a valuable book." — Arnold Kling, Econlog