Uneducated Guesses: Using Evidence to Uncover Misguided Education Policies [Hardcover]

Howard Wainer (Author)

Book Description

Publication Date: August 8, 2011 | ISBN-10: 0691149283 | ISBN-13: 978-0691149288 | Edition: 1st ed., 1st Ptg

Uneducated Guesses challenges everything our policymakers thought they knew about education and education reform, from how to close the achievement gap in public schools to admission standards for top universities. In this explosive book, Howard Wainer uses statistical evidence to show why some of the most widely held beliefs in education today--and the policies that have resulted--are wrong. He shows why colleges that make the SAT optional for applicants end up with underperforming students and inflated national rankings, and why the push to substitute achievement tests for aptitude tests makes no sense. Wainer challenges the thinking behind the enormous rise of advanced placement courses in high schools, and demonstrates why assessing teachers based on how well their students perform on tests--a central pillar of recent education reforms--is woefully misguided. He explains why college rankings are often lacking in hard evidence, why essay questions on tests disadvantage women, why the most grievous errors in education testing are not made by testing organizations--and much more.

No one concerned about seeing our children achieve their full potential can afford to ignore this book. With forceful storytelling, wry insight, and a wealth of real-world examples, Uneducated Guesses exposes today's educational policies to the light of empirical evidence, and offers solutions for fairer and more viable future policies.

Editorial Reviews

Review

[T]hought-provoking. . . . He questions the anecdotal and statistical evidence that underpins many of today's education policies and reform efforts. (Library Journal )

An absolutely absorbing book. Feels like a must for politicians, reformers, educators--math educators in particular. (Cut the Knot Insights )

Renowned statistician and research scientist Howard Wainer applies the tools of his trade to answer a question that affects every American: What is wrong with our education system? . . . Wainer pokes holes in almost every aspect of conventional education policy--college rankings, admissions, aptitude tests--including a scathing critique of No Child Left Behind. (Bruce Walsh Metro )

[Wainer's] overall message rings clear and true for much more than assessment: Policy that is formed without full analysis of the breadth of data available on a topic is policy that will fail. (Laurent Rigal Education Gadfly )

Tired of yelling at the TV when he saw news accounts of policy changes based on flawed evidence, Wainer uses his book to present evidence to help assess 11 such trends, including the entrance-exam-optional policies in many colleges and teacher evaluations based on student performance. . . . Wainer applies more than statistical evidence to education policy; he also brings common sense to bear. (Maureen Downey Atlanta Journal-Constitution )

With its timely reminder that high stakes decisions often rely on anecdotes, laden with emotion, and that 'the plural of anecdote is not data,' Uneducated Guesses ought to be read by anyone who is concerned about the weaknesses (and wrong-headed assumptions) in current educational policies. (Glenn C. Altschuler Tulsa World )

I would recommend this book to anyone with an interest in testing, especially for college admissions or advanced placement. . . . Wainer is a gifted writer with a notable talent for analyzing and presenting data. (Bill Satzer MAA Reviews )

The book provides a model for the development of rational public education policies, something that America needs desperately. (Robert A. Bligh Education Review )

Educators and education policymakers interested in helping students realize their potential will benefit from reading Wainer's book because the implications reach beyond postsecondary school instruction. Teachers and administrators at all levels can follow the logic of Wainer's ideas as they seek to use evidenced-based pedagogical strategies in their classrooms. (Denise G. Brassell Mathematics Teacher )

From the Inside Flap

"Uneducated Guesses is an insider's look at using test scores to make high stakes decisions in education. In this rigorous, refreshing rebuttal of conventional thinking, Wainer argues that in the world of education policy, we all would be better served by examining the evidence that demonstrates that our ideas will improve the systems we're trying to transform."--Dennis Van Roekel, president, National Education Association

"With his usual verve, flair, and disdain for pious nonsense, Howard Wainer offers a refreshingly fact-based view of a complex problem: the use of tests in educational selection and evaluation. A must-read for anyone involved in these issues and a fun read for anyone who wishes to be educated and entertained at the same time."--Daniel Kahneman, Nobel Laureate in Economics, Princeton University

"Howard Wainer's account of a selection of important scientific issues arising from educational testing is lucid, wise, and entertaining, and should be required reading for anyone interested in improving educational policy."--Stephen M. Stigler, University of Chicago

"Uneducated Guesses is a must-read for enthusiasts of evidence-based decision making and for those who make public policy decisions without consulting the evidence. The former will be sobered by a real and random world that may not match their theoretical models. The latter will be surprised to learn from past research the power and limits of public policy decisions. Wainer lays it all out in engaging and accessible prose and numbers."--Arthur E. Wise, president emeritus, National Council for Accreditation of Teacher Education

"Uneducated Guesses is a compelling, entertaining, and provocative book that elucidates some of the subtle and important issues of educational policy. In typical Wainer fashion, the graphics nicely complement and illuminate the text and tables, and I really enjoyed the variety of examples used in the book. I learned a lot about calibration, examinee choice, the history of testing, triathlon optimization, and the health of the professional football betting industry."--Nicholas Horton, Smith College

"It is always a pleasure to read Wainer's work, and this book was certainly no exception. His carefully chosen and extremely interesting examples, his conversational tone, and his great sense of humor lead to a work that is hard to put down. Uneducated Guesses is a fabulous book, and one of great significance."--Karl W. Broman, University of Wisconsin-Madison

See all Editorial Reviews

Product Details

• Hardcover: 200 pages
• Publisher: Princeton University Press; 1st ed., 1st Ptg edition (August 8, 2011)
• Language: English
• ISBN-10: 0691149283
• ISBN-13: 978-0691149288
• Product Dimensions: 6 x 0.8 x 9 inches
• Shipping Weight: 12.8 ounces (View shipping rates and policies)
• Average Customer Review: 3.7 out of 5 stars  See all reviews (7 customer reviews)
• Amazon Best Sellers Rank: #774,940 in Books (See Top 100 in Books)

More About the Author

Dr. Wainer received his Ph. D. from Princeton University in 1968. After serving on the faculty of the University of Chicago, a period at the Bureau of Social Science Research during the Carter Administration, and 21 years as Principal Research Scientist in the Research Statistics Group at Educational Testing Service, he is now Distinguished Research Scientist at the National Board of Medical Examiners and Professor (adjunct) of Statistics at the Wharton School of the University of Pennsylvania.

Dr. Wainer has a long-standing interest in the use of graphical methods for data analysis and communication, robust statistical methodology, and the development and application of generalizations of item response theory. His work on testlet response theory has combined all three. His book , Uneducated Guesses, (Princeton University Press) was published in September 2011. His latest book, A Statistical Guide for the Ethically Perplexed (Chapman & Hall), appeared in August, 2012. His next, Medical Illuminations (Oxford University Press) is due out in the fall of 2013.

Dr. Wainer was elected a Fellow in the American Statistical Association in 1985 and a Fellow of the American Educational Research Association in 2009. He was awarded the Educational Testing Service's Senior Scientist Award in 1990 and selected for the Lady Davis Prize and was named the Schonbrun Visiting Professor at the Hebrew University in 1992. He received the 2006 National Council on Measurement in Education Award for Scientific Contribution to a Field of Educational Measurement for his development of Testlet Response Theory and given NCME's career achievement award in 2007, and he received the Samuel J. Messick Award for Distinguished Scientific Contributions Award from Division 5 of the American Psychological Association in 2009 and was included in Who's Who in America, 2009 - 2012 and Who's Who in the World, 2010-2011.

He was on the editorial board of Psychological Methods and the editor of the Journal of Educational and Behavioral Statistics from 2002 until 2004 and is a former Associate Editor of the Journal of the American Statistical Association, and Applied Psychological Measurement as well as a former Treasurer of the Psychometric Society. Since 1990 he has written a popular column on data visualization in the statistics magazine Chance.

Most Helpful Customer Reviews

7 of 7 people found the following review helpful

4.0 out of 5 stars An enjoyable read December 30, 2011

By Gary Echternacht

Format:Hardcover|Amazon Verified Purchase

I very much enjoyed reading this book. Essentially the book is a collection of Mr. Wainer's technical papers which have been rewritten and packaged for a more general audience. These all related to current movements in testing and education. One does not have to read the book sequentially chapter by chapter, but can pick from among the various chapters.

Though now retired, I spent all of my professional life in the testing business. I was lucky in that I was able to do all the major functions in the business--research, test development, statistical analysis, operations, management, and marketing--which gives me a little different perception about each of those functions. I can't say how well someone who isn't in the business will react to all of the chapters, but I will say that someone who is in the business will take away something new in reading the book. For me, that was the discussion of choice in testing. Didn't know that such ever existed or that it has had a long history.

My sense of the education world is that it is dominated by fads and politics. At least that has been the case since the late 50s-early 60s when I first remember new science curricula making their way into my high school classes. Sputnik was the spark creating that. In that sense I think it is great that people like Mr. Wainer take a step back, do some follow-up thinking and relevant data collection (sorry but I still prefer the use of the word data over the use of the word evidence) to see what is happening as we implement the movements of the times. We need more people doing the same thing.

I too follow Mr. Wainer's call for "constant experimentation, in which small changes are made to the process," and then assessing the change in light of the process outcomes. Though that sounds simple and direct, I can assure you that it isn't. Let me give you an example from Chapter one of Mr. Wainer's book. In that chapter he discusses Bowdoin College's policy of making the SAT optional for applying. He shows that applicants not submitting SAT scores and who are accepted to Bowdoin that they both score low on the SAT and have lower first year grades than those who do submit scores and are accepted. This finding is by no means surprising to someone familiar with the SAT. But is it important or even relevant to the process of admitting Bowdoin students? People often view the admissions process as something like a machine with inputs of things like high school grades, SAT scores, and things like extra curricular activities or demonstration of some particular talent and an output of something like an ordered list of desirable individuals. Put two applicants with identical inputs into the machine and they should come out with the same place in the ordered output list. Admissions is not like that, however. Selective schools like Bowdoin generally look for an entering class that has a certain look, a look that is not just about freshman grades. Admissions is messy. Several audiences have to be served. Sometimes an admissions officer just wants to make sure that an individual, say one with a special talent, has enough smarts to make it through the freshman year (i.e., not flunk out). Bowdoin needs to ask itself if it is satisfied with its incoming students. If it isn't, then they may or may not need to change their SAT option, but that depends on what those making that decision think. An no, I've never had a contact with Bowdoin.

I did read the Kindle version of the book and did find a lot of typos and it was hard to refer to the tables when I wanted to. Better to by the old fashioned book in my opinion. But do buy it and read it no matter the version.

7 of 7 people found the following review helpful

5.0 out of 5 stars Fascinating, Funny, and Authoritative September 23, 2011

By Joanne M. Friedman

Format:Kindle Edition|Amazon Verified Purchase

If you're wondering how we got into this educational morass, this is the book for you. If you're a teacher frustrated by the Education Initiatives that every President seems bent on launching, this is the book for you. If you're in Congress or running for President, this is definitely the book for you!

The editorial review says that statistician and renowned research scientist Wainer challenges "everything our policymakers thought they knew" about education reform. That's right to the point. "Thought they knew" is the biggest issue, and Dr. Wainer uses clear statistics and a witty, straight-forward writing style to shred the approach that most education policy has taken toward solving the huge questions that have always plagued the system. Is it truly possible to gauge a learner's future success with any certainty? Is it possible to gauge a teacher's ability based on educational outcomes in students? Can we ever hope to bridge the gap between teaching and learning? And how important is it for colleges, for instance, to be able to guess how a student will perform or for government officials to guess how much funding plays a part in learning?

Those are the questions (among others) Howard Wainer addresses. This is a hard book to put down, so be prepared. Teachers will be quoting from it for years to come.

3 of 3 people found the following review helpful

5.0 out of 5 stars Some very valuable help in identifying and fixing the problems in education December 28, 2011

By Charles Ashbacher HALL OF FAMETOP 500 REVIEWERVINE™ VOICE

Format:Hardcover

Despite many claims to the contrary, creating a "fair" grading or testing system is currently impossible. There are too many variables (unknowns) and the variability in the subjects far too wide for such a word to have any meaning. The best that can be hoped for is to be "fair enough" to avoid litigation or reduce the errors to those that can be recovered from.
Wainer is a superb research scientist in the area of testing research and he sets forward many flaws in the testing and ranking strategies currently in use. Many of the intractable problems in testing for ranking are summed up in the phrase he repeatedly used, "Was Mozart a better musician than Einstein was a better physicist?" On the surface, the question is meaningless, yet that is exactly what is being done, although not to those extremes, when students are ranked on the basis of their performance in different areas. A more realistic question when it comes to college admission is along the lines of "Is Jill a better flute player than John is a quarterback?"
Furthermore, in many if not most cases, the real elephants are the ones not in the room, in other words the data that was not used in the analysis or simply is not known. Wainer does a superb job in his case studies in explaining the data that was not used or known and why it is so important. He also demonstrates some of the easier ways to game the system so that the results are more favorable. One of the simpler ways is to do something so the high achievers do not take the pre-test (lowering the score) and then do something so that the low achievers do not take the post-test (raising the score). Doing this right can yield a very impressive measured level of improvement.
So many people think that the problems with education in America are due to one factor, for example one often hears the teacher's unions being blamed. However, as Wainer conclusively proves to anyone with a sense of the scientific method, designing quality tests for the masses that truly measure something is very hard and therefore there is no one specific solution to any problem in education.

Published in Journal of Recreational Mathematics, reprinted with permission.