Customer Reviews

What's Wrong with the Rorschach? Science Confronts the Controversial Inkblot Test

5 star:		(7)
4 star:		(0)
3 star:		(0)
2 star:		(0)
1 star:		(6)

 

 

Readers of this book-length criticism of the Rorschach Test and, more pointedly, the Rorschach Comprehensive System (CS) developed by John Exner, may have been alerted by controversial articles appearing in professional journals, popular media outlets (e.g., New York Times Magazine, Scientific American) and symposia at professional meetings over the past several years. Wood, Nezworski, Lillienfeld, and Garb have developed a peculiar fixation on the Rorschach Test, adopting an unrelentingly skeptical posture that equates use of the test with fortune telling, claims of the paranormal, and junk science (what they refer to as "widespread quackery"; see also Lohr, Fowler, & Lillienfeld, 2002). The authors characterize Rorschach developers as "wizards" whose "tricks" were designed to defraud and mislead the gullible. They insinuate that Rorschach clinicians are members of a cult-like religious movement whose motivations are mercenary, or worse. A review of their output suggests that this fixation has come to characterize a significant portion of their scholarly interest. This overdetermined fixation on the Rorschach Test seems curious. One wonders why they haven't focused on widespread unsubstantiated assessment methods and practices that have profoundly negative effects, for example, polygraph or plethysmographic testing, efficacy of anger management, so-called attachment therapies, or diagnostic mammography.

What's Wrong with the Rorschach? is an informative critical biography and cultural history of the Rorschach Test's development, advent in the United States, dissemination during the glory days of the `40's and `50's, period of apparent decline during the '60s, and resurgence with the advent of the Comprehensive System. In contrast to most books on the Rorschach, the authors take a less than adoring view of Rorschach "heroes." What's Wrong with the Rorschach? is not exactly Rorschach hagiography. In fact, the authors debunk the cult of Rorschach "great men," and dispel many of the bombastic claims made about the test over the decades. They pick out certain of the Rorschach "great men" for particular lambasting, for example, Bruno Klopfer and John Exner. Marguerite Hertz, on the other hand, is lauded throughout, based on the rigor of her dogged insistence on grounding the test scientifically.

Much of this material is informative, some of it amusing. At many points, the author's arguments are incisive and critical. They point out a number of factors that well-informed Rorschach clinicians have been aware of for years. Some of their criticisms deserve serious attention from the Rorschach establishment, for example, the weak scientific database derived from Exner's "in house" research, Exner's disinclination over the years to include or recruit the work of researchers outside of his establishment, the problem of "R" (why the Rorschach community didn't adopt a fixed R a long time ago as did the Holzman Inkblot Test has long been a mystery to me), the mediocre performance of DEPI or some of the other CS indices, the recent "overpathologizing" debate, and the embarrassing issue of the duplication of a significant number of records in the so-called "normative data." (I have known John Exner since the late '70s when I took my first Rorschach workshop. Exner resisted calling these samples of outpatient, inpatient, depressed, etc., subjects "normative." He referred to them as "reference data"). To their credit, Wood, et al.'s criticisms of the Rorschach Test have stimulated a flood of Rorschach research in refereed journals examining both clinical and methodological issues. Overall, the author's effort to convince the informed reader of the Rorschach Test's fundamental bankruptcy fails given their selective review of the scientific evidence (MacCoun, 1998) and undisguised polemical intent. Despite the book's easygoing, somewhat avuncular tone, it becomes clear quite early on that a dispassionate assessment of the Rorschach Test is not the authors' intent. What's Wrong with the Rorschach? is popular, albeit well-written polemical screed, not a scientific treatise.

The authors thoughtfully included the undersigned as one of the "supporters and followers" of their work. They make one specific and two general errors that deserve comment. They take me (along with my former student, Greg Meyer, current editor of the Journal of Personality Assessment) to task on p. 232 and again on p. 266 for asserting that "Rorschach scoring reliability of .61 is "good" and that reliability of .74 is "excellent," disingenuously, perhaps, failing to remind the reader that standards of what constitutes adequate reliability depend on whether one is referring to an intra-class correlation or a kappa coefficient. It is generally accepted that kappa coefficients .61 and above do indeed reflect "good" reliability (Acklin, McDowell, Verschell, & Chan, 2000).

The second issue concerns the authors' failure, despite having the opportunity of an interested reader's attention for 324 pages, to give a good accounting of what the Rorschach Test is really about. They suggest the test's persistence and popularity has to do with wizardry, or perhaps chicanery, but realistically speaking, all of their attention in a growing list of publications does not yield a clue.

We have adopted the position over the years in both our theoretical and research endeavors that the Rorschach Test yields a sample of verbal behavior that embeds the respondent's stylistic dispositions and repertoire of ego states. This verbal material, in the form of a written protocol, may be coded by any number of pertinent coding schemes (We have proposed that computerized content or text analytic approaches may render the whole statistical debate moot; see Gottschalk, 1979). As such the material provides a deeply personal, that is, idiographic view of the person (Stricker & Gold, 1999).

On occasion, the authors give credit where credit is due, for example, the previously mentioned lauding of Marguerite Hertz, recognition of the Joseph Zubin (a thoughtful and well-informed critic of the Rorschach Test), mention of the Thought Disorder Index by Philip Holzman, or the work of Edward Aronow, who uses the Rorschach as a clinical technique not a psychometric instrument, and the strongly empirical work of Goldfried, Stricker, & Weiner (1971). Consistent with their selective use of the literature, however, they skate over the weighty scientific contributions of R. Holt and S. Blatt. There are excellent suggestions/recommendations concerning canons of Rorschach research and necessary foundational psychometric requirements for scientific respectability (Exhibit 9.2). They rightly criticize misuse of the test for purposes that it should not be used, for example, making determinations of vocational choice or confirmation of sexual abuse. They seem to identify the Rorschach Test as uniquely flawed among instruments commonly used in assessment psychology. Many of their criticism are relevant to any psychological test used inappropriately by a poorly trained clinician. They fail to place the Rorschach's capabilities into context with other assessment instruments (e.g., the MMPI), or the general weakness of "soft psychology" in general (Acklin, McDowell, & Ornduff, 1992; Cohen, 1962, 1988). They suggest that the Society of Personality Assessment and the Journal of Personality Assessment, two respected, long-standing organizations dedicated to psychological evaluation, are corrupt and self-serving. A third, serious weakness is most apparent in the way that the authors conclude their argument and, thus, the book. Chapter 12 ("Objection, Your Honor! Keeping the Rorschach Out of Court) and the Epilogue are notably weak, perhaps the weakest chapter in the book. The authors fail to take advantage of a decisive opportunity to make their argument that the Rorschach Test should be inadmissible in court by avoiding a thorough analysis of the Rorschach literature in relationship to current admissibility standards (Frye Test, Federal Rules of Evidence, Daubert and Kumho). This, of course, they could not do since theirs is not a dispassionate inquiry. The Epilogue suggests their ultimate befuddlement: How could or does the Rorschach Test maintain its endurance and clinical popularity? As clinicians persist in using the test despite the depredations of managed care and the criticisms of Wood, et al. and their ilk, there must be some other reason beyond fad, mendacity, or self-serving economic motivation. Here the authors appear clueless.

What's Wrong with the Rorschach? is not and does not purport to be a systematic analysis of the behavioral science foundations of the Rorschach test. Too bad. This has yet to be written. Because of this, despite an informative and generally well-written text, nicely packaged, the authors fail to make their case. It remains for the community of Rorschach researchers and clinicians to meaningfully address the valid critical points raised here in a way that establishes a firm empirical foundation for the Rorschach Test's continuing use. Although the authors call for a moratorium on the test, one of the more whimsical aspects of their criticisms, which even they admit is unlikely to occur, they should be satisfied if Rorschach Test users adopt a more critical and informed approach to all of their assessment endeavors.

References

Acklin, M.W., McDowell, C.J., & Orndoff, S. (1992). Statistical power and the Rorschach: 1975-1991. Journal of Personality Assessment, 59, 3 66-3 79.

Acklin, M. W., McDowell, C. J., Verschell, M.J., and Chan, D. (2000). Interobserver agreement, intraobserver reliability, and the Rorschach Comprehensive System. Journal of Personality Assessment, 74, 15-47.

Cohen, J. (1962). The statistical power of abnormal-social psychological research: A review. Journal of Abnormal and Social Psychology, 65, 145-153.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: LEA, Inc.

Gottschalk, L. A. (1979). Content analysis of verbal behavior: Further studies. Hillsdale, NJ: LEA, Inc.

Lohr, J.M., & Fowler, K.A., & Lillienfeld, S. O. (2002). The dissemination and promotion of pseudoscience in clinical psychology: The challenge to legitimate clinical science. The Clinical Psychologist, 55, 4-10).

McCoun, R. J. (1998). Biases in the interpretation and use of research results. Annual Review of Psychology, 1998, 49, 259-87.

Stricker, G., & Gold, J.S. (1999). The Rorschach: Toward a nomothetically based, idiographically applicable configurational model. Psychological Assessment, 11, 240-250.

Marvin W. Acklin, PhD, ABAP, ABPP
Honolulu, Hawaii
acklin@hawaii.edu

With the exception of a very accurate depiction of the history of the Rorschach, the book is a biased, poorly argued and narrow minded attack on a well respected performance-based measure of clinical assessment.

Truly the most laughable part of the text is an argument made by one of the authors about the lack of clinical utility in "reflection responses" as potentially indicative of narcissism, particularly when he argues that the index has no validity given that he, the accomplished, high functioning, respected, prolific author and professor that he is, actually obtained reflection responses in his own Rorschach protocol...HOW COULD THAT BE???!!! Well, it is not a predictive measure of narcissistic personality disorder, but of the tendency to self aggrandize and inflate one's own self worth, a coping strategy that many people rely upon that ultimately makes them interpersonally inaccessible. Nevertheless, rather than critiquing the imperfection of some of the scales, the authors flagrantly attack the Rorschach, and do so with a self-inflated perspective that theirs is the single, solitary correct attitude to adopt with regard to the Rorschach if you are a psychologist.

There are other contemporary "critical" investigations of Exner's Comprehensive System for the Rorschach (published in peer reviewed journals) that point out the system's assets and liabilities without advocating throwing out the baby with the bathwater! Seek those out not this text.

If you are looking for an unbiased review of the current Rorschach empirical literature, this book is NOT the place to look. That's what I was looking for when I read this book, and was quite disappointed. Yet, I hate to admit that I was initially taken in by the authors' emotional writing style. It was only after some perspective and looking up some of the research cited (and not cited) in this book that I realized the irony -- that is, the authors use the very method that they condemn to make their 'case.' In that sense, I would rate this book a '4 or 5' for anyone interested in critical analysis and persuasion (even if you're not interested in the Rorschach) as a wonderful example of how information can be misused, distorted, and employed for personal gain.
Here are some of the issues: First, as implied by the title of the book, its purpose is to evaluate the status of the Rorschach from a scientific perspective. Yet, this was only a VERY small portion of the book. That is where I'm most regretful that I actually bought the book. Furthermore, in their 'scientific review,' the authors frequently actually 'cite' correspondence from Internet discussion lists as their 'scientific data!' Ha! This is absurd. And they do so in the context of building up a 'case' in their 'scientific review' that there is some big conspiracy among these Rorschach researchers that the authors are letting the public in on. In fact, this not-so-concealed conspiracy theory comprises most of the theme of the book! One begins to wonder why the authors are so invested in passionately and condescendingly dissing this test and the test's researchers. Very poor form, authors.

Here are more examples:
1. In the first few pages of the book, the first author (Wood) begins by telling us some of his own Rorschach test results. And then he proceeds to expect us to take his word for it that any negative implications of these results are absurd, since he's such a well-adjusted guy (by his own report). Yet I found it funny that his very book seemed to support some of these interpretations; if they're not true, he should have been more careful to not display those characteristics in his book. Especially the interpretation that says he's egocentrically focused on himself. (does anyone else see the irony here?) That was actually humorous! I got a kick out of that.

2. If the reader is already lulled into sympathizing with this poor author and the invalidity of the test from these first few pages, they will be further inclined to believe the next section (and use it as verification of the author's conclusions in the preceding pages). The next few pages discuss a court case involving child sexual abuse; an abusing father was not identified and was allowed to have custody of the child. As you might guess from the authors' title of their book, this was the Rorschach's fault. Yet, this was a case where there were MANY lapses and errors on the part of *all* the professionals involved in the case (and hindsight is 20-20). In a court case (or any evaluation), any personal report of the people involved and any 'test' results can be misused, misinterpreted, ignored, and invalid. Yet the authors seem to think it is the fault of one test. This is a great section to show how emotional manipulation can lead one to lose their own critical abilities.

3. Throughout the book, the authors use emotional manipulation and single 'case studies' like these, as well as conclusions based on minor details of research and out-of-context posts from Internet discussion (!) groups. This is ironic and somewhat funny, since these are the very methods that the authors themselves most strongly critique!

4. However, their book is a psychologist's treat in examples of primitive defenses: Splitting, idealization, devaluation, projection. The authors are good and the Rorschach researchers are bad (splitting). There is even a chapter title that characterizes anyone who attempted to integrate the past Rorschach approaches as "Still Waiting for the Messiah" on the part of the psychologists interested in the Rorschach (idealization). Then the authors proceed to nitpick -- no, not exactly the Rorschach research but -- the Rorschach researchers themselves (!), making them sound like idiots and suspicious characters (devaluation and paranoia). Yet, if one actually READS the published articles they cite (especially by Gregory Meyer who they cite and frequently villainize), one finds a quite thoughtful and actually VERY scientific approach. Wow. The authors of the book "What's Wrong with the Rorschach" aren't even in the same league.

So, what's wrong with this book? Answering that question is the only reason to read it.

These authors are all scientists who should understand research, statistics, and scientific theory. How they could knowingly write a book as misleading as this challenges their objectivity (and ethics) as scientists. Suspicions are evident from the outset, when the first author uses his own Rorschach responses to demonstrate how fallible the test is. Given his familiarity with the Rorschach, he gives answers that suggest problems; however, the variables he chooses are ones easiest to malinger (e.g., single response items versus variables based on ratios or percentages). To use his informed protocol to challenge the test is ludicrous; a psychologist who has studied the Wechsler Adult Intelligence Scale would not give himself the WAIS to determine his IQ.

The authors then give a description of a Rorschach being used incorrectly by inept practitioners to discredit the instrument. If a cook burns a steak, it doesn't mean there was something wrong with the food.

Much of the book is devoted to the Rorschach as used when it was first developed a century ago. Practitioners often used questionable interpretation methods, particularly because they lacked the statistical methods to develop a valid scientific basis for the test. In the 1970s, John Exner tried to take the strongest factors of the numerous scoring systems and develop a scientifically-based system. Since that time, he has been working on the Comprehensive System, which is the only Rorschach method that has sufficient scientific backing. I will not address the aspects of their book that do not concern Exner, because everything else they write is for the sole purpose of making their articles book-length. [Surely they are aware of the difference between the old "blind interpretations" and the current strict methodology for administering and scoring the Rorschach, yet they equate the CS to everything that came before, which is then equated to fortune-telling, Tarot cards and tea leaves. This is such an obvious misrepresentation that to suggest they are connected should be considered a lie. A similarly misleading point is their boldly pronounced claim (in the form of a chapter title) that proponents of the Rorschach call it an "x-ray" of the mind; they later admit that current clinicians do not describe it that way]

Wood et al have published numerous articles attacking the Rorschach, so most of their arguments are reprints from--and have been repeatedly discredited in--scientific journals. Their failures to impose a moratorium on the Rorschach by the scientific community, who see through their shoddy work, have apparently led them to try to dupe the general public (through this book and articles in the New York Times, among other publications). They engage in many of the tactics they accuse Rorschachers of employing, such as an exclusive bias towards research that supports your viewpoint, personalizing the issue and abandoning professional discourse, and dismissing all research against your argument as biased. The flaws in their reasoning are immediately apparent, and reminiscent of political screeds (e.g., by Limbaugh, Coulter, Hannity, etc.) in their distortions, lies, and half-truths.

In short, here are some facts about the current state of the Rorschach:

1. When the CS is used, validity and reliability is acceptable and comparable to other accepted scientific tests. Some of the research in this area is negative (as Wood et al quickly point out), but the majority of it is supportive (as they neglect to mention). However, it is important to ensure that the CS is used for administration, scoring, and interpretation. This is true for any scientific test, but the Rorshach may be particularly susceptible to individualization.

2. The Rorschach arguably meets all the Daubert criteria that would qualify it for use in legal situations. However, it is not always appropriate to use in legal situations (The same is true of any test, of course).

3. The Rorschach is not attacked as much as these authors would like you to believe. Most of the negative publications against the Rorschach are the work of a very few determined authors, who also wrote this book.

4. The American Psychological Association's PAWG group determined that psychological testing--including the Rorschach--is beneficial for numerous reasons. Wood et al's dismissal of these findings as biased is an insult to the name of the esteemed researchers on the work group, and gives evidence to their own biases.

Wood et al argue that practioners who use the Rorschach have questionable ethics. I think the most questionable ethics are those employed by scientists who knowingly mislead the public by using obviously shoddy methodology, reasoning, and arguments.

- STRONGLY recommended to all observers of how science impacts culture; certainly required reading for all interested in how psychology influences society.
- "Rorschach test" has entered the language as an idiom for an ambiguous stimulus that prompts responses showing more about those who respond than about the stimulus itself. This book tells the fascinating story of how a parlor game became a psychological instrument that also influenced popular culture. Just as most of us know IQ = intelligence, we know Rorschach Test = how our biases control our perception. Or we think we know this. It's worth reading the book just to trace this history. But the book tells a more profoundly important story.
- Readers are likely to come away with a sober view of how psychologists' clients and those caught in the legal system, including innocent children pulled between divorcing parents, are at the mercy of those who evaluate them. The importance of what may seem like niggling concerns about the Rorschach then becomes clear in this context.
- In an evenhanded, careful, interesting manner Wood and colleagues give a simple, but non-condescending overview of the science involved in controversies about the Rorschach -- and there is definitely real science to be considered. One need not come to a technical understanding to appreciate the practical importance of the scientific controversies associated with the Rorschach.
- Rank this book with Stephen J. Gould's classic _The Mismeasure of Man_ in showing how we can injure ourselves and others by failing to heed scientific evidence that disconfirms our biases.
- I have been using the Rorschach in my professional practice for about 25 years. I was trained by Rorschach experts. I've been teaching psychologists in training to use the Rorschach for the past 15 years. For much of that time I have been a cautious defender of the inkblot test. Paraphrasing an old joke, I believed there was a pony buried in all the Rorschach's accreted manure. My doubts grew as students working under my supervision were unable to confirm Rorschach claims. This book shows that others shared my doubts: Most who have tried to test the Rorschach, including many true believers, have come away frustrated. But "Rorschachers" seem unable to see the problems and consumers have not had a fair chance to judge for themselves.
- Wood and colleagues have tilted the balance in my mind. I will no longer be using the Rorschach in psychological evaluations. Their book is now required reading for my graduate students.

What's Wrong with the Rorschach is clearly a piece of propaganda from authors who have consistently ignored all evidence that runs counter to their claims about this much-researched instrument. It is written for an uninformed audience to sway popular opinion of the Rorschach, as evidenced by one of the authors taking the Rorschach himself, then disputing the results. (An expert on the Rorschach cannot take the Rorschach because he would know exactly what his responses would indicate, and would be able to alter them accordingly.) If only the debate on the Rorschach's reliability and validity were based on actual sound science rather than dominated by two diametrically opposed camps waging propaganda wars!

This book is a waste of trees, not to mention a waste of intelligence.

Almost everyone has heard of the Rorschach test. It consists of a series of ten ink blots that are presented to the patient with a request to tell what he sees in them. The responses constitute, or so it is claimed, something of an x-ray of the respondent's total personality and character.

Why should such perceptions of ink blots give a such a powerful picture of mental life ? The reasons are said to lie in psychoanalytic theory, which hold that unconscious motivations, unique to an individual, govern our every perception. Moreover, it has been said, the Rorschach test "works," i.e. it can predict behavior, and it can tell the mentally sick from the healthy.

But, as these authors painstakingly demonstrate, there is amost no validity whatever in these claims. It is true that gross mental illness can generally be detected by this test, as, for instance, when a patient claims that he can very clearly see God conversing with Elvis Presley in Card I of the test. But then again, this kind of diagnosis surely needs no Rorschach. Beyond that, the test has failed each and every reasonable attempt at validation. Scientifically oriented psychologists have rejected the Rorschach a long time ago. Some clinical psychologists hold on to it, for reasons that are not -- to put this politely -- edifying.

To someone like me who has been exposed to Rorschach instruction in graduate school many years ago, this book gives truly fascinating information. The abysmal failure to validate the test, of course, is no news. But the details about the internal battles within clinical psychology give a rare insight into the foibles of apparently educated men. The authors are particularly good at analyzing why it is that for certain people the test, to this day, retains credibility. The answer here turns out to be strikingly similar to the reasons why fortune tellers can thrive in a presumably science-dominated world.

When I was an undergraduate psychology student a truly long time ago, I had a great teacher, Gardner Murphy, whose memory I cherish to this day. Upon reading this book, I took Gardner Murphy's old text from my shelf to see what he had to say about the Rorschach. I was shocked to find that he accepted the unfounded claims of the Rorschachers of his day, citing studies that were never replicated by anyone else. So it turns out that Murphy had feet of clay.

But Murphy wrote long before the devasting negative findings that Wood, et. al. cite in this book. So he had something of an excuse for his gullibility. No such excuse can be advanced for those who support the Rorschach today.

I think that this is an exceedingly important book, and I reward it five stars without qualm. Nevertheless I must mention that it could have benefitted from more than a bit of editing. It is inexcusably repetitious. Beyond that, the prose is needlessly rough. But these are minor complaints. The book is a jewel, a rough diamond to be sure, but a true jewel.

What's Wrong with the Rorschach is an amazing book. It provides background into the origins of the Rorschach and a cogent analysis of the scientific basis ( or lack thereof) of this widely used
test. Wood and his fellow authors skillfully describe the theoretical underpinnings of the Rorschach and then explain its subsequent development in simple, easy to understand language.
Despite the book's academic focus, it reads like a well written, tightly plotted whodunit and entertains as it informs. I predict that What's Wrong With the Rorschach will ignite a debate that
will have implications far beyond the test it critiques and will spark an rigorous examination of clinical practice in general. It is a must read for mental health professionals and will be of interest to general readers as well.

Dr. Wood and his colleagues have been publishing these same arguements regarding their reservations about the Rorschach Test for several years. Despite scholars' efforts to explain the authors' misunderstandings and misrepresentations of the instrument, its current use, and the research regarding the Rorschach's psychometric properties, these authors persist in repeating inaccuracies regarding its scientific properties.

This book is an embarressment to scientific experts who actually understand the research that these authors present with such misunderstanding.

Don't let the other reviews fool you - many psychologists, especially in the forensic area where the testing and conclusions advanced are subject to scrutiny, are abandoning the Rorschach. The authors have mixed the methods of their arguements, using anecdotes for accessiblity, but here are many, many formal studies behind their conclusions. I predict the Rorschach, in its present form, will be dead in less than 20 years. Nobody will want to pay for a test with such little proven validity and utility, and so subject to overpathologizing people.