Customer Reviews

Designing Social Inquiry: Scientific Inference in Qualitative Research

5 star:		(1)
4 star:		(5)
3 star:		(1)
2 star:		(1)
1 star:		(3)

 

 

This is a response to reviewers who think this is a stat book. This book is not meant to serve as a stats textbook (if you want one there are plenty of good ones written by statisticians and econometricians). This book is designed to serve as a guide to research design in social science in terms of developing a question, following systematic research procedures and measurement while using qualitative research methods. In that regard it does not do a great job as they are stuck up with applying simplistic statistical techniques (predominantly regression analysis)to qualitative methodology. As a result the work ends up appearing weak to both the statistically inclined (including myself) and those who use predominantly qualitative methods. Arguably the biggest problems with this work is in their treatment of constant dependent variable designs. This arises from their notion of a "causal effect" that is quite different from what qualitative researchers might see as causality. In statistical terms their notion is correct but when we move towards a qualitative interpretation of the same the concept becomes problematic primarily because it is difficult to discern the appropriate differentation between values of the dependent variable in qualitative work.

Nonetheless, this book should be treated on its own terms for attempting to synthesize quantiative and qualitative research methods. This book started a controversy that continues till this day and did a great job in forcing people to actually think more deeply about their research design and methods.

If you want to study statistics or econometrics forget this book (choose what you want to know about....regression analysis, time-series models, bayesian models....your choice). If you want to study qualitative research well read this book but then read Brady and Collier 2002 and George and Bennett 2004. George and Bennett's work is arguably the best book on research design I have ever read.

Hands down, this is one of the best texts of qualitative methodology available for the political scientist. The ideas and arguments made in this volume are very pertinent to study creation. Moreover, King et al. are both willing and able to criticize one of the most common logical fallacies that we find in the literature: the misuse of inference. What my colleague from the Netherlands overlooks is the clear and oft-stated differentiation between correlation and how it applies to THEORIES OF CAUSATION. By not reading the text in a clear way, my colleague has also confused the issue of theory vs. hypothesis as well as the focus of the work on testing hypotheses derived from theories objectively. The mathematical notations used are SPECIFIED as only being applicable in the abstract. In fact, one does not need the math to understand the points made. Moreover, my colleague notes that there are some problems with categorization, despite the fact that King et al acknowledge that if you can't categorize it or find data on it, then you should change your hypotheses and try again. Quite honestly, I question whether or not this gentleman bothered to read the book. I don't see how the points made in this volume could be any clearer. I would recommend this book to anyone seeking an all encompassing approach to qualitative analysis. However, if you are a person that sees little or no value to testing theories or are very polarized in the qualitative vs. quantitative debate, then you are most likely better off reading a good novel than this book.

Especially in an election year, the careful and critical examination of public policy and statistical findings is essential. In "Designing Inquiry..." King, Verba and Keohane explain basic statistical and methodological concepts previously only understandable to those studying the advanced social sciences. Concepts such as "endogeneity" and other logical fallacies are explained in language that is easy for the layman to understand, and in enough detail to be a gem for experts in the social sciences. The book explains in simple detail concepts that could be used by anyone to fairly evaluate the results of any study, and does it in a way that anyone can understand. Given the fact that many studies are passed off as "scientific" by journalists, politicians, and special interest groups when, in reality, they are fundamentally flawed, this book offers anyone the opportunity to learn how to critically evaluate studies, and how to reject studies t! hat are often utilized more to fool the voting public and appeal to emotion rather than logic. This book is one of the most significant ones of the decade, and should be read by all wanting to critically participate in a world where the term "scientific study" is often used more as an attempt to convince people of flawed findings rather than logically grounded results.

Sean A. McKitrick

This book takes the basic logic of statistical inference and applies it to qualitative research design in political science. As several reviewers note, it is not a book on statistics, nor indeed does it pretend to be. However, it extends the logic of statistical research design into nonquantitative research.

That much it does very well. By thinking about how to test hypotheses and how to increase variation in a qualitative research design, it has been very influential. Most important, it has sparked extensive criticism, modifying and delineating its claims.

The book has some amusing flaws. Most of the examples come from the authors' colleagues and graduate students at Harvard, which suggests either that good research is not done by people without that connection or that the authors don't read anything written by people who don't have an office down the hall. The two non-quantitative coauthors have both done extensive qualitative research that demonstrably violates the advice given here--both before and after this book. This is evidence that the advice is hard to follow, that they have not read the book, or that good scholars take other factors into consideration when designing research.

The last hypothesis is in fact the right one. There are many factors that go into good research design, and positivistic hypothesis testing provides only a few. Even many of the examples they give are less appropriate than appears at first glance, addressing evidence that goes well beyond what this book's advice would be.

In short, don't rely on this as a bible. Don't believe its claims that all good research must meet these standards. Still, it's a good handbook for what it seeks to accomplish.

A lot of the other reviews give great insight into what this book is and isn't.
I simply want to say that this book is excellent as a guide to what an optimal qualitative reasearch design should be, if it is to be as valid and reliable as possible. Qualitative research seems like a "haven" for researchers that want to follow their "heart" or "feelings", and this book contends unscientific research in a way that surely offends many of these researchers. Not that feelings should be excluded, it's just that the design must be more than a subjective view presented as research, and this book will help, even if you don't agree with everything they say.

King, Keohane and Verba (KKV) present an argument for unifying qualitative and quantitative methodologies in political science under one overall rubric. However, the book falls short in several respects.

First, KKV have a tendency to repeat themselves. They say the same thing over and over in different ways. In other words, they replicate what they just said. Duplication is a big problem as well. (Get the hint yet?) More editing would have helped.

Second, not everything they say can be accepted at face value. They take seemingly innocent ideas and state definitive conclusions, without letting the reader know that what they're saying overturns years of epistemological research into the nature of scientific research. People who disagree with their perspective are brushed to the side.

Third, their attempt to unify qualitative and quantitative methods suffers from a bias to the quantitative side of things. They first explain ideas in quantitative or statistical terms, use a formal analysis, and then state something like, "the same method can be used in qualitative analysis" without always saying how, precisely, that should be accomplished. Sometimes, I grant, they provide qualitative examples, but those serve primarily as foils against which they base their hypothesis. Qualitative researchers quoted by them tend to get torn to shreds by the end.

This book is usually required reading in research design/ research methods courses because it's been the source of most disagreements in political science methodology over the 10 years since its publication. But if you don't have to read it for a class, don't bother.

Several reviewers have commented that this book basically applies quantitative and statistical reasoning to qualitative research, and that the authors make some major errors in doing so. I don't want to address their misunderstandings of statistics, which other reviewers have identified, but to point out that, contrary to those reviewers who think this book "is excellent as a guide to what an optimal qualitative research design should be," KK&V fundamentally misunderstand qualitative research, and attempt to force this into a quantitative framework that completely misses the actual logic and rigor of qualitative inquiry. Their book has, however, been quite influential and controversial, and has prompted several major rejoinders within political science that attempt to correct KK&V's misunderstandings (Henry Brady & David Collier, Rethinking Social Inquiry; Alexander George and Andrew Bennett, Case Studies and Theory Development in the Social Sciences; Gary Goertz & Jack Levy (Eds.), Explaining War and Peace: Case Studies and Necessary Condition Counterfactuals).

Understanding this debate requires noting a peculiarity of qualitative research in political science (often called "case study research" in this field): that qualitative researchers in political science often describe what they're doing in terms of "variables." In almost all other fields, qualitative researchers don't think of what they're doing in terms of variables, but in terms of events and processes. This difference is connected to two quite different ways of understanding causality: the "regularity" view (derived from David Hume's analysis of causality) that defines causality as simply the regular association of variables, and denies that there is anything beyond this (the standard view in philosophy for much of the 20th century), and a "realist," "process," or "causal/mechanical" approach, which has become prominent in philosophy more recently, that sees causality as the actual mechanisms and processes by which one event or phenomenon influences another. I don't have the space to go into this issue in the detail it requires (in philosophy, see Wesley Salmon, Causality and Explanation; Peter Manicas, A Realist Philosophy of Social Science: Explanation and Understanding; for the implications for qualitative research, see my paper "Causal Explanation, Qualitative Research, and Scientific Inquiry in Education," published in Educational Researcher 33(2), pp. 3-11, March 2004). The point is that imposing a "variable" or "regularity" understanding of causality on qualitative research completely misconceives the way in which most qualitative researchers think about what they're doing. For a more accurate understanding of qualitative research design, see my book Qualitative Research Design: An Interactive Approach, or Catherine Marshall & Gretchen Rossman, Designing Qualitative Research.

Joe Maxwell

the KKV (king, Keohane and Verba) is one the best tool for beginners in research for social, political, International Relations students or academics. it delivers advices and problems that researchers will come across at one moment of the research life. it also offers a good overview and critical analysis of what research is.
the authors make sure the heavy subject that is reasearch in social sciences is not too heavy to read. the books approaches the different methodologies that research will have to chose.
if not owned yet, it is in need to be buy and must belong the student private library.

KKV is the book behind the excellent paper by Gerardo L. Munck entitled Canons of Research Design in Qualitative Analysis [Studies in Comparative Literature, Vol. 33, No. 3 (Fall 1998), p. 18-45].

The book has the details although it is harder to read than Munck's excellent paper. This is a must read for all doing Qualitative Research.

Many books contain errors --usually typos or other small ones. The book by King and others, however, really takes the biscuit, by making fundamental malinterpretations of statistics and methodology. The book aims at proposing a research methodology for both qualitative and quantitative research. The result, however, you wouldn't want to show to your undergraduate students. The most annoying malinterpretations for a social scientist are probably the following. First, relying on Pearson's 1892 (!) monograph, King e.a. state that method is more important than content of science. Apparently, nothing more has been learned in the last 120 years. Second, correlation is malinterpreted as causation. This is something you can't take for serious, especially when it is proposed in the introductory chapters and heavily used in the rest of the book. Third, some assumptions are proposed that your data should meet. These assumptions only apply to some sense to the statistical 'ordinary least squares' (OLS), but even in stats or econometrics these assumptions can be by-passed by using other estimation techniques. Furthermore, how would you test homogeneity in your cross-section when your qualitative reseach focuses on, e.g., motives of actors, while you cannot categorise that variable? Fourth, decision rules are proposed for 'constructing causal theories'. King e.a. seem to unconsiously mislead the innocent reader by mixing the concepts of 'hypothesis' and 'theory'. Fifth, some more assumptions concerning data quality are heavily misleading, and at most apply to OLS, but can easily be solved by using other estimation techniques. Some last bloopers: 'A research desing that explains a lot with a lot is not very informative' (p. 123); this is nonsense as long as long as your degrees of freedom are sufficient, and you don't face multicollinearity; 'Abstract, unobserved concepts [...] can be a hindrance to empirical valuation [...] unless they can be defined in such a way that they can be observed and measured' (p.109). This statement completely ignores that research on latent variables as, e.g., done by Maddala or others in the 1980s and more recently.

In sum, if you wish to buy a book on methodology: buy a book on methodology, written by a methodologist, and not by some political scientists. If you wish to get some introduction into the field of statistics or econometrics, then buy ANY other book that has a title as 'introductory econometrics' or similar. In either case, don't spend your money on this one.