Since I enjoyed the Manga Guide to Statistics, I guess the author achieved at least one objective of good teaching - keep the learner interested. The use of well thought out graphics and humorous examples are likely to encourage a learner to attend to the content.
Still, maintaining interest and good teaching, while related, are not identical. One can maintain interest in ways that detract from learning as well as in ways that enhance learning.
The tendency in this text to oversimplify (e.g., the discussion of what is and is not "measurable" at the beginning of the book, the underemphasis of the importance of random selection) are definite negatives. They will lead a learner with no background in the use of statistical procedures to mistaken conclusions about the meaning of measurements and the generalizability of findings.
In at least one case, the oversimplification proceeds to the point of presenting information that is wrong (i.e., the examples of alternative hypotheses on pp. 172-173). To be fair, there are many "gentle" statistics texts that, as does the Manga Guide to Statistics, present the notion that the alternative hypothesis is simply "not the null hypothesis."
Despite the popularity of this view, Neyman and Pearson (who developed statistical hypothesis testing theory 75 years ago) noted that the "not the null" formulation of the alternative hypothesis would lead to the acceptance of trivial effects as meaningful simply because they were "statistically significant."
The "not the null" formulation of the alternative hypothesis creates other problems.
For example, the null hypothesis on page 173, "The allowances of high school girls in Tokyo and Osaka are the same," has as its alternative, "The allowances of high school girls in Tokyo and Osaka are not the same." Stating the alternative hypothesis in this way does not permit an evaluation of the power of a statistical test (power refers to the probability that a test will detect a difference, change or relationship when it is present). As Neyman noted, since the test would have to detect an infinitesimal difference, the power would necessarily be infinitesimal as well.
Instead, an alternative hypothesis should specify a minimum effect, e.g., "The allowances of high school girls in Tokyo and Osaka differ by an average amount of at least Ą500." By specifying a minimum effect to be detected, we can find the probability that a statistical hypothesis test would detect a difference of at least Ą500 (the test's power).
Since I have to devote time to "unteaching" the "not the null" formulation of the alternative hypothesis, I am far from thrilled to see it here. Convincing learners that the easily understood "not the null" definition is wrong usually requires a lot of work and pain.
After all, who likes being told that what they thought they understood, is what they still do not understand?
This makes it more difficult for me to help my students understand the central importance of power to statistical testing. And, as Neyman pointed out, the power of a test is the main determinant of how useful it is.
It may seem that I am asking too much of an introductory text.
I do not think so.
It is my experience that one must engage in some fairly sophisticated reasoning to understand the meaning of the results of a statistical analysis. The simple, obvious interpretation is almost always wrong (cf., Darrell Huff's How to lie with statistics).
We do a learner no favors by simplifying a complex process to the point where we deceive the learner into thinking that they understand something that they do not.
The trick (which I am still working on mastering) is to help learners learn how to enjoy the challenge of minimizing, but still living with, uncertainty (an important element of all statistical reasoning) and also to help them learn to be suspicious of "easy" answers.
I recently got around to reading W. Edwards Deming's book, Out of the Crisis. In it, he made an observation about maintaining learner interest and quality teaching that is relevant to this book: "In my experience, I have seen a teacher hold a hundred and fifty students spellbound, teaching what is wrong." The Manga Guide to Statistics held my interest from the moment I started reading it. In fact, I read it in one sitting. I honestly enjoyed reading it, but it is wrong in too many places.
I purchased the Manga Guide to Statistics thinking that I might use it in my introductory research methods courses. I shall not use it. I shall not recommend it. I shall not mention it.
Note: I apologize for the lengthy discussion of the alternative hypothesis. I am afraid that I am not clever enough to find another way to demonstrate the problem of oversimplification.
Deming, W.E. (1986).
Out of the Crisis. Cambridge, MA: MIT Center for Advanced Engineering Study.
Huff, D. (1954).
How to Lie with Statistics. NY: Norton.
Neyman, J. & Pearson, E. (1933). On the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society of London. Series A, Mathematical and Physical Sciences, 231, 289-337.
