Customer Reviews

Usability Testing and Research (Part of the Allyn & Bacon Series in Technical Communication)

5 star:		(5)
4 star:		(1)
3 star:		(0)
2 star:		(0)
1 star:		(1)

 

 

I am the Usability SIG Manager for the Washington, DC chapter of the STC (Society for Technical Communication). I wrote this review which was published in the October 2002 edition of the Usability Interface, the quarterly newsletter of the Usability SIG of the STC.

Dont be fooled by the somewhat unmemorable cover of Usability Testing and Research. Carol Barnum combines research findings with practitioner experience to produce probably the most comprehensive but concise resources on usability testing now available.

This book is part of the Allyn & Bacon series in technical communication. Designed in part to fulfill the needs of students in a graduate-level class in usability, it is also a great resource for usability practitioners as a tool to update and upgrade their skills. The structure of this book should suit both neophytes and the experienced. Each chapter and its subsections are well structured with a logical progression from one section to the next. The table of contents is well organized and very readable so that a knowledgeable individual can scan to find those sections that are of most immediate interest. The index is likely to be a great reference tool as it was created by a professional usability tester, someone who should know what a reader is likely to need.

Dr. Barnum, a usability consultant and professor to graduate usability students, draws upon nearly every prominent usability authority to build a comprehensive bible of usability testing. The authors academic background is evident in the careful footnoting of every page and the detailed listing of scores of references at the end of each chapter. If there is an issue the reader wishes to further explore, the source is cited for ready access. The findings from top usability professionals are distilled to their essence and woven into comprehensive work on usability. The reader gains the primary benefit of hundreds of books, papers, and articles without having to filter though this sea of information. There is little if anything of importance that is left out of this 448-page book of concentrated wisdom and knowledge.

The reader, whether a student or an experienced practitioner, gets the benefit of both analytic research and its practical application. Many examples of actual real-world, usability test plans, data, analysis, or summary reports are reprinted. These examples are great models for the practice of usability. This book has not just general how you might do something but also how it was actually done  not just theory but actual practice.

Anyone interested in the usability of hardware, software, computer games, Web sites  any product that has a user interface  will find a great wealth of information. The first chapter starts off defining usability and explaining usability testing models and theory. It also provides a methodology for cost justification and the basis to proselytiz for usability.

In Chapter 2, the author should be commended for taking a holistic view of usability and discussing how heuristics, surveys, and focus groups play a role in the design of a usable product. Chapters 3 through 7 explain user and task analysis as a precursor to designing, planning, preparing, and conducting the test. Chapter 8 details not just the analyses of the collected data but also different methods of reporting the findings to obtain the best result. The final chapter builds on everything before and discusses those issues specific to Web usability. Nearly every chapter has an appendix with real-world examples specific to that chapter. The end-of-the-book appendix discusses how to make usability testers work as a well-functioning team.

Some books are chock full of scholarly research and empirical data and great for academia. This book has a solid base in research but was written for the real-world practitioners of usability. Its this steady focus on practical real-world application that most impresses me about this book.

Still not convinced this book is for you? Check out the companion Web site, ..... Download material from the books appendices and sidebars. Peruse the many usability tools and link to other usability resources on the Web.

Usability Testing and Research is not only of great value as a resource of practical information to usability professionals but also as a tool to explain the benefits of usability design and testing to skeptical management. Carol Barnum should be congratulated for a great job in bringing together in one book such a complete, well-organized compilation of usability theory and practicality.

It's been a long time since we've had a new textbook on usability testing. Dumas and Redish came out in 1993, Rubin in 1994 and although I still use both of them constantly, I've been looking out for a solid textbook that has more awareness of the web in it. Carol Barnum's new book meets that need.

The book opens with chapters on `What is Usability and What is Usability Testing', `Other Methods for Getting Feedback About Product Usability', `User and Task Analysis', and `Iterative Testing for User-Centred Design'. I can see that Carol wants to set user testing in context, but I was concerned that if you're really new to usability testing then you might be put off by Chapter 2 `Other methods', as it is a very densely written chapter that describes many techniques very briefly.

The meat of the book starts at Chapter 5 with `Planning for Usability Testing' and continues through `Preparing for Usability Testing', `Conducting the Usability Test', and `Analysing and Reporting Results'. The book then changes course slightly with a chapter on 'Web Usability', giving some design principles as well as details of applying the methods to the web.

Our Open University students love the plentiful examples in our course on User Interface Design and Evaluation. Carol Barnum's book should also appeal because of its extensive use of examples. She gives lots of detail from a student team's test of Hotmail (Microsoft's web-based e-mail service) so you can see the process as they tackled it. I found it a little frustrating that there weren't any screen shots of Hotmail as it stood at the time of the test. As well as the Hotmail example she uses excerpts from a test of a University web site, and has lots of anecdotes and smaller examples as well, many of them aimed at testing documentation - a neglected area. Perhaps the amount of space taken up the examples means that there is less meat in the core of the book, but if I were a beginner I'd find it very reassuring. Conversely, though, experienced practitioners might find Chapter 5 onwards a bit basic.

Academics and practitioners who like to follow up interesting ideas will be glad to know that there is extensive referencing. The appendices placed in context with the chapters broke the flow for me somewhat when I was reading the book at a sitting, but I think they would be more convenient placed where they are when using the book to actually plan and conduct a test. Each chapter closes with questions/topics for discussion and exercises which looked helpful to me if you were planning to use this as a textbook, or if you are a new practitioner who is using the book as a guide through your first usability tests.

Carol Barnum's style is clear and easy to read as you would expect from a Professor in Technical Communication. She often uses comments from Chauncey Wilson, a very experienced practitioner to give some practical tips and insights, but I sometimes found myself wishing that she had put more a more personal touch, more of her own practical experience, explicitly into the book. Apart from a couple of anecdotes, the word `I' hardly appears until we get some of her own opinions on web usability at the end of Chapter 9. We can guess at one of her concerns because she includes an interesting appendix on `Making it work as a team', which I though was a good, concise introduction.

I would recommend this book as an introductory text for undergraduates because of the extensive examples, fairly reasonable price and referencing. I think it would also be good for practitioners - for people who are getting started with user testing - to help them through their first test. I think that I'll find myself recommending that readers should start with Chapter 5, and then come back to Chapters 1 to 4 later.

(This review was written for 'Interfaces', the magazine of the British HCI Group)

Usability Testing and Research (UTAR), by Carol M. Barnum,is a book that every technical communicator should have on his or her bookshelf. Although written primarily as a textbook for college courses, UTAR is an excellent introduction to user-centered design and usability testing for those in industry who have had no formal training in the area, or for those who would like a refresher. The book is designed so you can read straight through, skip around and read topics of interest, or dip into for specific information. An excellent table of contents and thorough index make topics immediately accessible and make it easy to follow them through the book. It shouldn't be surprising, however, that a book about usability that is this good would provide excellent access.

The very brief summary of the highlights of UTAR are:

* A structure that moves from theory to practice in every step of user-centered design and usability testing
* Well-written text with lots of heads for easy navigation
* Anecdotal and humorous information in sidebars to make the theory real
* Great descriptions of low-fidelity usability testing with examples of how to do it
* Samples of the artifacts of usability testing such as plans, forms, reports, questionnaires, testing questions, and checklists
* Real case studies with their artifacts that are threaded throughout the book in the appropriate chapters and appendixes
* Discussion questions and exercises for those who want to teach from the book
* A chapter on Web usability testing
* An appendix on creating teams to do usability testing

Then, as if the book weren't good enough by itself, Barnum has included the URL of a Web site that provides resources, templates, forms, and other support. The PDF templates and forms are useful as models for developing your own or to print and use as is. They fit nicely into the structure and content of UTAR. A support link on the Web page provides PowerPoint presentations and charts a teacher could use in the classroom or a technical communicator or manager could use to educate decision makers about the benefits of usability testing.

One of the buttons on the home page is "About the Author," so I clicked it to see what Barnum's credentials are. She a Fellow of the Society for Technical Communication (STC), and a recipient of STC's Jay R. Gould Award for Excellence in Teaching Technical Communication. That explains the excellence of Usability Testing and Research.

I manage a group of technical communicators in a large telecommunications company, and I'm getting a copy of Usability Testing and Research for our library. We'll use it to help with our usability testing and to mentor those who have little or no experience in that activity. My suggestion is to buy it, read it, refer back to it often.

Art Elser, PhD
Manager, Information Development

As a Usability professional, I have the opportunity and obligation to keep current on the literature surrounding usability research. Usability Testing and Research not only meets those criteria, but also gives readers the opportunity to delve into and understand the usability testing process.

Carol Barnum's book is at first a typical academic textbook that explains not only the principles of usability, but also the argument for usability and user-centered design. However, it is the presence of appendices at the end of most chapters that will help readers perform usability testing.

The appendices are detailed copies of documents and protocols used in previous usability tests. The aforementioned principles and arguments would stand on their own merits, but the added appendices give context to the principles and arguments, thus giving the reader the opportunity to see the principles and prototypes in action.

From my standpoint as a professional in the field, valid usability testing starts with careful, in-depth preparation. Usability Testing and Research handholds readers through this process, starting with objectives and ending with participant recruitment. This is a very important aspect of testing, as poor planning always equals poor results.

Barnum devotes several pages of the book to a very overlooked aspect of usability testing: reporting the results. Besides the detailed attention given to writing the results in a formal report or a quick report (or roadrunner as the book explains), the book also explains how to present and prepare for an oral report. No matter how valid the test may be, without an accurate, to-the-point report that gives the audience what it needs, the test results and associated costs are wasted. Barnum even discusses visual aids and the importance of the highlight tape. This, of course, is paramount to the report's success, since seeing is believing.

The book even contains a chapter devoted solely to Web usability that details common Web problems as well as insight into the Web's goal-driven users. Although research on the Web is constantly changing due to the experience level of the user base and innovative technology, the Web chapter offers an easy to understand benchmark for all usability professionals who evaluate the Web.

All in all, Usability Testing and Research covers everything that an individual new to the field needs to know. It also contains practical advice and how-to that even seasoned usability professionals need to review from time to time.

Ken Kellogg
Manager - Usability Research

I'm frankly startled at the universally positive reviews for this book. As a full-time usability researcher and an alumnus of Apple's human interface technologies team, I felt this to be one of the worst books on usability I've ever read. I could not possibly recommend it for either academic or professional purposes.

I'm going to limit specific comments to just the first 10 pages of this book. Critiquing the remainder would warrant a full dissertation, not an Amazon review.

1) Subpar fact-checking.

For example, on page 3: "Alan Cooper, a usability specialist who did pioneering work at Apple..." Alan Cooper never worked for Apple, let alone perform pioneering work there. Alan *Kay* worked for Apple. And Alan Kay's most significant work was by far at Xerox PARC; his Apple work with the Vivarium is rarely cited in comparison.

2) Too many overstated claims lacking conclusion validity

Given the author's own academic background, I'm startled by the poorly grounded research-based claims littered throughout the book. For example, on page 5: "a belief confirmed by a study that shows that '27% of all Web transactions are abandoned at the payment screen." ''This claim, as presented, is indefensible: no study author would have access to every single web transaction to perform such an categorical assessment. Were it even true at the time of the study, who's to say it's true now? Presumably she meant "27% of Web transactions *sampled*..."?

Especially given the author's added responsibility in writing for students who are likely concurrently learning basic research methods skills -- including assessing conclusion validity -- this book's own research claims should be responsibly situated. This aspect of the textbook disturbed me the most, because it presents the moral hazard of students tacitly learning to make similar such overreaching and false claims from their own work as usability researchers.

3) Author pushes usability testing as it may have been 10-15 years ago, not as it works today.

For example, on page 10, she quotes usability testing salesman Jakob Nielsen: "If no [usability test] information is available, you might as well choose by tossing a coin, and you will have a 50% probability of choosing the best interface".

Go tell this to Apple: there was not any formal usability testing on the iPhone, and yet reviewers hailed it as the most usable (and best interface) ever shipped on a phone. Instead, Apple iterated designs rapidly, and relies upon extensive use of peer cognitive walkthroughs and an emphasis in employee recruiting for good usability sense. By blindly dismissing the value of other methods of getting to good design, the author presents a skewed and misleading image of usability practice in reality.

Beyond countless such "yowser" moments in reading this book, it's just a mediocre book attempting to straddle the line between a practitioner book and a student textbook - while ultimately failing to meet the needs of either audience.

For practitioners, it's far too verbose and lacking in visual design affordances to use in practice; you can't scan a section for a few seconds before a study, because the book largely consists of paragraph after paragraph of text with the occasional bulleted list and header.

For students, it's just a low-value textbook: the printing is cheap (on-demand), and the price is high. The book's content suggests that author received no significant editorial or professional design assistance. The most important messages key to designing one's first usability study are buried in paragraph after paragraph of academic tangents. The author's widespread use of overreaching research-based knowledge claims makes it hard to know which assertions to trust, and which not to trust. Publishers should be investing a lot more resources into a book if they're charging such a high price (contrast with Baxter & Courage's beautiful user research book, which sadly omits usability testing).

Students - and practitioners - deserve a better usability testing book than this one.

Most of the existing books about usability testing are ten years old. While these books have a lot to offer a current student of user-centred design and usability testing, some students find them too out-of-date to be taken seriously. This book addresses that issue. The audience for the book is clearly for those who are new to usability and usability testing. However, experienced professionals will find a fair amount of information that is useful to them as well.

The first four chapters form an introduction to user-centred design: definitions, testing models, other methods for gathering usability data, user analysis, task analysis, and prototyping. The remaining chapters discuss the process of usability testing itself: planning the test, preparing for the test, and conducting the test. One interesting chapter discusses a topic that is usually overlooked in the other usability literature: analysing the data and writing the report. This chapter also discusses the Common Industry Format for reporting usability results. The inclusion of this discussion increases the usefulness of this book for usability professionals.

Dr Barnum devotes a chapter to the sometimes nebulous topic of usability testing for the web. Although there are other books that discuss this topic alone, the inclusion of this material in this book is excellent. Students who are new to usability testing will find this book more useful than a book devoted solely to web usability testing. After completing this, a student can then go on to read web-specific books such as Nielsen's recent "Designing Web Usability".

Students and others learning about the field will find the extensive examples used throughout the book to be extremely helpful. Together, these examples provide sufficient material for a student to model their complete usability test, from planning to reporting the results. Additional examples are available on a website maintained by the publisher. These examples are less useful for current practicioners, but do not detract from the overall text for this audience.

For those who are interested in this field, this book an excellent introduction. It is well-written, and the examples provide a cookbook for students to emulate. Experienced professionals will probably not find this book to be as useful as a new student, although it is a very useful refresher.

Disclaimer: I am a past student of Dr Barnum's, and the work that my student team produced for her class is used in this book. I am now a human factors engineer for a large corporation. I keep this book on my shelf in my office, and often lend it out to our interns who are learning about this field.

It's been a while since there's been a new book published on usability testing. It isn't a topic that gets dated, so the books that have been out there a while are still great ones. But it's good to have a new one too. This one by Carol Barnum is a thorough, practical, highly usable book on Usability Testing. Carol starts the book with chapters on user centered design. These chapters are pretty good, but the real value, I think is the detailed chapters on how to plan, carry out, analyze and report on usability tests that are really excellent.

Besides a lot of detail on usability testing protocol (well written and highly readable) there are lots of examples of facilitator scripts, recruiting and screener forms, Think Aloud instructions, etc.

I wish the book itself were of a higher quality. The binding, paper, and printing seem low budget, yet the price is not low! No matter, buy it anyway. Whether you are new to usability testing or an experienced tester, it's a useful book.