Moderating Usability Tests: Principles and Practices for Interacting (Interactive Technologies) [Paperback]

Joseph S. Dumas (Author), Beth A. Loring (Author)

Book Description

ISBN-10: 0123739330 | ISBN-13: 978-0123739339 | Publication Date: March 14, 2008 | Edition: 1

Many aspects of usability testing have been thoroughly studied and documented. This isn't true, however, of the details of interacting with the test participants who provide the critical usability data. This omission has meant that there have been no training materials and no principles from which new moderators can learn how to interact.

Moderating Usability Tests is the place for new and experienced moderators to learn about the rules and practices for interacting that have never been described in one place before. Authors Dumas and Loring draw on their combined 40 years of usability testing experience to develop and present the most effective principles and practices - both practical and ethical --for moderating successful usability tests.

To help usability professionals, students, and novices understand these principles, the authors provide videos from their lab that demonstrate good and poor interaction as well as commentary from a panel of testing experts on why certain techniques succeed or fail. The videos are accessible from the publisher's companion web site.

. Presents the ten "golden rules" that maximize every session's value
. Offers targeted advice on how to maintain objectivity
. Discusses the ethical considerations that apply in all usability testing
. Explains how to reduce the stress that participants often feel
. Considers the special requirements of remote usability testing
. Demonstrates good and bad moderating techniques with laboratory videos accessible from the publisher's companion web site

Editorial Reviews

Review

Joe and Beth really know their stuff, and they've put together a book that's enormously valuable for usability professionals and usability amateurs. Whether you've conducted hundreds of tests or are about to try your first one, you owe it to yourself--and your team...and your test participants--to read this. - Steve Krug, author of Don't Make Me Think

Interacting with participants in a calm and neutral manner may well be the most difficult part of doing usability testing. Now you no longer have to worry about how to do that. Just follow Dumas and Loring's wonderful, practical advice and you will be prepared not only for typical encounters, but also for the unusual and unexpected, for doing remote testing, and for working with special populations. Moderating Usability Tests is a great resource for anyone who interacts with usability test participants. - Janice (Ginny) Redish, President, Redish & Associates, Inc.

Everyone talks about research methods, but the formal aspects of those methods only get you so far. The difference between getting a little data or a lot of data, only discovering problems or getting ideas about solutions, bias or validity, throw-away data versus generalizable insights, often depend on the soft skills, the ability to effectively moderate testing. In the past, you were expected to get these skills through apprenticeships or trial and error. Moderating Usability Tests: Principles for Interacting with Participants removes the mystery and provides practical advice on how to get the most out of research. It will be invaluable to students learning about usability testing for the first time, people newly charged with evaluating products, and even old hands looking to refine and improve their technique. - Arnold Lund, Director of User Experience, Microsoft

You may not think that being a "Gracious Host" is among your assignments in moderating a usability test, but you will learn why this and other roles with similarly illuminating names are important to your success. In this generous book, Dumas and Loring give the benefit of their decades of experience and astute observation of both the foundational and the subtle aspects of conducting usability tests. Many questions you didn't think to ask until you were on the hot seat are answered here, and will help you achieve a level of confidence as a test moderator that may have seemed beyond reach, even if your participants are from challenging-to-test populations. With this highly ethical and thoroughly grounded program for developing moderator skills and avoiding pitfalls, Dumas and Loring make a strong contribution to the body of knowledge on testing products. The big surprise of the book is that their clear, reasoned, and detailed suggestions about interacting with test participants and developers will likely spill over and improve your relationships with co-workers, family, neighbors, and friends. - Elisabeth Bayle, Bayle Collaborations

At this point, virtually everyone in the software industry knows what usability testing is. An unfortunate side effect of this awareness is that many people are conducting usability testing who have no idea how to do so in a way that will yield valid, reliable and useful data. Other than the design of the test itself, proper and effective moderation of test sessions is one of the most important - and least understood - aspects of usability testing. Here is a book by two highly regarded experts that covers this topic thoroughly in a very readable format. No one who has not already been well trained should attempt to conduct usability testing without first reading this book cover to cover, and viewing all the excellent videos the authors provide. - Deborah J. Mayhew, Deborah J. Mayhew & Associates

About the Author

Joe Dumas is a recognized expert in usability evaluation. He has 25 years experience as a usability professional. He as moderated or observed others moderate thousands of usability testing sessions and taught numerous students and usability professionals how to moderate. He is the author of A practical guide to usability testing (with Ginny Redish), Designing user interfaces for software, and numerous articles, both for researchers and practitioners. He is currently a Usability Consultant for Oracle Corporation. He was a Senior Human Factors Specialist at Bentley College's Design and Usability Center and taught graduate courses in the college's Human Factors in Information Design Master's Degree program. He has a Ph.D. in Cognitive Psychology.

Product Details

• Paperback: 208 pages
• Publisher: Morgan Kaufmann; 1 edition (March 14, 2008)
• Language: English
• ISBN-10: 0123739330
• ISBN-13: 978-0123739339
• Product Dimensions: 9.2 x 7.6 x 0.5 inches
• Shipping Weight: 14.4 ounces (View shipping rates and policies)
• Average Customer Review: 4.7 out of 5 stars  See all reviews (3 customer reviews)
• Amazon Best Sellers Rank: #514,635 in Books (See Top 100 in Books)

More About the Author

Discover books, learn about writers, read author blogs, and more.

Customer Reviews

Most Helpful Customer Reviews

3 of 3 people found the following review helpful:

5.0 out of 5 stars Comprehensive guide to nuances of participant interaction, July 14, 2008

By 

Michael Hawley - See all my reviews

This review is from: Moderating Usability Tests: Principles and Practices for Interacting (Interactive Technologies) (Paperback)

It's one thing to moderate a usability test. It's another thing to do it right. In their book Dumas and Loring provide detailed discussions on the many nuances of interaction with participants during a usability test and a guide on how to maximize time with test participants.

The book is well organized and concisely written making it easy for inexperienced moderators to learn quickly. It would also be useful for usability test moderators who already facilitate sessions but never had any formal training. There are considerations for interaction with participants that are not obvious, but have implications on validity of results. For example, do you think about the height of your chair compared to the participants? Do you think about where you are positioned during a study? Do you consider the wording when assisting a participant on a task?

In addition, in my opinion, the book would also be useful for experienced moderators as it provides a comprehensive view of all elements of interacting with participants. Experienced moderators can fall into bad habits and develop patterns that introduce bias and minimize the effectiveness of study results. This book exposes all of the areas where this could happen and provides instruction on best practices.

Especially helpful is the format of 10 golden rules for interaction with participants. It provides a straight forward checklist of elements to consider when moderating a test - almost a reminder checklist to review. The other chapters on interacting in a remote test and with diverse populations are timely and helpful.

I would strongly recommend this book to all those who are looking to improve their usability test moderating technique.

-- Michael Hawley, VP Experience Design, Mad*Pow Media Solutions, LLC

3 of 3 people found the following review helpful:

4.0 out of 5 stars An Excellent Overview, June 7, 2008

By 

T. Bazler (Indiana) - See all my reviews
(REAL NAME)   

This review is from: Moderating Usability Tests: Principles and Practices for Interacting (Interactive Technologies) (Paperback)

Dumas and Loring provide an excellent overview of best practices for usability test session facilitators and how to interact with session participants. Although best suited for new/inexperienced facilitators, the book does provide a good review and teaching tool for the experienced facilitator.

The book discusses the full gamut of test facilitation from recruiting participants to escorting the participant out after the session and everything in between. In cases where a particular method of moderating cannot be shown by research as being superior to other methods, the authors discuss various options and provide the pros and cons of each for the reader to consider. Developing testing protocols and how to use them to reduce the influence of facilitator bias is discussed.

The authors illustrate ideas using examples from their own experiences, as well as interviews with experienced moderators. In most cases, the examples provided help clarify the topics addressed. In some cases though, the examples seem somewhat artificial or the example used seems like an over exaggeration in the moderator's interpretation of participant actions/comments. I would strongly caution any moderator against reading too much, especially of a personal nature, into comments or notes from participants.

The role-playing videos provided online as an accompaniment to the book were not especially helpful to me, but less experienced moderators might find value in them. The discussion panel videos were more interesting and allowed experts in the field to comment on what was shown in the role-playing videos, as well as debate their various points of view.

Beyond the actual information provided, from the usability professional's point of view, I am disappointed when print and online materials targeting the usability community fail to follow basic usability and good design practices. For example, throughout the book, sections providing suggestions or research examples are displayed on a gray background and a lightweight font is used. The contrast is poor making the content difficult to read, especially in lower light conditions or for those with visual impairments. On the website, a number of videos are provided to illustrate various points in the book. When the user clicks on a link to view a video, they are taken to a screen that provides no navigation. The only way to return to the homepage is to use the browser back button. Given the lack of navigation, users may assume it is a pop-up window and close it, forcing them to open a new browser window and type in the URL again to return to the homepage.

Overall, I would recommend this book as a good teaching tool for new facilitators and certainly plan on using it as a training tool for graduate students working in the usability lab at Indiana University.

~ Tara N. Bazler Manager
Enterprise Service Presentation & Delivery, Indiana University

1 of 1 people found the following review helpful:

5.0 out of 5 stars fantastic resource for usability people of all levels, March 18, 2009

By 

Lena Dmitrieva - See all my reviews
(REAL NAME)   

This review is from: Moderating Usability Tests: Principles and Practices for Interacting (Interactive Technologies) (Paperback)

If you would like to get a good understanding of what it takes to facilitate a usability test, this is a perfect guide. It provides an incredibly insightful overview of the entire process paying particular attention to the issue of interacting with the participants. You will get tons of practical information as well as theory and observe videos demonstrating examples of test moderation. It's just a great book!