Moderating usability tests : principles and practice for interacting

Moderating Usability Tests provides insight and guidance for usability testing. To a large extent, successful usability testing depends on the skills of the person facilitating the test. However, most usability specialists still learn how to conduct tests through an apprentice system with little formal training.

This book is the resource for new and experienced moderators to learn about the rules and practices for interacting. Authors Dumas and Loring draw on their combined 40 years of usability testing experience to develop and present the most effective principles and practices - both practical and ethical - for moderating successful usability tests. The videos are available from the publisher's companion web site.

• Preface p. xiii
• Acknowledgments p. xv
• About the Authors p. xvii
• Chapter 1 Introduction p. 1
• 1.1 Why This Book? p. 1
• 1.2 What Is Usability Testing? p. 2
• 1.3 The Importance of Moderating Skills p. 3
• 1.4 The Golden Rules of Moderating p. 3
• 1.5 Cultural Points of View p. 4
• 1.6 About the Sidebars in This Book p. 4
• 1.7 About the Videos that Accompany This Book p. 4
• 1.8 About the Companion Web Site p. 5
• Chapter 2 Getting started as a test moderator p. 7
• 2.1 What Makes a Great Moderator? p. 7
• 2.1.1 Understanding usability testing p. 7
• 2.1.2 The basics of interacting p. 8
• 2.1.3 The ability to establish and maintain a rapport p. 8
• 2.1.4 Lots of practice p. 9
• 2.2 Roles of a Moderator p. 9
• 2.2.1 The Gracious Host p. 9
• 2.2.2 The Leader p. 10
• 2.2.3 The Neutral Observer p. 10
• 2.2.4 Other possible roles p. 11
• 2.2.5 When roles change p. 11
• 2.2.6 When roles conflict p. 11
• 2.3 Testing Locations p. 12
• 2.4 Test Preparation p. 12
• 2.4.1 Planning ahead p. 12
• 2.4.2 Planning for many tasks p. 14
• 2.4.3 Understanding the domain and product p. 15
• 2.5 Jump-Starting Your Moderating Skills p. 15
• 2.5.1 Six things you can do first p. 15
• 2.5.2 The big challenges p. 16
• Chapter 3 Golden rules 1 through 5 p. 19
• 3.1 Rule 1: Decide How to Interact Based on the Purpose of the Test p. 20
• 3.1.1 The type of test p. 20
• 3.1.2 The product's stage of development p. 21
• 3.1.3 Relationship with developers p. 22
• 3.2 Rule 2: Protect Participants' Rights p. 23
• 3.2.1 Compensation p. 23
• 3.2.2 Informed consent p. 23
• 3.2.3 Confidentiality p. 24
• 3.2.4 Balance of purpose and risk p. 26
• 3.2.5 Priorities p. 26
• 3.3 Rule 3: Remember Your Responsibility to Future Users p. 26
• 3.3.1 Letting participants struggle p. 27
• 3.3.2 Conflicting responsibilities p. 28
• 3.4 Rule 4: Respect the Participants as Experts, but Remain in Charge p. 28
• 3.4.1 Controlling logistics and pacing p. 28
• 3.4.2 Stopping unacceptable behavior p. 29
• 3.4.3 Dealing with uncertainty p. 29
• 3.4.4 Projecting authority p. 30
• 3.4.5 Dispelling negativity p. 31
• 3.4.6 Managing visitors p. 32
• 3.5 Rule 5: Be Professional, Which Includes Being Genuine p. 33
• 3.5.1 Dos for making and maintaining a connection p. 33
• 3.5.2 Don'ts for making and maintaining a connection p. 35
• 3.5.3 When you're not in the room p. 37
• Chapter 4 Golden rules 6 through 10 p. 39
• 4.1 Rule 6: Let the Participants Speak! p. 39
• 4.1.1 Speakership p. 40
• 4.1.2 Appropriate interruptions p. 40
• 4.1.3 Judicious speaking p. 41
• 4.1.4 Silent communication p. 41
• 4.2 Rule 7: Remember That Your Intuition Can Hurt and Help You p. 42
• 4.2.1 Your intuition can hurt you p. 42
• 4.2.2 Your intuition can help you p. 42
• 4.3 Rule 8: Be Unbiased p. 43
• 4.3.1 Use an unbiased test script p. 43
• 4.3.2 Use unbiased questions p. 44
• 4.3.3 Keep answers unbiased p. 45
• 4.3.4 Watch nonverbal cues p. 45
• 4.4 Rule 9: Don't Give Away Information Inadvertently p. 46
• 4.4.1 Giving an assist p. 46
• 4.4.2 Explaining the designer's intent or being defensive p. 47
• 4.4.3 Recording all suggestions p. 48
• 4.5 Rule 10: Watch Yourself to Keep Sharp p. 48
• Chapter 5 Initial contacts p. 51
• 5.1 Recruiting p. 51
• 5.1.1 Contacting participants p. 52
• 5.1.2 Advertising p. 53
• 5.1.3 Explaining the test p. 54
• 5.1.4 Screening candidates p. 57
• 5.1.5 Confirming appointments p. 59
• 5.2 When Participants Arrive p. 59
• 5.2.1 Greeting participants p. 60
• 5.2.2 Creating comfort p. 60
• 5.2.3 Obtaining informed consent p. 61
• 5.3 The Pretest Briefing p. 64
• 5.3.1 Preparing yourself p. 64
• 5.3.2 Preparing participants p. 64
• 5.3.3 Using a script or checklist p. 65
• 5.3.4 Practicing the think-aloud technique p. 66
• 5.3.5 Confirming that participants are ready p. 67
• 5.4 Transitioning to the Tasks p. 68
• 5.4.1 Starting the tasks p. 68
• 5.4.2 conducting a pretest interview p. 68
• Chapter 6 Interacting during the session p. 71
• 6.1 Interacting for a Reason p. 71
• 6.2 Keeping Them Talking p. 72
• 6.2.1 Prompting as a reminder p. 72
• 6.2.2 Prompting the silent ones p. 73
• 6.3 When and How to Probe p. 73
• 6.3.1 Probing questions p. 74
• 6.3.2 Planned versus spontaneous probes p. 74
• 6.3.3 Probes to avoid p. 75
• 6.3.4 Common probes p. 75
• 6.4 Providing Encouragement p. 76
• 6.4.1 Encouraging statements to avoid p. 76
• 6.4.2 Common encouraging statements p. 77
• 6.5 Dealing with Failure p. 77
• 6.5.1 Participants' self-blame p. 78
• 6.5.2 The moderator's distress p. 78
• 6.5.3 The participant's distress p. 79
• 6.5.4 Responses to participants' distress p. 80
• 6.5.5 Stopping a test p. 81
• 6.5.6 What you shouldn't do when a participant fails p. 82
• 6.6 Providing Assistance p. 83
• 6.6.1 An assist p. 83
• 6.6.2 Giving assistance p. 84
• 6.6.3 Interventions versus assists p. 85
• 6.6.4 Not giving assistance p. 85
• 6.6.5 Levels of assistance p. 86
• 6.6.6 Completing a task for a participant p. 87
• 6.6.7 Measuring assists p. 88
• Chapter 7 Interacting during post-test activities p. 91
• 7.1 Maintaining Your Roles p. 91
• 7.2 Determining the Order of Activities p. 92
• 7.3 Clarifying Things That Occurred during the Test p. 93
• 7.4 Administering Ratings and Questionnaires p. 94
• 7.4.1 Questionnaires p. 94
• 7.4.2 Reasons behind the ratings p. 95
• 7.4.3 Accuracy of ratings p. 96
• 7.5 Asking Open-Ended Questions p. 97
• 7.6 Allowing Others to Interact with Participants p. 98
• 7.7 Final Activities p. 100
• 7.7.1 Providing incentives p. 100
• 7.7.2 Ending the session p. 101
• Chapter 8 Interacting in a remote test session p. 105
• 8.1 What Is Remote Testing? p. 105
• 8.1.1 Synchronous and asynchronous testing p. 105
• 8.1.2 The technology p. 106
• 8.1.3 Advantages and disadvantages p. 107
• 8.2 Preparing for the Session p. 109
• 8.2.1 Recruiting p. 109
• 8.2.2 Obtaining informed consent and confidentiality p. 110
• 8.2.3 Determining the equipment candidates have p. 110
• 8.2.4 Establishing willingness and downloading software p. 113
• 8.3 Interacting during the Session p. 114
• 8.3.1 Establishing what participants see on their screen p. 116
• 8.3.2 Providing instructions on thinking aloud p. 116
• 8.3.3 Making the task scenarios available to participants p. 117
• 8.3.4 Avoiding dependencies between tasks p. 118
• 8.3.5 Managing visitors during the session p. 118
• 8.3.6 Dealing with distractions at the participants' end p. 119
• 8.3.7 Making a connection despite the physical distance p. 121
• Chapter 9 Moderator-participant arrangements p. 123
• 9.1 A Bit of History p. 124
• 9.2 Physical Arrangement p. 125
• 9.2.1 The moderator in the test room p. 125
• 9.2.2 The moderator not in the test room p. 126
• 9.3 Beliefs about Arrangements p. 126
• 9.3.1 Physical separation p. 126
• 9.3.2 Physical proximity p. 127
• 9.4 Choice of Arrangement p. 128
• 9.4.1 Being physically close to participants p. 128
• 9.4.2 Being physically separated from participants p. 129
• 9.5 Considerations for the Practitioner p. 130
• Chapter 10 Interacting with diverse populations p. 133
• 10.1 General Guidelines p. 134
• 10.2 People with Physical Disabilities p. 138
• 10.2.1 Interacting with the physically disabled p. 139
• 10.2.2 Interacting with blind participants p. 140
• 10.2.3 Interacting with deaf and hard-of-hearing participants p. 143
• 10.3 The Elderly p. 145
• 10.3.1 Recruiting elders p. 145
• 10.3.2 Interacting with elders p. 146
• 10.4 People Who Have Low Literacy Skills p. 147
• 10.4.1 Functional illiteracy p. 147
• 10.4.2 Interacting with functionally illiterate participants p. 148
• 10.4.3 Testing with low-literacy participants p. 149
• 10.5 Children and Teens p. 149
• 10.5.1 Grouping children by age p. 150
• 10.5.2 Recruiting children and teens p. 150
• 10.5.3 Interacting during the session p. 152
• 10.6 People From Other Cultures p. 152
• 10.6.1 Interacting with participants from the same and other cultures p. 153
• 10.6.2 Adapting your techniques p. 155
• Chapter 11 Integrating the videos p. 157
• 11.1 About the Videos p. 157
• 11.1.1 The test session videos p. 157
• 11.1.2 Use of the videos p. 158
• 11.2 Content of the Videos p. 159
• 11.2.1 Video 1: Pretest briefing with a checklist p. 159
• 11.2.2 Video 2: Pretest briefing following a script p. 159
• 11.2.3 Video 3: Interacting during the session, example 1 p. 162
• 11.2.4 Video 4: Interacting during the session, example 2 p. 162
• 11.2.5 Video 5: Interacting in a remote testing situation p. 162
• 11.2.6 Video 6: Post-test interview p. 162
• 11.3 The Future of Usability Testing p. 162
• 11.3.1 Current trends p. 164
• 11.3.2 What's next? p. 167
• References p. 169
• Index p. 175