Cognitive interviewing methodology

AN INTERDISCIPLINARY PERSPECTIVE TO THE EVOLUTION OF THEORY AND METHODOLOGY WITHIN COGNITIVE INTERVIEW PROCESSES

Providing a comprehensive approach to cognitive interviewing in the field of survey methodology, Cognitive Interviewing Methodology delivers a clear guide that draws upon modern, cutting-edge research from a variety of fields.

Each chapter begins by summarizing the prevailing paradigms that currently dominate the field of cognitive interviewing. Then underlying theoretical foundations are presented, which supplies readers with the necessary background to understand newly-evolving techniques in the field. The theories lead into developed and practiced methods by leading practitioners, researchers, and/or academics. Finally, the edited guide lays out the limitations of cognitive interviewing studies and explores the benefits of cognitive interviewing with other methodological approaches. With a primary focus on question evaluation, Cognitive Interviewing Methodology also includes:

Step-by-step procedures for conducting cognitive interviewing studies, which includes the various aspects of data collection, questionnaire design, and data interpretation Newly developed tools to benefit cognitive interviewing studies as well as the field of question evaluation, such as Q-Notes, a data entry and analysis software application, and Q-Bank, an online resource that houses question evaluation studies A unique method for questionnaire designers, survey managers, and data users to analyze, present, and document survey data results from a cognitive interviewing study
An excellent reference for survey researchers and practitioners in the social sciences who utilize cognitive interviewing techniques in their everyday work, Cognitive Interviewing Methodology is also a useful supplement for courses on survey methods at the upper-undergraduate and graduate-level.

• Foreword p. xiii Gordon Willis
• Acknowledgments p. xv
• Contributors p. xvii
• 1 Introduction p. 1 Kristen Miller
• 1.1 Cognitive Interviewing Methodology p. 2
• 2 Foundations and New Directions p. 7 Valerie Chepp and Caroline Gray
• 2.1 Introduction p. 7
• 2.2 Sociology and the Interpretivist Tradition p. 8
• 2.3 New Directions: Interpretation and Cognition p. 9
• 2.4 Methodological Implications for Cognitive Interviewing p. 11
• 2.5 Conclusion p. 14
• 3 Data Collection p. 15 Stephanie Willson and Kristen Miller
• 3.1 Introduction p. 15
• 3.2 Cognitive Interviewing Study Sample p. 15
• 3.2.1 Considerations of Sample Design p. 16
• 3.3 The Cognitive Interview p. 20
• 3.3.1 Differing Approaches to Cognitive Interviewing p. 20
• 3.3.2 Different Kinds of Data: Respondent as Evaluator versus Respondent as Story Teller p. 22
• 3.4 The Role of Interviewer p. 28
• 3.4.1 Interviewer as Data Collector p. 29
• 3.4.2 Interviewer as Researcher p. 30
• 3.5 Conclusion p. 33
• 4 Analysis p. 35 Kristen Miller and Stephanie Willson and Valerie Chepp and J. Michael Ryan
• 4.1 Introduction p. 35
• 4.2 Analysis of Cognitive Interviews: Overview p. 36
• 4.3 Analytic Steps for Cognitive Interviews p. 38
• 4.3.1 Step1: Conducting the Interview p. 39
• 4.3.2 Step Two: Producing Interview Summaries p. 40
• 4.3.3 Step Three: Developing a Thematic schema p. 42
• 4.3.4 Step Four: Developing an Advanced Schema p. 44
• 4.3.5 Step Five: Making Conclusions p. 47
• 4.4 The Benefits of a Complete Analysis p. 49
• 4.5 conclusion p. 50
• 5 Assessing Translated Questions via Cognitive Interviewing p. 51 Alisu Schoua-Glusberg and Ana Villar
• 5.1 Introduction p. 51
• 5.2 Why Use Cognitive Testing in Multilingual Survey Research p. 51
• 5.2.1 Multilingual Research Settings p. 52
• 5.2.2 Instrument Production in Multilingual Settings p. 53
• 5.3 Translation and Translation Assessment Procedures p. 54
• 5.3.1 Team Translation Approaches p. 54
• 5.3.2 Translation Assessment Procedures p. 54
• 5.3.3 Pretesting as Part of Translation Assessment p. 55
• 5.4 Cognitively Testing Translations of Survey Questions p. 57
• 5.4.1 Cognitive Interviewers p. 58
• 5.4.2 Respondent Selection p. 58
• 5.4.3 Introduction, Protocol, and Implementation p. 59
• 5.4.4 Analysis p. 59
• 5.5 Problems Uncovered by Cognitive Testing of Translations p. 60
• 5.5.1 Uncovering Translation Problems p. 61
• 5.5.2 Uncovering Problems with the Source Question p. 64
• 5.5.3 Uncovering Problems Related to Cultural Differences Need for Adaptation p. 65
• 5.6 Conclusion p. 66
• 6 Conveying Results p. 69 Valerie Chepp and Paul Scanlon
• 6.1 Introduction p. 69
• 6.2 Contents of a Cognitive Interviewing Report p. 70
• 6.2.1 Introduction p. 70
• 6.2.2 Summary of Findings p. 71
• 6.2.3 Methods p. 73
• 6.2.4 Question-by-Question Review p. 77
• 6.2.5 Appendix p. 80
• 6.3 Characteristics of a Cognitive Interviewing Report p. 80
• 6.3.1 Transparency p. 81
• 6.3.2 Reflexivity p. 82
• 6.4 Conclusion p. 84
• 7 Case Study: Evaluation of a Sexual Identity Question p. 85 Kristen Miller and J. Michael Ryan
• 7.1 Introduction p. 85
• 7.2 Background p. 86
• 7.2.1 Intended Construct for the National Health Interview Survey Sexual Identity Question p. 86
• 7.2.2 Review of Data Quality Problems p. 87
• 7.2.3 Development of an Improved Sexual Identity Question p. 89
• 7.3 Case Study: Cognitive Interviewing Evaluation of the National Health Interview Survey Revised Sexual Identity Question p. 93
• 7.3.1 Recruitment and Respondent Demographics p. 93
• 7.3.2 Interviewing Procedures p. 93
• 7.3.3 Data Analysis p. 95
• 7.4 Case Study Findings p. 96
• 7.4.1 Summary of Question Performance p. 96
• 7.4.2 Basis of Respondents' Answers p. 97
• 7.4.3 Cases of Response Problems p. 99
• 7.4.4 Interpretation of Categories p. 100
• 7.4.5 Study Conclusions p. 105
• 7.5 Conclusion p. 106
• 8 Analysis Software for Cognitive Interviewing Studies: Q-Notes and Q-Bank p. 107 Justin Mezetin and Meredith Massey
• 8.1 Introduction p. 107
• 8.2 Q-Notes Analysis Features p. 108
• 8.2.1 Level 1: Conducting Interviews p. 108
• 8.2.2 Level 2: Summarizing Interview Notes p. 110
• 8.2.3 Level 3: Comparing Across Respondents p. 112
• 8.2.4 Level 4: Comparing Across Groups p. 115
• 8.2.5 Level 5: Drawing Conclusions about Question Performance p. 118
• 8.3 Project Management Features p. 118
• 8.3.1 Streamlined Communication p. 119
• 8.3.2 Interview Data Collection p. 119
• 8.3.3 Respondent Descriptors p. 121
• 8.3.4 Controlled Access by Project p. 121
• 8.3.5 Adding Questions p. 122
• 8.3.6 Question Translations p. 124
• 8.3.7 Coding Schemes p. 124
• 8.4 Q-Bank: Making Cognitive Interview Findings Publicly Accessible p. 125
• 8.5 Q-Bank Features p. 126
• 8.5.1 Searching for Questions p. 126
• 8.5.2 Advanced Search p. 127
• 8.5.3 Question Details p. 128
• 8.5.4 Value of Q-Bank p. 129
• 8.6 Q-Bank: Challenges for the Past and Future p. 130
• 8.7 Conclusion p. 130
• 9 Cognitive Interviewing in Mixed Research p. 133 Isabel Benitez Baena and Jose-Luis Padilla
• 9.1 Introduction p. 133
• 9.2 The Mixed Research Paradigm: Characteristics and Design p. 134
• 9.2.1 Cognitive Interviewing Studies and Research Design p. 135
• 9.3 Mixed Method Research and Survey Question Evaluatin p. 136
• 9.3.1 Case 1: Cognitive Interviewing and Survey Field Testing p. 137
• 9.3.2 Case 2: Cognitive Interviewing and Differential Item Functioning (DIF) p. 145
• 9.3.3 Case 3: Cognitive Interviewing and Psychometric Scales p. 148
• 9.4 Conclusion p. 152
• 10 Conclusion p. 153 Kristen Miller and Stephanie Willson and Valerie Chepp and Jose-Luis Padilla
• 10.1 Introduction p. 153
• 10.2 Summary of Practices p. 154
• 10.2.1 Data Collection p. 154
• 10.2.2 Analysis p. 156
• 10.2.3 Documenting Study Findings p. 157
• 10.3 New Directions p. 159
• 10.3.1 Topics for Examination p. 159
• 10.3.2 Mixed Method Research p. 160
• 10.3.3 Accepted Standards of Cognitive Interviewing Studies p. 161
• Key Concepts p. 163
• Question Evaluation Resources p. 167
• References p. 171
• Index p. 181