To understand and improve health disparities often requires collecting data using questionnaires to address the research questions, e.g., self-report measures of various study concepts (e.g., pain, social support, financial hardship). In selecting measures for a particular study, investigators should pretest key measures to determine whether they are conceptually and psychometrically appropriate for the specific population of interest.
Rationale for Pretesting Measures in Diverse Populations
In two papers, we describe some of the specialized methods for adapting and testing measures in diverse populations.
Discusses the importance of ensuring the adequacy of measures of health-related quality of life across diverse groups. We advocate culturally sensitive strategies for testing measures, and provide examples from the literature of such efforts.
Describes conceptual and psychometric issues that should be addressed to assure the appropriateness of self-report measures. Based on a 2001 conference convened by several Resource Centers for Minority Aging Research (RCMARs).
Cognitive Interviews
Cognitive interviews are a form of pretest to assure that a survey instrument is appropriate and clear for potential respondents. They provide insight into how respondents interpret and answer questions, and can identify items that respondents do not understood as intended as well as problems with instructions, response choices, and respondent burden.
Cognitive interview methods reflect a theoretical model of the survey response process that involves four stages:
- Comprehension or interpretation (respondent understands the question)
- Information retrieval (respondent can recall the information requested)
- Judgment formation (respondent decides on relevance)
- Response editing (respondent formulates an answer in the format provided)
We provide several resources to understand and design cognitive interviews:
- Slide set: Reviews types of potential problems with questions including words/phrases not understood as intended, inadequate response choices, and unclear instructions. Describes how to create and conduct a cognitive interview.
- Annotated bibliography: Provides key publications and good examples of methods of using cognitive interviews to modify a survey or measure. We “annotate” each publication by summarizing its content.
- Sample Cognitive Interview: An example of how to design a cognitive interview to learn how respondents perceive structured questions. It includes a survey followed by questions/probes for a subset of questions. It illustrates the variety of probes, depending on the issue anticipated by researchers.
- Publication: This methodological article by CADC faculty describes the methods and results for the sample cognitive interview described above.
Other Formats for Cognitive Pretest Interviews: A pdf by two RCMAR investigators provides an alternative strategy for asking pretest questions – this example embeds the pretest questions in the survey.