Rationale for Testing Measures in Diverse Populations
To understand and improve health disparities requires selecting measures that are psychometrically sound and culturally appropriate for the disparity population. Because most measures were developed in non-minority, well-educated samples, it is important to determine whether such measures are conceptually and psychometrically appropriate for disparity populations. We describe some of the specialized methods for developing/adapting and testing measures in disparity populations in two publications.
Stewart AL, Nápoles-Springer A. Health-related quality of life assessments in diverse population groups in the United States. Medical Care, 2000;38(9 Suppl):II102.24. PMID:10982096
Stewart AL, Nápoles-Springer AM. Advancing health disparities research: can we afford to ignore measurement issues? Medical Care, 2003;41(11):1207-1220. PMID:14583684
Need Mixed Methods Approaches
Both qualitative methods (e.g., focus groups and cognitive pretest interviews) and quantitative methods (testing factor structure) can be used to test whether concepts and/or measures are appropriate for one or more diverse population groups. We provide several types of resources on the use of these qualitative and quantitative methods.
Qualitative Methods - Focus Groups: Focus groups are widely used in research on disparities and minority aging, and we summarize their use in exploring concepts and generating items for new measures, and pretesting existing measures.
- Develop concepts and generate items for a new measure: Focus groups are used in the formative stage of research to obtain the perspectives of persons from the ethnic groups being included. They allow research to reflect the concerns and issues of those being studied. Focus groups are especially useful to explore poorly understood and complex phenomenon, such as health disparities.
- Test existing measure: Another application of focus groups is in the process of determining if existing measures are culturally sensitive and appropriate. Using focus group methodology to test existing measures can yield evidence of the face validity of the constructs or domains to be assessed in a structured survey for use in the targeted groups.
Our annotated bibliography provides examples of using focus groups for these purposes. We “annotate” each publication by summarizing its content; links are included (e.g., PMCIDs or websites). There are also a few step-by-step guidelines for these methods which are included in the annotated bibliographies.
Qualitative Methods - Cognitive Interview Pretesting: Cognitive pretest interviews are used widely during the pre-testing phase of questionnaires to detect items that are not understood by respondents as intended by the survey developers. Cognitive interview methods reflect a theoretical model of the survey response process that involves four stages:
- Comprehension or interpretation (respondent understands the question)
- Information retrieval (respondent can recall the information requested)
- Judgment formation (respondent decides on relevance)
- Response editing (respondent formulates an answer in the format provided)
Cognitive interviews thus identify the types of errors made by respondents and help us understand how they interpret and answer questions. Our annotated bibliography provides examples of using cognitive pretest interview. We “annotate” each publication by summarizing its content; links are included (e.g., PMCIDs or websites).
Download: Using cognitive interviews
Quantitative Methods – Examining Invariance between Groups: Because most measures are developed in non-minority, well-educated samples, it is essential to determine whether such measures are conceptually and psychometrically equivalent in diverse subgroups. Conducting a psychometric evaluation of an instrument to establish validity and reliability is essential before using the test in a cross-cultural setting. Without equivalence, overall findings and observed group differences may contain measurement bias.
Analysis of invariance allows researchers to examine whether the variables of interest represent the same theoretical constructs across groups. We provide three publications that reflect work from our Analysis Core team.
Gregorich SE. Do self-report instruments allow meaningful comparisons across diverse population groups? Testing measurement invariance using the confirmatory factor analysis framework. Medical Care. 2006 Nov;44(11 Suppl 3):S78-94. PMC1808350.
Stewart AL, Nápoles-Springer AM, Gregorich SE, Santoyo-Olsson J. Interpersonal processes of care survey: patient-reported measures for diverse groups. Health Services Research. 2007 Jun;42(3 Pt 1):1235-56. PMC1955252
Wong ST, Nordstokke D, Gregorich S, Pérez-Stable EJ. Measurement of social support across women from four ethnic groups: evidence of factorial invariance. Journal of Cross Cultural Gerontology. 2010 Mar;25(1):45-58. PMC2836242.