257524 Survey comprehension across multiple languages: Results from the California Health Interview Survey Discrimination Module

Monday, October 29, 2012 : 2:30 PM - 2:45 PM

Timothy Johnson, PhD , Survey Research Laboratory, University of Illinois-Chicago, Chicago, IL
Gordon Willis, PhD , Applied Research Program/DCCPS, National Cancer Institute, Bethesda, MD
Salma Shariff-Marco, PhD, MPH , Cancer Prevention Institute of California, Fremont, CA
Nancy Breen, PhD , Applied Research Program/DCCPS, National Cancer Institute, Bethesda, MD
Margarita Alegria, PhD , Psychiatry--Center for Multicultural MH Research, Harvard Medical School, Somerville, MA
E. Richard Brown, PhD , Department of Health Services, School of Public Health, University of California, Los Angeles, Los Angeles, CA
Gilbert C. Gee, PhD , School of Public Health, Community Health Sciences, University of California, Los Angeles, Los Angeles, CA
David Grant, PhD , California Health Interview Survey, UCLA Center for Health Policy Research, Los Angeles, CA
Nancy Krieger, PhD , Department of Society, Human Development, and Health, Harvard School of Public Health, Boston, MA
Hope Landrine, PhD , Center for Health Disparities, East Carolina University, Greenville, NC
Benmei Liu, PhD , Surveillance Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute, Bethesda, MD
Vickie M. Mays, PhD, MSPH , Psychology, University of California, Los Angeles, Los Angeles, CA
Ninez Ponce, MPP, PhD , Department of Health Services, UCLA, Los Angeles, CA
Bryce B. Reeve, PhD , Lineberger Comprehensive Cancer Center & Department of Health Policy and Management, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC
David T. Takeuchi, PhD , School of Social Work, University of Washington, Seattle, WA
David R. Williams, PhD, MPH , Department of Social and Behavioral Sciences, African and African American Studies, and Sociology, Harvard School of Public Health, Boston, MA
Cross-cultural variability in respondent comprehension of survey questions remains an important factor that may serve as a source of bias in research conducted in multi-ethnic and multilingual settings, including areas of the United States. To examine this we report on data collected as part of a field test nested within the 2009 California Health Interview Survey (CHIS), to evaluate a newly developed racial/ethnic discrimination module. Specifically, we employed Behavior Coding to detect ‘problems in the interaction' between interviewer and respondent, by coding behaviors that are seen as indicators of comprehension problems (e.g., the respondent interrupts, requests clarification, provides an uncodable answer). Behavior codes were applied to a sample of 550 interviews that had been completed in Cantonese, Mandarin, Korean, Spanish and Vietnamese, and English. A total of 19,800 interviewer-respondent interactions to individual survey items were coded, and included in a hierarchical linear model which examined the effects of language of interview, and respondent race/ethnicity, on question comprehension, controlling for other respondent characteristics (i.e., age, gender, education), question characteristics (i.e., length, reading level, time frame and response format) and interviewer behavior (i.e., reading the question correctly). Codes indicating comprehension difficulties were more likely to be assigned to non-English interviews across all racial/ethnic groups. We discuss the nature of the cross-cultural variation in the results obtained. To conclude, we outline a research agenda for continued investigation of this important challenge to the conduct of multi-ethnic and multilingual survey research.

Learning Areas:
Diversity and culture
Public health or related research
Social and behavioral sciences

Learning Objectives:
Describe the uses for behavior coding analysis in assessing survey comprehension across racial, ethnic, and linguistic groups. Demonstrate the application of hierarchical linear modeling to examine the effects of language and racial/ethnic group on question comprehension. Discuss a future research agenda for improving question comprehension in multi-ethnic and multilingual survey research.

Keywords: Survey, Measuring Social Inequality

Presenting author's disclosure statement:

Qualified on the content I am responsible for because: I am the Director of the Calif. Health Interview Survey and have been substantively involved in this research project for several years.
Any relevant financial relationships? No

I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.