Survey comprehension across multiple languages: Results from the California Health Interview Survey Discrimination Module

Cross-cultural variability in respondent comprehension of survey questions remains an important factor that may serve as a source of bias in research conducted in multi-ethnic and multilingual settings, including areas of the United States. To examine this we report on data collected as part of a field test nested within the 2009 California Health Interview Survey (CHIS), to evaluate a newly developed racial/ethnic discrimination module. Specifically, we employed Behavior Coding to detect ‘problems in the interaction' between interviewer and respondent, by coding behaviors that are seen as indicators of comprehension problems (e.g., the respondent interrupts, requests clarification, provides an uncodable answer). Behavior codes were applied to a sample of 550 interviews that had been completed in Cantonese, Mandarin, Korean, Spanish and Vietnamese, and English. A total of 19,800 interviewer-respondent interactions to individual survey items were coded, and included in a hierarchical linear model which examined the effects of language of interview, and respondent race/ethnicity, on question comprehension, controlling for other respondent characteristics (i.e., age, gender, education), question characteristics (i.e., length, reading level, time frame and response format) and interviewer behavior (i.e., reading the question correctly). Codes indicating comprehension difficulties were more likely to be assigned to non-English interviews across all racial/ethnic groups. We discuss the nature of the cross-cultural variation in the results obtained. To conclude, we outline a research agenda for continued investigation of this important challenge to the conduct of multi-ethnic and multilingual survey research.