Effect of Question Characteristics on Responses to Health Survey Questions

It is commonly understood that survey questions vary in terms of how easily respondents are able to comprehend their meaning. To investigate this problem, we report data from a set of 600 adult interviews conducted in Chicago that were included in a randomized factorial experiment designed to examine multiple question design features that may be associated with respondent ability to adequately comprehend the meaning of survey questions. Among the question characteristics examined are question response format (e.g., yes/no vs. numeric vs. verbal labels), question topic (physical vs. mental health), type of judgment (time-qualified vs. not), type of report (self vs. proxy), question length, and language complexity. We employ mixed regression models to evaluate the independent effects of each question characteristic on respondent comprehension, which is measured by behavior coding of respondent reactions to each question asked of them. Models control for respondent characteristics and behavior code assessments of the adequacy with which interviewers read each survey question to respondents. Recommendations regarding optimal design of survey questions will be presented.