Lost and found: On the effects of failure to include hard-to-reach respondents in public health research

Objectives: In longitudinal studies, it may be difficult to retain hard-to-reach populations, including welfare recipients, the poor, the unstably housed, and other vulnerable and economically or socially marginalized groups. Differential attrition of hard-to-reach respondents may lead to biased and inaccurate results.

Methods: We report findings from a nonresponse simulation of a survey of 498 female cash aid recipients in a California county. Hard-to-reach respondents were reinterviewed 12 months after baseline only after extended tracking efforts. A simulation based on a study previously published in The American Journal of Public Health examined the consequences of failure to include these hard-to-reach respondents.

Results: Standard bivariate comparisons of baseline characteristics revealed few apparent differences between hard- and easy-to-reach respondents. However, a deeper exploration through simulation analyses showed that failure to include hard-to-reach respondents would have decreased response rates from 89% to 71%, and to well below 70% in important subgroups, as well as biasing and nullifying results of the original, published study.

Conclusions: The effects of failure to include hard-to-reach respondents in public health research are not necessarily predicable based on simple non-response comparisons. The pursuit of more effective strategies for tracking and retaining hard-to-reach respondents should become a priority in public health research, particularly research that includes members of vulnerable or hard-to-reach populations.