Revising the YTS questionnaire: Results from a 2007-2008 split-ballot experiment

In 2007 and 2008, Youth Tobacco Survey (YTS) split-ballot experiments were conducted in Virginia and Mississippi to identify whether changes were needed to the YTS questionnaire to improve data quality. We evaluated the hypothesis that modifying the wording of survey questions would enable respondents to interpret and answer questions easier, resulting in significant differences in the distribution of responses. We changed zero from a numeric to a text field in a response option to determine if this change would clarify the meaning of the category for nonsmokers, and decrease both the number of students choosing the text category, and the amount of missing data.

YTS data was collected from 11,521 students and used in this analysis. Results from a standard and test questionnaire were compared across questions in both states' questionnaires. Missing data was defined as blank, out of range, or inconsistent answers as defined by our editing procedures. Rao-Scott chi-square tests were used to determine statistical significance between standard and test versions at p<0.05.

Overall, there were few significant differences between the test and standard questionnaire versions, and the amount of missing data did not decline. The data indicated that text rather than zero decreased the number of students selecting that option but it did not decrease the percentage of missing data. Some particular changes to the wording of questions were also indicated.