Validation of the Connecticut bioterrorism nursing curriculum exam

Background: Four nursing schools in CT have taught a class on bioterrorism annually since 2006. Students in the class complete a pre- and post-test. The test appeared to have validity problems, since the pre-test scores were high and scores on the post-test did not appear to demonstrate learning. This project was undertaken to discover problematic test items (stems & options) and suggest improvements.

Methods: Statistical tests identified problematic items and principles of good test design were used to determine why items were performing poorly. To detect items that were too easy at baseline, a difficulty index was constructed. Individual item discrimination was assessed using the corrected item total correlation. To determine how well the test measured learning following exposure to the curriculum, cross tabulations of correct/incorrect pre- and post-test item responses were computed. Finally, items were also assessed based on Haladyna's Guidelines for Developing MC Items.

Results: Problems identified during the analysis included: grammatical cues and other hints in the item stem that gave away the correct answer, implausible item options, and curriculum errors that led to poorly functioning test items. As a result of the item analysis, the test has undergone substantial revisions. The modified pre-test has been administered and baseline difficulty, discrimination and reliability have all improved. Complete results will be available in May, following administration of the post-test.

Conclusion: To ensure that tests measure learning, standards of item design should be followed. Item analysis can be used to identify poorly performing items.