203379 Validation of the Connecticut bioterrorism nursing curriculum exam

Sunday, November 8, 2009

Lauren C. Babcock-Dunning, MPH , Edward J. Bloustein School of Planning and Public Policy, Rutgers University, New Brunswick, NJ
Linda Honan Pellico, MSN, PhD, RN , School of Nusring, Yale University, New Haven, CT
Amanda J. Durante, PhD, MSc , Yale Center for Public Health Preparedness, Yale University, New Haven, CT
Linda Degutis, DrPH, MSN , Yale Center for Public Health Preparedness, Yale University, New Haven, CT
Background: Four nursing schools in CT have taught a class on bioterrorism annually since 2006. Students in the class complete a pre- and post-test. The test appeared to have validity problems, since the pre-test scores were high and scores on the post-test did not appear to demonstrate learning. This project was undertaken to discover problematic test items (stems & options) and suggest improvements.

Methods: Statistical tests identified problematic items and principles of good test design were used to determine why items were performing poorly. To detect items that were too easy at baseline, a difficulty index was constructed. Individual item discrimination was assessed using the corrected item total correlation. To determine how well the test measured learning following exposure to the curriculum, cross tabulations of correct/incorrect pre- and post-test item responses were computed. Finally, items were also assessed based on Haladyna's Guidelines for Developing MC Items.

Results: Problems identified during the analysis included: grammatical cues and other hints in the item stem that gave away the correct answer, implausible item options, and curriculum errors that led to poorly functioning test items. As a result of the item analysis, the test has undergone substantial revisions. The modified pre-test has been administered and baseline difficulty, discrimination and reliability have all improved. Complete results will be available in May, following administration of the post-test.

Conclusion: To ensure that tests measure learning, standards of item design should be followed. Item analysis can be used to identify poorly performing items.

Learning Objectives:
1) Identify appropriate statistical methods for validating test items 2) Describe principles of good test design and their application to item validation

Keywords: Bioterrorism, Nursing Education

Presenting author's disclosure statement:

Qualified on the content I am responsible for because: tbd
Any relevant financial relationships? No

I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.