Differing Psychometric Properties of an HIV Knowledge Scale Administered With Populations At High Risk for HIV Infection

Community-based HIV prevention program managers commonly use self-administered paper-and-pencil surveys and a pre-/post-test design to measure intervention effectiveness. These surveys routinely include questions about participants' knowledge of what puts them at risk for HIV infection, based on the theory-driven principle that knowledge is protective against HIV infection. Program managers who oversee large-scale (i.e., city-wide) HIV prevention programs that serve multiple high-risk populations often use the same survey instrument with all program participants. While this approach is advantageous for easier analysis and reporting, it could mask important variations in the survey's ability to reliably measure HIV knowledge across all of the program's priority populations. To assess the presence of this potential threat to the measurement of HIV knowledge in large-scale programs that use a single survey, psychometric analyses were conducted on program evaluation data that were collected as a part of a city-wide community-based HIV prevention program conducted in a Midwestern urban area from 2005-2009 (n=5,027). As expected, the analyses revealed important variations in the ability of the survey to measure HIV knowledge among high-risk priority populations. By sexual orientation, the scale performed better for heterosexual males and females combined, worse for Lesbians, but best overall for participants who identified as Transgender. By race/ethnicity, the scale performed better for Hispanics but worse for African Americans. By age group, the scale performed best for 50-59 year olds and worse for 19-24 year olds. Potential reasons for these differences and practical strategies to address them will be presented.