Reliability and validity of a colorectal cancer screening questionnaire

Background: Accurate self-report measures of health-related behaviors are important because they frequently are used to estimate prevalence, monitor time trends, and measure outcomes in descriptive and intervention studies. Objectives: The primary objective of our study was to evaluate the validity of the National Cancer Institute self-report questions to measure colorectal cancer screening (CRCS) behaviors by comparing self-reports to medical record data for the total sample, by mode of administration (mail, telephone, face-to-face), and by different subgroups based on sociodemographic and other factors. With a sub-sample, we evaluated the test-retest reliability of the questionnaire using three time periods (2-weeks, 3-months, and 6-months). Methods: We collected validity data on 847 cases and controls and reliability data on close to 200 cases and controls for each time period. For each CRCS test, we computed estimates of concordance, sensitivity and specificity across all modes of administration, stratified by mode, and stratified by sociodemographic and other factors. We estimated kappa to evaluate test-retest reliability at each of the 3 time periods. Results: Specificity estimates for fecal occult blood test (FOBT), sigmoidoscopy (SIG), and colonoscopy (COL) were 0.85, 0.87, and 0.87. FOBT and SIG self-reports had lower sensitivity estimates (0.81 and 0.77) than COL (0.91). For FOBT self-reports, sensitivity was lower for mail respondents (0.73) compared with telephone (0.87) and face-to-face (0.83) respondents. For COL self-reports, the sensitivity estimate was higher for mail respondents. Kappa statistics indicated good agreement between initial and 2-week surveys. Kappa at 2-week follow-up was over 0.70 for all tests.