An Examination of the Internal and External Validity of Health Risk Assessment Instruments used in the Senior Risk Reduction Demonstration under Medicare

The Senior Risk Reduction Demonstration (SRRD) is an RCT that tests whether health promotion programs that have been developed in the private sector can be tailored for Medicare beneficiaries. Participating vendors use health risk assessments (HRA) to identify health risk factors and customize participants' wellness and prevention programs.

As part of the evaluation of the SRRD, we examined the internal and external validity of the HRA instruments. Internal validity was assessed by testing whether there was a negative relationship between self-rated health status and obesity, depression, unhealthy diet, stress level, tobacco use, and lack of physical activity. To examine external validity we compared the HRA data to national benchmarks, including ACS and BRFSS. Also, we linked the HRA data to Medicare claims and examined the concordance of the data sources.

The internal validity tests indicate that HRAs accurately capture the dimensions they seek to measure: self-rated general health status was negatively correlated with the variables listed above. Similarly, the HRA and Medicare claims are concordant for race/ethnicity, and HRA risk measures are positively correlated with total Medicare expenditures. Chronic condition rates are higher for the HRAs than for claims. Based on comparisons of the HRAs to national benchmarks, SRRD participants were more likely to be White and overweight or obese, but were similar in their tobacco use.

The results of the internal and external validity tests indicate that HRAs capture information about Medicare beneficiaries' health risk factors and health status that can be used to create an individualized wellness intervention.