200970 Composite Score Methods to Evaluate Part D Organization's Performance over multiple Measures

Tuesday, November 10, 2009: 9:30 AM

Ying Wang, PhD , IMPAQ International, Columbia, MD
Sungsoo Oh, MS , Centers for Medicare & Medicaid Services, Baltimore, MD
Arthur Kirsch, PhD , IMPAQ International, Silver Spring, MD
Christopher Powers, PharmD , Center for Strategic Planning, Centers for Medicare and Medicaid Services, Baltimore, MD
Oswaldo Urdapilleta, PhD , IMPAQ International, Columbia, MD
Vikki Oates , Centers for Medicare & Medicaid Services, Baltimore, MD
CMS recently launched its newest effort in providing information to beneficiaries through the use of a star rating system which evaluates Part D contracts' performance. This change includes using a composite score to assess performance over multiple measures, which has been a widely used approach to empower individuals' choice. The specific goal of this study is to identify an appropriate grouping structure to sort single measures into certain domains and generate a composite score for each domain as well as summary score across domains. The study population is all renewing Part D sponsors: 80 stand-alone prescription drug and 569 Medicare Advantage contracts that provided drug benefits in CY2008. We studied 19 performance measures, drawn from multiple data sources, including CMS administrative data and consumer survey data. An extensive literature-review on grouping methods was conducted prior to selecting the most relevant grouping methodologies. Results from principal factor analysis and cluster analysis methodologies were compared for grouping patterns, and relevant sensitivity analyses to adjust factor rotation models, factor number, and significance level were conducted. Different techniques generated similar grouping patterns and not all measures were necessarily loaded into a three- and four-factor structure (the predefined grouping was up to four domains). Coupling the statistical outcomes with each measure's practical business logic generated a more meaningful domain structure. In addition, a coefficient, based on the relative variance of individual measure stars, was used to adjust plans' summary scores. This approach addressed the central tendency of the summary score based simply on the mean.

Learning Objectives:
Define the meaningful approach to group Medicaer Part D performance measures/variables into domains to reduce the infomation for Medicare beneficiareis. Demonstrate how the statistical method can be used in the field of performance measurement. Evaluate the alternative grouping methods. Explain the selection process among multiple alternative methods. Assess the values of incorporating the non-statistical factor in the method evaluation and selection process.

Keywords: Performance Measures, Statistics

Presenting author's disclosure statement:

Qualified on the content I am responsible for because: I received a doctoral degree in public health with a concentration in program evaluation and analytic methods. I am currently supporting CMS on a project to develop Part D plan star ratings and this presentation is developed based on this project.
Any relevant financial relationships? No

I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.

See more of: Who gets Health Insurance? A Statistical Perspective
See more of: Statistics