270067 New uses for not-so-old survey data: Pooling annual CAHPS data via Cochran-Mantel-Haenszel test, comparing actionable sub-populations of patients in a large urban Medicaid health plan

Tuesday, October 30, 2012

Earl Leonard III, MS , Healthcare Outcomes & Analysis, L.A. Care Health Plan, Los Angeles, CA
S. Rae Starr, MPhil, MOrgBehav , HealthCare Outcomes & Analysis, L.A. Care Health Plan, Los Angeles, CA
  
Each year, government agencies, employers, and accreditation organizations use CAHPS and related surveys to get representative assessments about the quality of services received by millions of patients. CAHPS samples compare and rate health plans, but are rarely large enough for drilldown to actionable causes and targetable subpopulations of members and providers.

Pooling survey data across years is desirable to get the benefit of larger samples. Health plans often have several years' worth of CAHPS data, gently aging on the shelf. Simple pooling can overstate p-values by implying that the pool is a single large sample. This presentation will discuss a familiar technique for calculating adjusted p-values, the Cochran-Mantel-Haenszel test, in concert with tests to confirm homogeneity of odds ratios.

The presentation will discuss:

(1) Applications: Drilldowns not feasible without a valid method for pooling (a) demographics (age, gender, ethnicity); (b) provider groups or clinics to target interventions; (c) disease cohorts; (d) geographical areas for pilot programs.

(2) Setting: Applied statistical consultations for decision support at a large and diverse urban Medicaid health plan in the southwestern United States. Discussion of value for analysis of services to vulnerable populations.

(3) Application to decisions: Case examples of previously intractable policy questions facing management: Comparisons of performance between contracted healthcare providers.

(4) Extensions: Potential uses in other healthcare quality measurement applications (HEDIS, provider satisfaction surveys, access-to-care surveys).

(5) Implications: Pooling appropriately allows testing for root causes of substandard healthcare services to guide the design of interventions.

Learning Areas:
Biostatistics, economics
Conduct evaluation related to programs, research, and other areas of practice
Implementation of health education strategies, interventions and programs
Planning of health education strategies, interventions, and programs
Program planning

Learning Objectives:
1. Describe why pooling of conventional, agency-compliant CAHPS data is necessary for causal analysis. 2. Discuss the types of questions and decisions that are difficult to pursue with single-year survey samples of patients. 3. Identify common independent variables for which surveys of patients are consistently underpowered. 4. Describe a step-by-step approach for appropriately pooling and testing archival survey data about health care quality. 5. Discuss the interpretation, reporting, and caveats necessary in using pooled analysis on survey data.

Keywords: Member Surveys, Quality Improvement

Presenting author's disclosure statement:

Qualified on the content I am responsible for because: I am employed as a Senior Biostatistician at L.A. Care Health Plan – the Local Initiative Health Authority of Los Angeles County, California. L.A. Care is a public entity competing with commercial insurers in the Medicaid and S-CHIP markets in L.A. County. I participated in the design, analysis, interpretation, and writing of the reported findings. I'm primarily responsible for deciding to use CHM as the analysis tool for pooling trended CAPHS results.
Any relevant financial relationships? No

I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.

Back to: 4178.0: Statistical Poster Session