 Back to Annual Meeting
|
|
Back to Annual Meeting
|
APHA Scientific Session and Event Listing |
Abstract #152964
Audio (mp3) recordingMarc N. Elliott, PhD1, Elizabeth Goldstein, PhD2, William G. Lehrman, PhD2, Katrin Hambarsoomians, MS1, Laura Giordano, RN, MBA3, and Alan M. Zaslavsky, PhD4. (1) RAND, 1776 Main Street, P.O. Box 2138, Santa Monica, CA 90407, 310-393-0411, elliott@rand.org, (2) Division of Beneficiary Analysis, Centers for Medicare & Medicaid Services, 7500 Security Blvd., Mail Stop S1-01-23, Baltimore, MD 21244-1850, (3) Surveys, Research & Analysis, Health Services Advisory Group, 1600 E. Northern Avenue, Suite 100, Phoenix, AZ 85020-3933, (4) Department of Health Care Policy, Harvard Medical School, 180 Longwood Ave., Boston, MA 02115-5899
Research Objective: To determine the effects of mode of survey administration on patient responses to the HCAHPS (CAHPS Hospital) Survey. Study Design: Within each of 45 randomly sampled hospitals, 27,229 patients were randomized to four modes of survey administration: Mail Only, Telephone Only, Mixed Mode (mail with telephone follow-up), or Active IVR (interactive voice response, in which patients respond via telephone keypads). These patients completed the CAHPS (Consumer Assessment of Healthcare Providerslans and Systems) Hospital Survey, in which recently discharged patients evaluate aspects of their hospital care. Population Studied: English- or Spanish-speaking adult patients with non-psychiatric primary diagnoses discharged alive after at least one overnight stay in a general acute care US hospital with at least 1200 annual discharges. Principal Findings: Linear regression was used to model each HCAHPS outcome (2 overall ratings and 7 composites) from fixed effects for mode, hospital identifiers, and patient characteristics. Substantial and statistically significant (p<0.05) mode effects were found for both overall ratings and 4 of 7 composites (Cleanliness & Quiet, Responsiveness of Hospital Staff , Pain Management, and Discharge Information); mode effects were smaller for the 3 composites involving communication (with doctors, nurses, and about medications). Patients provided more positive evaluations in the Telephone Only and Active IVR modes than in the Mail Only and Mixed modes. Differences between Telephone Only and Active IVR responses and between Mail Only and Mixed Mode responses were small. The Telephone Only and Active IVR scores for the 6 most affected outcomes noted above were at least 0.4-0.5 hospital-level standard deviations higher and sometimes as much as 1 standard deviation higher than scores from Mail Only and Mixed modes. These mode effects varied little by hospital. Response rates varied strongly by randomized mode (p<0.0001), ranging from 41.2% for Mixed Mode to 20.7% for Active IVR; these patterns were consistent across hospitals. Conclusions: Because a hospital's choice of vendor or survey mode may be confounded with factors related to underlying quality, an external mode experiment is necessary to estimate mode effects for subsequent fieldings of the survey. We conclude that adjustments for mode effects are necessary to make the reported HCAHPS scores comparable; in the absence of such adjustments a hospital that would have ranked at the 50th percentile in the Mail Only mode would be ranked at the 66th to 84th percentile in the Telephone Only mode for a majority of outcomes.
Learning Objectives:
- Describe the effects of survey mode on patients evaluations of hospital care
- Describe the procedure used by CMS to adjust the HCAHPS survey for survey mode
- Select sample sizes adequate for patient evaluation of hospitals
- Compare trends in patient surveys even when survey mode changes.
Presenting author's disclosure statement:
Any relevant financial relationships? No
Any institutionally-contracted trials related to this submission?
I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.
The 135th APHA Annual Meeting & Exposition (November 3-7, 2007) of APHA