141st APHA Annual Meeting

In This section

289966
How consistent is the reliability of hospital performance based on simulation and multilevel approaches?

Monday, November 4, 2013 : 9:30 AM - 9:50 AM

Tsung-Tai Chen , Department of Public Health, Fu Jen Catholic University, New Taipei, Taiwan
Objective: Many studies have used the hierarchical logistic model to estimate the reliability of provider/insurer performance. The shrinkage factor for the hierarchical model is assumed to be normally distributed with and the assumption may be incorrect. In this study we compare the reliability of individual diabetes mellitus (DM) process measures based on simulation and hierarchical logistic approaches. Data Source: This study utilizes data from Taiwan's National Health Insurance (NHI) database for 2008. Our DM sample size is 1,276,114. Methods: We derive the individual measures from 5 evidence-based process measures. The reliability index is defined as the ratio of the true score variance to the observed score variance. To estimate the reliability of the simulation approach, we compute the two variances above from 1,000 bootstrap samples (replicates). To estimate the reliability of the hierarchical logistic model, we apply the hierarchical logistic model to estimate the true score variance and then use the Spearman Brown Prophecy Formula to estimate the reliability index. Principal Findings: The reliability estimated from the hierarchical logistic model almost has a slightly lower reliability, especially for cases with smaller intraclass correlations (ICC). For example, the Annually Serum Creatine Test has the lower ICC (3%), and thus its reliability estimated by hierarchical model (89%) has more deviance from the reliability (97%) estimated from the simulation methods. The deviance for the Annually Serum Creatine Test is larger than the Annually LDL Test, which has the higher ICC (14%). Conclusions: Our analysis has shown that the reliability derived from the simulation method is almost consistent with the reliability estimated from the hierarchical logistic model. However, when using the hierarchical logistic model to estimate the reliability of provider performance one should be aware of its limitations, especially for providers with smaller ICC.

Learning Areas:
Public health or related research

Learning Objectives:
Compare the reliability of individual diabetes mellitus (DM) process measures based on simulation and hierarchical logistic approaches

Keywords: Medical Care, Diabetes

Presenting author's disclosure statement:

Qualified on the content I am responsible for because: Among my scientific interests has been the quality composite score
Any relevant financial relationships? No

I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.

Back to: 3054.0: Quality of in-patient care