193397 Detecting relative bias using the Bland-Altman method: Consequences of unequal imprecision

Tuesday, November 10, 2009: 11:33 AM

John Baird, MSc , Department of Mathematics, Columbia University Teachers College, Austin, TX
Devices to successfully detect glaucoma are important tools for treating the disease; Bland-Altman plots are often used to determine the relative bias between two instruments. Bland-Altman assumes that the two instruments have equal imprecision but this strong assumption is often unchecked. This study demonstrates the consequences when imprecisions are unequal. Using simulated intraocular pressure (IOP) data for two devices (N=1000) four models illustrating possible scenarios were constructed. In the first model both methods have equal imprecision standard deviations and a bias between the devices increasing with each increase in the true level. The second had equal imprecision and no bias. The third had unequal imprecisions and no bias. The fourth had unequal imprecision and the same bias as the first model. In the two models that held the assumption of equal imprecision, the Bland-Altman method correctly identified the bias (P<0.0001) in the first model and the lack of bias (P=0.6534) in the second. In the third model, Bland-Altman incorrectly concluded there was a bias (P<0.0001). In the fourth model, Bland-Altman failed to detect the bias (P=0.7609). In cases where two devices have the same imprecision, Bland-Altman plots are effective tools for determining relative bias. When this assumption is violated, false positives and false negatives indistinguishable from true cases can occur. Researchers must strongly justify the assumption of equal imprecision when using Bland-Altman due to the high risk of erroneous conclusions. These mistakes can greatly hinder the development of effective glaucoma detection technologies.

Learning Objectives:
Describe the consequences of violating Bland-Altman assumptions when comparing two glaucoma detection devices

Keywords: Methodology, Technology

Presenting author's disclosure statement:

Qualified on the content I am responsible for because: For the last year I have researched and analyzed glaucoma related data with the GIG Group at the University of Pittsburgh Medical Center and worked with some of the top names in the field.
Any relevant financial relationships? No

I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.