Objectives: The kappa statistic is a commonly used measure of agreement, or repeatability in epidemiological studies. Through the assessment of repeatability, epidemiologists can assess inter and intra-observer reliability of different procedures or instruments. However, the application of kappa statistics in certain situations is incorrect. In addition, statistical packages often make errors when computing kappa. The purpose of this study is to discuss and suggest methods to overcome the conceptual difficulty, as well as to discuss the computational difficulties that various statistical packages have. Methods: The kappa statistic is not appropriate for categorical variables measured at more than two levels when at least one of the possible categories is not chosen. This conceptual difficulty was illustrated using examples. Seven frequently used statistical packages were reviewed to see if they calculated kappa correctly. Of the seven packages reviewed, Stata, Systat, SAS, BMDP, and SPSS will each compute kappa. The Minitab and S-plus do not have this capability. Results: So long as the cross-table has no row or column with all zero elements, each of the five packages calculating kappa do so correctly. However, if there is a row or/and column with all zero elements, some statistical packages fail to calculate kappa statistics correctly. Conclusions: The conceivable approaches to overcome the conceptual difficulty are suggested. Users of this statistic should be aware of these potential problems which arise in the face of sparse data in the situation where there are more than two possible responses.
Learning Objectives: At the conclusion of the session, the participant in this session will be able to: 1. recognize the conceptual difficulty of kappa statistics. 2. list the approches to overcome the conceptual difficulty of kappa statistics. 3. discuss the certain circumstances that statistical packages claculate kappa statistics incorrectly.
Keywords: Statistics, Methodology
Presenting author's disclosure statement:
Organization/institution whose products or services will be discussed: None
I do not have any significant financial interest/arrangement or affiliation with any organization/institution whose products or services are being discussed in this session.