Data collection, analysis and graphics tools for dependent/clustered agreement data in early childhood dental caries

In many public health studies, raters or examiners grade the disease outcome in more than one site per person. Williamson et al. (2000) published methodology to estimate the kappa agreement statistic in nominal categorical data clustered within individuals, but the software to estimate cluster-adjusted kappas and their confidence intervals is not readily available. Bangdiwala (1987) conceived the agreement plot to extend Quade's pair chart (1973) and Friendly (2000) developed a SAS macro to render the agreement plot. However, the agreement plot displays concordance on the off-diagonal, which is the opposite of the traditional contingency table layout with concordance on the diagonal. Thus, we developed SAS macros to estimate cluster-adjusted kappas for nominal disease outcomes and to graphically display agreement on the diagonal. We illustrate the methods with calibration data from an early childhood dental caries prevention trial. We also have developed a flexible and customizable Flash application called the CAries Research INstrument (CARIN) software system (http://techtransfer.universityofcalifornia.edu/NCD/19030.html) to record dental caries, plaque, and periodontal status data for each tooth and tooth surface. The agreement chart uses colors corresponding to traditional dental caries charts used in CARIN (e.g. red for decay and blue for amalgam fillings). Clinicians and researchers have found CARIN to be easy to use and the revised agreement chart to intuitively explain agreement. Support: USDHHS/NIH/NIDCR U54DE019285 & R21DE08650.