Inter-reader agreement in ILO readings of radiographs for pneumoconiosis - can anything new be added to this?

The ILO system for classification of radiographs for pneumoconiosis has been accepted for use in occupational health studies. NIOSH recommends using multiple ILO trained readers to increase accuracy and precision of readings. The significance of between-reader agreement on the estimates of prevalence of ILO abnormalities in nuclear weapons workers has not been studied well. We screened 757 workers with a chest x-ray as part of the U.S. DOE Former Nuclear Weapons Worker Medical Screening Program. The films were reviewed by three experienced ILO readers and the agreement was found to range from moderate to substantial for both parenchymal (simple kappa k=0.57;95%CI 0.47-0.67 for reader 1vs.2, to k=0.67;95%CI 0.59-0.76 for 1vs.3) and pleural abnormalities (k=0.53;95%CI 0.41-0.64 for 1vs.3 to k=0.61;95%CI 0.50-0.72 for 1vs.2) expressed as dichotomous outcomes. The agreement in ordinal profusion scoring was substantial between all readers (weighted kappa k=0.68; 95%CI 0.57-0.78 for reader 1vs.2; to k=0.72; 95%CI 0.64-0.80 for 1vs.3). We found 5.9% workers with parenchymal abnormalities defined as small opacities profusion score=>1/0; 4.9% with isolated pleural and 2.5% with coincident parenchymal and pleural abnormalities. These prevalence estimates were based on the median ILO score and are higher (for parenchymal) and lower (for pleural and parenchymal and pleural) than those reported previously for other DoE workforces. This pattern of ILO readings could reflect an effect of age, increased sensitivity by non-B readers or a true high rate of pneumoconiosis. A sample of films is being reviewed by independent ILO B-readers for quality assurance purposes and results will be reported.