Validity and Reliability of the Direct Observation Methodology: A Focus on Ohio Local Public Health

The purpose of this study is to examine the reliability, validity, and acceptability of a novel direct observation methodology to examine the role of local health departments (LHDs) in prevention of foodborne illness. Trained public health student observers will shadow Registered Sanitarians (RS) during food inspections completing an observational form that will be the main instrument used in the analysis. Methods: Inter-rater reliability will be assessed 1) using shared viewing of an inspection video (25 responses); and 2) through student observers functioning in pairs during early observational sessions, jointly observing the same inspections (anticipate a total of >120 observations). In addition, RS and student observers will be interviewed to determine their perceptions of the intrusiveness, effectiveness and accuracy of the direct observation methodology. Inter-rater reliability will be examined for the observations through the use of Cohen's kappa to assess observer consensus on all observations. Interviews will be transcribed and analyzed for themes. Results: It is anticipated that there will be strong concordance on most observations, with higher levels of variability guiding changes in the observational protocol. We expect analysis to show that RS and student observers perceive the direct observation methodology as useful, unobtrusive, and accurate. Analysis will also examine the level of professional behavior among observers; whether observational bias occurs, causing changes in RS behavior and whether error variation is reduced through direct observation. Conclusion: Direct observation is a novel methodology in the public health setting and its validity and reliability are crucial for its future application in research.