Technical Standards for Policy Surveillance: Delphi Process and Technical Guide

We convened a committee of experts to build consensus on technical standards for policy surveillance. This committee assisted in proposing standards through a Delphi process. Topics covered included quality control, resources, content expertise, and the publication of legal datasets. The Delphi process unfolded over two rounds of independent assessment followed by a discussion of consensus results.  

The technical standards identified by the committee were used to develop a technical guide for policy surveillance, which details the process of collecting and measuring the law and making the resultant data available for public use. The guide articulates a process for systematic research and collection of the law including redundant research, consultation with experts and creation of detailed research protocols. Development of a coding scheme and coding of legal texts is addressed in several chapters that stress the iterative nature of coding and the need to apply stringent quality control measures including redundant coding, calculation of reliability measures and daily review of coding and the coding scheme. Finally, the technical guide outlines a process for publication of the data, and a detailed process for updating and ongoing measurement, ensuring it remains relevant and useful in analysis and evaluation.  This presentation reviews the Delphi process and its results, and explains key features of the process as set out in the technical guide.