Balancing anonymity and analytic value in surveys of patients in vulnerable populations: The mathematics and logic of selecting and coding variables to control risk while maximizing information from CAHPS surveys in a large urban Medicaid health plan, 2008-2013

Public agencies and health plans rely heavily on live surveys of anonymous patients to assess the quality of services. Anonymity protects patients, but also limits the ability to identify and fix sources of poor health care.  Conventional approaches restrict data by type -- severely aggregating key analytic variables such as provider group or medical condition; and restrict important demographics such as ethnicity, and geographical area (needed in analysis because “where you live affects your health and well-being”).  This paper reports rational methods to calculate and limit risk to patients in limited (but still usable) data sets.

(a) Importance of tradeoffs between anonymity and actionability: Anonymity protects patients from overt retaliation or inadvertent harm from providers they are rating. But anonymity limits the ability to drill down in the data to target corrective actions or address disparities. 

(b) Applied example: This paper reports implications of common anonymity rules for surveys in a large and diverse community-affiliated Medicaid health plan from 2008 to 2013, evaluating services in a complex, dispersed provider network.  Analytically important member and provider characteristics are discussed, with methods to render those variables safer.   

(c) Quantitative and qualitative factors: The presentation explains the calculation of the risk of breaching anonymity, noting examples of misapplied or ambiguous anonymity rules.  Qualitative factors (types of risks, degrees of anonymity) are also discussed as parameters in choosing which demographics to request in anonymous datasets.  Methods are briefed for single surveys; and adaptions for multi-year survey databases used in continuous quality improvement (CQI).