Business-list data vs. ground observations - there's just no substitute for shoe leather in food-environment research

INTRODUCTION: Measuring the food environment through direct observation is resource-intensive and logistically challenging. A common solution is to use commercially-available business-list data. We sought to determine how a comprehensive dataset, by an industry leader in business lists, performs on the ground compared to direct observation. METHODS: Using a random sample of 150 street segments across the Bronx, researchers compared business-list data to direct ground observations. The main outcome measures were sensitivity, specificity, and positive and negative predictive value (PPV and NPV) of the business-list data for various types of food retail. To categorize food retail, researchers used Standard Industrial Classification (SIC) codes. RESULTS: The comprehensive business list had an overall sensitivity of 56.2%, specificity of 52.8%, PPV of 58.5% and NPV of 50.5%. Sub-groupings of food retail by SIC codes (e.g. “variety stores”, “grocery stores”, “specialty food stores”, and “restaurants”), showed better specificities and NPVs, with ranges from 83% and 80% respectively (for restaurants) to 94% and 97% respectively (specialty food stores). Sensitivities varied from 35% (grocery stores) to 68% (specialty food store). PPVs varied from 45% (grocery stores) to 68% (restaurants). CONCLUSIONS: Despite advertised completeness and accuracy of the business-list data, overall the data poorly represent the actual food environment on the ground, with especially poor sensitivity and PPV for grocery stores. Such data is wholly inadequate to substitute for direct ground observations, raising serious concerns about the validity of food-environment research using these kinds of datasets.