267316 If you build it, they will answer: Web-based versus paper surveys and potential response biases amongst small practice providers practicing in urban under-served areas

Monday, October 29, 2012

Mandy Smith Ryan, PhD , Primary Care Information Project (PCIP), Division of Healthcare Access and Improvement, NYC Department of Health and Mental Hygiene, Long Island City, NY
Jason Wang, PhD , Primary Care Information Project, NYC Department of Health and Mental Hygiene, Long Island City, NY
Sarah Shih, MPH , Primary Care Information Project, New York City Department of Health and Mental Hygiene, Long Island City, NY
Chloe Winther, BA , Primary Care Information Project (PCIP), Division of Healthcare Access and Improvement, NYC Department of Health and Mental Hygiene, Long Island City, NY
Background: Low survey response rates compromise the generalizability of findings. Previous research suggests that providers are resistant to answer web-based surveys, and that, given the time it takes to develop and administer they might not be worth the time and effort.

Objective/purpose: To determine the feasibility of a web-based provider survey as an alternative to paper, and to determine whether there are systematic differences between providers who answer a paper versus a web survey.

Method: A survey to measure providers' satisfaction with tasks related to workflows prior to adopting an electronic health record (“EHR”) was administered to 644 small practice providers that went live on an EHR between 7/07 and 4/11. The invitation was sent via a URL-embedded email if available; otherwise a paper survey was mailed. 345 providers (54%) responded.

Results: Of those initially invited via email/web (67% of the sample), 30% responded via web, and an additional 27% responded via paper; 46% of the paper-invite group responded.

The average response time was quicker (5.2 days) for the web group compared to the paper group (28.4 days, p<.0001). Of those who were invited via email but later sent a paper survey, the response time was 44.5 days (p<.0001 compared to paper).

There were significant differences regarding comfort with computers. Namely, 44% of providers in the paper group were very comfortable sending a document to the printer compared to 58% of the web group, p<.01. 31% of paper-takers were very comfortable restarting the computer if it froze compared to 45% of web takers, p<.01.

Discussion/conclusions: Web-based surveys provide quicker response times but provider characteristics and experiences may differ for respondents by paper. Multiple modes of survey administration for providers of medically under-served communities should be considered when assessing use of HIT and their effectiveness for improving health care.

Learning Areas:
Conduct evaluation related to programs, research, and other areas of practice

Learning Objectives:
1) By the end of the session, the participant will be able to identify the challenges involved in conducting provider surveys. 2) By the end of the session, the participant will be able to explain the advantages and disadvantages of administering provider surveys via paper and/or web. 3) By the end of the session, the participant will be able to describe potential response biases involved when conducting provider surveys via web and paper.

Keywords: Providers, Urban Health Care

Presenting author's disclosure statement:

Qualified on the content I am responsible for because: I have administered many surveys, specifically provider surveys. Among my research interests is the methodological implications of survey design.
Any relevant financial relationships? No

I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.

Back to: 3397.0: Innovation in Technology: The Challenge of Health Outcomes Posters 3