Online Program

Do website reviews of internists inform patients with diabetes or hypertension about the quality of care they would receive from these physicians?

Monday, November 4, 2013 : 12:30 p.m. - 12:50 p.m.

Bradley Gray, PhD, MS, MS, Evaluation and Research, American Board of Internal Medicine, Philadelphia, PA
Jonathan Vandergrift, MS, Research and Evaluation, American Board of Internal Medicine, Philadelphia, PA
Gordon Gao, PhD, Robert H. Smith School of Business, University of Maryland,
Jeffrey McCullough, PhD, School of Public Health, Univerity of Minnesota
Rebecca Lipner, American Board of Internal Medicine, American Board of Internal Medicine, Philadelphia, PA

Assess the accuracy of patient ratings of physicians obtained by internet searches.


Physician quality measures (QMs) were drawn primarily from patient charts for 1,042 internists who completed a diabetes or hypertension practice improvement module (PIM) (08/01/2011-11/12/2012). PIM completion is part of the American Board of Internal Medicine's (ABIM) maintenance of certification process and included abstracting 25 charts and collecting 25 patient survey responses for patients with the applicable chronic condition.

Chart PIM-QMs applicable to both patient cohorts included: share of patients with blood pressure/LDL controlled and providing smoking cessation advice. Patient survey PIM-QMs included patient assessment of care quality/physician self-care support as very-good or excellent. We applied an expert panel based PIM-QM composite, comprising mostly chart-based QMs. PIM-QMs mirror quality measures applied by NCQA/CMS.

We mimic consumer searches by entering each internist's name and location into a Google search, extracting ratings from the first two health-websites on the search list. Searches captured 1,013 ratings for 602 internists (58% of our sample) from eight websites. We normalized ratings by dividing each rating by that website's maximum score (e.g., 2 out of 5 stars yielded a normalized rating of 40%). We categorized the normalized website rating as top score if >=80% (59% of ratings) and bottom score if <=0% (12% of ratings).


We estimated the relationship between internists' PIM-QMs and having any website rating using probit regression. PIM physician QMs were evaluated as dependent variables using binomial-regression (Logit-link) where a website rating was the explanatory variable controlling for sub-specialization, PIM, website. We accounted for physicians being rated on multiple websites.


Our analysis indicated that patients with diabetes/hypertension obtain limited information from internet searches regarding which physicians to avoid (bottom rating associated only with patient share with LDL controlled (p-stat<.05)) but not whom to choose (normalized rating or top rating was not associated with any PIM-QM, only 6 ratings per website). However, ratings from the most popular website (HealthGrades (n=465)) yielded information regarding the overall quality of physicians (normalized rating was a significant positive predictor of PIM-QM composites (p-stat<.05)). Yet, more information about quality would have been revealed had ABIM patient ratings been accessible (PIM-QM patient reviews significantly correlated with PM-QM clinical composite (p-stat<.001) and these associations were significantly larger than website ratings associations (p-interaction<.01).


Consumers obtain limited physician quality information from website searches and yet there exists a great potential for patient reviews to aid consumers.

Learning Areas:

Basic medical science applied in public health
Chronic disease management and prevention
Communication and informatics
Public health or related public policy
Public health or related research

Learning Objectives:
Assess whether ratings of physicians obtained by internet searches by consumers relate to physician quality.

Keyword(s): Quality of Care, Hypertension

Presenting author's disclosure statement:

Qualified on the content I am responsible for because: I was responsible for the original research idea and took the leading role in all analysis for the study.
Any relevant financial relationships? No

I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.