EP257: Rating the Raters of Hospital Quality, With Karl Bilimoria, MD, From Northwestern Medicine

In this episode, Stacey Richter talks with Karl Bilimoria, MD, surgical oncologist and VP of quality at Northwestern Medicine, about rating the raters of hospital quality.

WHAT YOU'LL LEARN

✅ What prompted Dr. Bilimoria and colleagues to "rate the raters" of hospital and physician quality

✅ Which major publicly reported hospital rating systems were evaluated, and how they compared

✅ The six criteria used to judge the rating systems themselves

✅ What conflicts of interest show up among these rating systems

✅ Dr. Bilimoria's advice to employers on how to pick a rating system, and why to triangulate the data

WHY THIS MATTERS

Karl Bilimoria set out to answer a question that had been nagging at Stacey after a personal experience visiting a "top-rated" hospital that had readily apparent quality issues: how good are the major hospital rating systems, really? His team evaluated systems like CMS Hospital Compare, Healthgrades Top Hospitals, Leapfrog Safety Grade, and U.S. News & World Report Best Hospitals against six criteria, uncovering meaningful differences in methodology and conflicts of interest. His bottom-line advice for employers and health care consumers: interpret any single rating system cautiously, and triangulate across multiple sources rather than trusting one "gold standard."

=== LINKS ===

🔗 Show Notes with all mentioned links: Episode Page

🔗 Healthcare Industry Acronyms and Terms

✉️ Enjoy this podcast? Subscribe to the free weekly newsletter

🫙 Support the podcast with a small donation to the Tip Jar

📺 Subscribe to our YouTube channel

🎤 Listen on Apple Podcasts

🎤 Listen on Spotify

=== CONNECT WITH THE RHV TEAM ===

✭ LinkedIn ✭ Threads ✭ Bluesky ✭ X

00:00 Introduction.

02:06 The impetus for Dr. Bilimoria deciding to rate the raters.

03:38 How high the stakes are when considering these ratings.

05:01 Breadth vs depth when choosing how and what to measure among the rating systems.

05:38 What rating systems the Rate the Raters looked at and why.

06:11 Who got the best scores as a rating system and who got the worst?

06:58 Dr. Bilimoria and the Rate the Raters' qualifications on rating these rating systems.

07:35 The methodology and criteria that the Rate the Raters came up with to evaluate these rating systems.

08:01 The six criteria that the Rate the Raters system uses to evaluate rating systems.

08:39 "At every step, we included the rating systems."

09:14 The intent behind Rate the Raters.

09:55 Why having grades is a positive for the health care consumer.

10:41 What conflicts of interest might be of concern among these rating systems?

12:22 "Notable notes" for these rating systems, and what each rating system incorporates or doesn't incorporate, and how these things affected their rating.

18:22 Creating a gold standard and finding ways to move the field forward.

22:05 Getting to better data in the short term and the long term by doing meaningful audits.

24:09 Individual ratings vs institutional ratings and where these intersect.

26:27 Dr. Bilimoria's advice to employers and how to pick a rating system.

27:45 "These rating systems should be interpreted very cautiously, even the best of them."

28:03 "Triangulate the data."

30:07 What Dr. Bilimoria and Rate the Raters are currently working on.