Hospital ratings, decoded

Topics: Patient Populations, Providers, Hospitals, Federal Government

By: Rachel Schulze, associate editor

Hospital ratings gets lots of buzz in the news, but do you know what factors actually determine a hospital's rank?

We dove into four of the most popular lists—from CMS, Leapfrog, U.S. News & World Report, and Truven—to find out how they work and why they're so controversial.

1) CMS Hospital Compare Star Ratings

CMS unveiled its inaugural Hospital Compare Star Ratings in July 2016 to help Medicare patients select a hospital based on quality performance.

The ratings cover seven categories of metrics:

Effectiveness of care;

  • Efficient use of imaging;
  • Mortality;
  • Patient experience;
  • Readmission;
  • Safety of care; and
  • Timeliness of care.

Mortality, readmission, safety of care, and patient experience are worth 22% each, while the other three categories are worth 4% each.

CMS this year updated its methodology, causing a greater share of hospitals to fall at the ends of the bell curve and receive a 5- or 1-star rating.

In the latest list, released in December 2017, CMS reviewed 3,692 hospitals and awarded:

  • 337 (9.1%) five stars;
  • 1,155 hospitals (31.3%) four stars;
  • 1,187 (32.2%) three stars;
  • 753 (20.4%) two stars; and
  • 260 (7.0%) one star.

2) Leapfrog Group Hospital Safety Grade

The Leapfrog Group's Hospital Safety Grade examines how hospital perform on 12 process/structural measures, such as intensive care unit physician staffing, and 15 outcome measures, such as hospital-acquired infections. Each domain makes up 50% of a hospital grade.

Leapfrog, a private nonprofit, developed the ratings to encourage patients to assess hospital safety and encourage patients to use that information in hospital selection.

In the latest list, released in October 2017, Leapfrog reviewed 2,632 hospitals and awarded:

  • 832 As;
  • 662 Bs;
  • 964 Cs;
  • 159 Ds; and
  • 15 Fs.

3) U.S. News & World Report Best Hospitals

The annual U.S. News & World Report Best Hospitals list focuses on specialty care performance, ranking eligible hospitals' performance on up to 16 specialties.

U.S. News assesses 12 specialties based on four categories of metrics:

  • Outcome, worth 37.5%;
  • Structure, worth 30%;
  • Process, which includes the results of a reputation survey, worth 27.5%; and
  • Patient safety, worth 5%.

U.S. News ranks the four other specialties based solely on reputation.

In the latest list, 152 hospitals scored high enough to be included in the rankings. In addition, U.S. News recognizes the top 20 performers across the specialties on its Honor Roll.

4) Truven Health 100 Top Hospitals and 15 Top Health Systems

Truven Health's Top 100 Hospitals Rankings look at hospital quality as well as financial performance.

For the rankings, Truven examines hospitals within several groups:

  • Large community;
  • Major teaching;
  • Medium community;
  • Teaching; and
  • Small community hospitals.

The rankings include about 20 hospitals in each category.

The Top Hospitals ranking assess:

  • Extended outcomes, worth 20%;
  • Inpatient outcomes, worth 20%;
  • Process efficiency, worth 20%;
  • Cost efficiency, worth 10%;
  • Financial health, worth 10%;
  • Patient experience, worth 10%; and
  • Process of care, worth 10%.

The Truven Health 15 Top Health Systems aggregates information on individual hospitals into system-wide assessments. Truven ranks the top five small systems, top five medium systems, and top five large systems.

For the Top 15 Health Systems, Truven looks at:

  • Efficiency, worth 33%;
  • Extended outcomes, worth 22%;
  • Inpatient outcomes, worth 22%;
  • Patient experience, worth 11%; and
  • Process of care, worth 11%.

Why do hospitals rate so differently on different reports?

While the different ratings systems might seem similar on the surface, a 2015 paper that looked at four ratings systems—Leapfrog, U.S. News, Consumer Reports, and HealthGrades—found none of the 83 hospitals that all four raters reviewed were considered a high performer by multiple raters.

Further, the authors also drew attention to differences in risk-adjustment methods, finding, for instance, that 43% of hospitals that had below-average mortality by one risk-adjustment method had above-average mortality by another.

"These contradictions have created challenges for stakeholders concerned with hospital quality," the authors wrote. They continued, "For patients, differences across hospital ratings add complexity to ascertaining a hospital's actual quality. For payers, conflicting ratings make it difficult for them to recognize and reward hospitals for high quality. For hospital leadership, differences across rating systems complicate decisions regarding the focus of their improvement efforts."

Further, teaching hospitals and safety net hospitals take a higher share of sick and low-income patients and tend to perform relatively poorly in CMS Star ratings, which penalize for higher rates of readmission, Modern Healthcare reports.

J.B. Silvers, a professor of health care finance at Case Western Reserve University, told STAT in 2016, "Big teaching hospitals are taking care of the most complex cases. Hardly any of them are in the five-star category."

But even with the confusion, some say consumers can still use ratings to make informed decisions.

In an interview with WBUR, Ashish Jha, a professor at the Harvard T.H. Chan School of Public Health, said, "For most people, the number one [concern] is having a good outcome. If you're really sick, you don't wanna die. You wanna be treated with dignity and respect. You can go track that kind of data down for hospitals—that's out there, it's available."

Ratings matter—a lot

For hospitals, a lot can ride on a particular rating.

Saint Anthony Hospital in Chicago filed a lawsuit against the Leapfrog Group last fall, alleging the organization lowered the hospital's grade based on data Leapfrog knew to be inaccurate. The hospital in the suit said the lower score could hurt its reputation in the community, threaten its bottom line, and compromise potential partnerships.

Leapfrog responded to the lawsuit by saying the hospital previously had affirmed the data it submitted was correct and that the hospital failed to raise the issue during a post-submission review period, Mark Hagland reports for Healthcare Informatics. Leapfrog also said it doubts that the data in question is solely responsible for the lower grade.

Jha, who was on the committee that helped set standards when Leapfrog was established, told Kaiser Health News' Jenny Gold. "In highly competitive markets, hospitals are likely to see poor grades as a challenge, and I think many will be tempted to sue the rating agencies."

Jha added that he views hospitals reacting to the data as a positive. "If they're going to use that as motivation to get better, that's perfect. As a patient, you don't care why a hospital is investing in safety, you just care that they are."