AHCJ – Association of Health Care Journalists

07/05/2024 | News release | Distributed by Public on 07/05/2024 15:29

Hospital star ratings are about to be released. Here’s what journalists should know

U.S. Department of Health & Human Services headquarters in Washington, D.C. Photo by Sarah Stierch (CC BY 4.0)

Anyone who's been steered wrong by a Yelp rating can grasp the peril of summing up quality with a single data point. Yet that's what the federal Centers for Medicare and Medicaid Services does with its annual hospital star ratings, which are scheduled to be released July 31 via the Care Compare website.

The agency launched star ratings in 2016 with the aim of conveying hospital quality in an easy-to-interpret way for patients, and they generate plenty of attention and spin. Five-star hospitals boast about their scores, while 1 and 2-star hospitals explain how they're attempting to improve.

If you choose to cover star ratings, it's important to understand their limitations. Here are some key things to know, according to interviews with experts, government documents, and research reports.

Not every hospital gets rated

This year, CMS will give star ratings to 2,846 hospitals, down from 3,076 in 2023.

Star ratings are based on 46 quality measures in five groups: mortality, safety of care, readmission rates, patient experience, and timely and effective care. To get a star rating, hospitals must report at least three measures in at least three groups, including either mortality or safety.

Short-term acute care, Veterans Health Administration, and military hospitals are eligible. Specialty hospitals such as children's and psychiatric facilities are not.

Critical access hospitals, located in rural areas, may be rated but often don't treat enough patients to meet reporting thresholds.

Methodology can influence a hospital's score

Despite the government's rigorous process for developing measures, a 2022 study found that star ratings were highly sensitive to changes in the underlying methodology; in other words, tweaking how measures are assessed or weighted can affect a hospital's rating.

It's also worth noting discrepancies between CMS ratings and private rating systems that focus on different aspects of quality and use different scoring methods. A 2015 study found limited agreement between the star ratings and three other systems.

Star ratings aren't very informative for patients

By design, ratings are distributed on a bell-shaped curve. This year, 10% of hospitals have one star; 21% two stars; 29% three stars; 27% four stars; and 13% five stars.

But a 2021 study found that the distribution of scores is tightly clustered in the middle, with no hospitals being the best or worst at everything.

As a result, "there's not that much actual difference between even the top and the bottom of the distributions, and very little difference at all between adjacent star groupings," co-author David Nerenz, Ph.D., director emeritus of the Center for Health Policy and Health Services Research at Henry Ford Health in Detroit, said in an email.

Moreover, it found that star ratings don't correlate closely with performance on condition-related quality measures such as cancer or stroke care. A recent study published in JAMA Surgery highlighted the limitations of star ratings for choosing where to get surgery.

Some experts advise patients to examine measures specific to their care, ask a physician's advice on choosing a hospital, and consider a hospital's specialty certifications.

Star ratings are "just one of several sources of information to inform patient choice," Caitlin Gillooley, the American Hospital Association's director of quality & behavioral health policy, said in an interview.

A rating change can mean different things

This year, 22% of hospitals gained a star and 27% lost a star. Changes in stars can reflect better or worse performance or tweaks in the underlying measures.

Another wild card is CMS's practice of assigning hospitals to peer groups based on the number of measure groups they report - a proxy for hospital size. A hospital's rating can change if other hospitals in its peer group collectively perform better or worse or if its peer group assignment changes.

There's a big time lag

Data that go into star ratings are months or years old, owing to the need to collect robust sample sizes and process and validate measures. In fact, a small portion of data used to calculate this year's ratings predates the COVID-19 pandemic, diminishing their relevance for patients who are seeking care.

Some measures are susceptible to bias

Hospitals in marginalized communities can't control community factors such as homelessness, disability, and lack of access to follow-up care - which can affect patient outcomes measures such as mortality and readmission.

So far, CMS has not introduced social risk factors to its star ratings or their component measures, although the agency has taken steps to reward hospitals that provide high-quality care to underserved populations. One example is a "health equity adjustment" to CMS's Hospital Value-Based Purchasing Program.

Readmission measures exclude the use of observational stays, in which hospitals keep patients overnight while billing them as outpatients. Similarly, 30-day mortality measures do not count patients who die in hospice, which may advantage hospitals in areas where hospice care is readily available.

Progress may be on the horizon

If all this sounds discouraging, there may be hope for improvement as CMS switches to electronic clinical quality measures that extract granular data directly from electronic health records, replacing imprecise billing records and arduous manual reporting by providers.

Digital measures are expected to improve accuracy and expand quality evaluation to more aspects of care. They may also make public reporting more timely. In 2021, CMS announced a goal of completing the transition to digital measures by next year, but it's unclear when that objective will actually be met.

Resources