Measuring Quality: Promising, Needs More Progress

Dec. 16, 2014

Topics: Clinical Quality

by Joshua Zeitlin, staff writer

The U.S. spends about $2.8 trillion annually on health care, but as a country we aren't particularly sure how well that money is being spent. What we do know is that the U.S. spends far more than any other country but generally does not have better outcomes.

The Institute of Medicine defines quality care as that which is "safe, effective, patient-centered, timely, efficient and equitable." While the Affordable Care Act encourages providers to deliver quality care -- through a variety of ways, including value-based payments -- there is still an unanswered question of how that quality is actually measured.

"If we actually want to pay for value, which is a buzzword everybody loves saying, then we have to decide how to measure value, and it's not trivial, but it's not that hard," Ashish Jha, professor of Health Policy at the Harvard School of Public Health and a practicing internal medicine physician at the VA Boston Healthcare System, said.

On the whole, experts agreed that measuring care quality has improved significantly over time, but there still is a long way to go.

"Measuring quality is not harder than putting a man on the moon. It just isn't. It's that putting a man on the moon was a priority for the government and so we got it done," Jha said.

Casting Too Wide a Net

A number of experts said payers -- both government and private -- are casting too wide a net when determining what metrics to include in quality measurement. Princeton University health care economist Uwe Reinhardt was particularly critical of the federal government, which he said aims for "a degree of perfectionism that guarantees failure. ... The mental illness when you unleash bureaucrats at HHS into a room, they always shoot for the moon, when in fact, just going to San Francisco is good enough."

Reinhardt added, "It's much better to say, 'What are sort of the 20 things that can really go wrong with quality in a hospital?' And those 20 things, how can we capture them in some metric that's actually robust, reliable and so on? And start with that, something simple, that is not too expensive. You don't always, at the beginning, have to be so comprehensive."

Michael Chernew, a professor in the Department of Health Care Policy at Harvard Medical School, suggested that payers could start with a more limited set of metrics and then "ask for the more detailed information only as you begin to get indications folks might not be" providing quality care.

Another issue is that some providers -- particularly primary care physicians -- are inundated with having to report different measures to different payers. "Having standardized measures is really crucially important," Margaret E. O'Kane, president of the National Committee for Quality Assurance, said, adding, "Not only does [not having standardized measures] it make life very difficult for those physicians and those practices, but it really dilutes the signal." She said "it feels like a lot of the performance" on improving quality in recent years "is kind of flat, and I think that's part of the reason."

Several experts said that there are times when it is appropriate for different payers to ask for different quality metrics, particularly when considering different patient populations. However, providers get frustrated when they are asked "to report six different diabetes metrics for different payers that are basically measuring the same thing," according to the Center for Health Care Strategies' Rob Houston.

The Connection Between Risk Adjustment and Quality Measurement

Another factor complicating quality measurement is that the science of risk adjustment is still developing. Risk adjustment aims to make sure providers are not being penalized for caring for a larger number of sick patients, who typically will have worse health outcomes than healthier patients.

The system right now is fairly good at adjusting for risk in some areas, such as mortality rates, according to Jha. However, he argued that for many outcomes, including hospital readmissions, the risk adjustment models are not up to par. That has resulted in payers "penalizing hospitals that disproportionally care for a lot of sick patients because our models don't do a very good job of accounting for it," Jha said.

Jha suggested that payers should account for socioeconomic status when assessing care quality, a view he said about 80% or more of the health care policy community shares, with CMS being a major exception. Jha previously disagreed with such adjustments because he thought they would result in "giving credit to hospitals or organizations that provide lousy care to poor patients," but he has since changed his mind.

Jha pointed out that lower-income individuals have more difficulty obtaining transportation to and from medical appointments and face other issues, such as less access to healthful food, factors that can end up having "a huge effect and influence on whether you are going to end up back in the hospital or not."

However, Jha said there are two instances when adjustments shouldn't be made for socioeconomic status:

  • On process measures, as such measures largely are under a provider's control; and
  • Unless "there is some conceptual model that you can walk through about why socioeconomic status ought to matter," as is the case with social supports and readmissions.

Experts Identify 'Good' Quality Measurement Practices

Many observers understand the complexity of developing an effective system for measuring quality and the weaknesses of the current system. But what makes a "good" quality measure? The experts American Health Line spoke with said a quality measurement must:

  • Demonstrate a link to improving quality;
  • Be something that patients and clinicians care about;
  • Correlate with something that providers generally can control;
  • Be scientifically valid and reliable, including having a sufficient sample size; and
  • Be calculated at the physician level, possibly with secondary attribution, when possible and appropriate. 

Need More Focus on Patient Experience, Patient-Reported Outcomes

Several experts said it is also important to include patient experience and functional outcomes measures.

J. Michael McWilliams, an associate professor of health care policy at Harvard Medical School and a practicing general internist at Brigham & Women's Hospital, said patient experience is "an area of quality where we happen to have good measures and it's very important."

He added, "Given that a lot of what we do in medicine ... may not alter a patient's outcome, it's really important that part of medicine is teaching a patient about their diagnosis, expecting what to come, helping them through a sickness and making sure that they have timely access to care and the information that they need to deal with their illness."

Meanwhile, Anne-Marie Audet, Commonwealth Fund vice president for delivery system reform & breakthrough opportunities, said that "quality that is defined by people and what they expect of the health care system" is "what's missing most in our approach today." Audet called for more focus on functional measures, such as whether patients were able to resume their activities as they expected or whether they had complications six months after a procedure.

Audet said it also would help if more systems were to gain control over their own data to have a better grasp on how they are performing on quality and to improve care coordination. That way, providers are not "depending on claims data or reports from the payer or the health insurance to know what [they're] doing," she said.

Experts' Optimism for the Future

Nearly every expert expressed at least some optimism about the future of encouraging and measuring care quality.

Chernew said, "We're at the beginning of a relatively long process to do a better job, and there's a lot of things you could find to criticize about measures, but for the most part I think there's a lot of folks working in the area, they tend to be good folks, and for the most part we're moving in the right direction."

Reinhardt DropquoteReinhardt said, "You have thousands of bright young people working on it. It's just, initially, no one ever paid the geeks, they had to beg for money. But the young people are doing yeomen's work, and I'm very impressed with it. When I look at what happened in quality measurement in the last 10 years, I'm actually quite encouraged."

"It's just not there yet," he added.