Massachusetts Study Uncovers Arbitrary and Ineffective Use of Quality Measures

So much in health care depends on the accurate measurement of costs and quality. Doctors are judged by the costs racked up by their patients. Health care providers in some states are required to post the costs of their procedures online. The entire historic shift from fee-for-service to fee-for-value (that is, judging providers by the long-term health of their patients) assumes that we have a foundation for accurate measurements of costs and quality. And yet, as I wrote three years ago, such measurements are elusive. A recent study by the Massachusetts attorney general’s office suggests that the problems persist.

The data at the base of the study is very narrow. The most important data, for the purpose of this article, pertains to efforts at health insurers to institute some kind of quality-based, fee-for-value incentives to health care providers. Massachusetts, a national leader in health care and funding innovations, has a good deal of experience to draw on. For instance, Blue Cross/Blue Shield of Massachusetts instituted rewards for cost control in 2009. The various efforts at tying payments to quality in the state are called “alternative payment arrangements.”

(Of course, many of the techniques used to monitor and control costs are updated, analytics-infused versions of the old managed care paradigm from the 1980s. And Massachusetts continues to suffer from some of the highest health care costs in the country, albeit in support of some of the best care in the country. The governor wants to launch an investigation into these costs.)

The survey also looked who uses the “cost estimators” that insurers provide their members in Massachusetts. This information is correlated with various data about patients, such as age and the size of deductibles in their plans.

What did the attorney general’s office turn up from these specific inquiries?

The most significant finding concerned churn at insurers. The study reports that about half the patients switched insurers during the two-year period of the survey (page 20). This is enough in itself to cripple value-based efforts. The reason is clear enough to health care reformers: improving the health of patients, especially those with the chronic conditions that lead to most costs, has a long-term payoff. One invests today in behavioral coaching, monitoring, and other preventative efforts for a payoff twenty years down the road. Why go through the effort so that your competitor can reap the benefits?

This is why T.R. Reid’s classic book, The Healing of America: A Global Quest for Better, Cheaper, and Fairer Health Care, promoted a single-payer system. Only when an institution (probably the government) has responsibility for cradle-to-grave health care coverage can fee-for-value rest on a firm financial foundation. Of course, single-payer has problems as well. For the purposes of this article, the key weakness of single-payer is the same as what we see in today’s systems: quality depends on accurate, consistent, reliable measures of quality in costs and outcomes. The health care field can’t muster these measurements, which is an issue of analytics rather than politics.

Insurers aren’t the only players in health care who are ill-rewarded by alternative payment arrangements. Doctors are treated unfairly too. Basically, a primary care physician (PCP) is rated by the costs incurred over time by her patients. Some specialists are rated this way too. But the entire structure of our health care system, as found by the study (pages 22-24), undermines the accuracy of these ratings. Patients can visit specialists without even telling their PCP. And who is this PCP, anyway? Many patients don’t have a regular PCP, so their costs are arbitrarily attributed to the most recent PCP they saw, even if that was more than two years earlier.

As commentary, I’ll mention that the patients’ disconnection from primary care can be partly blamed on a shortage of PCPs, because the job is grueling and poorly compensated. It’s also probably worth stating the obvious: that without regular visits to a PCP, both the patient and payer lose out on the benefits of coordinated, holistic, preventative care.

The attorney general’s study confirms the common observation that PCPs are likely to refer patients to the hospital with which the PCP is affiliated, regardless of the cost of procedures. This suggests the following scenario driving rising health care costs:

  1. High-cost hospitals rake in profits for overpriced procedures.

  2. The high-cost hospitals then buy up local practices containing PCPs and specialists.

  3. The PCPs and specialists refer their patients to the high-cost hospital with which they are now affiliated.

  4. Return to step 1.

One might hope that the cycle just described could be undercut by the cost estimators required by law. So now we must turn to the primary finding of the survey, which matches results seen in other states and should be no surprise: very few patients take advantage of the cost estimator. Those who do tend to be young (page 11) and have high-deductible plans (page 15). The study questions whether the search for lower costs has much benefit to the patient, because she will probably still end up paying the same deductible. The study also suggests that health care providers can cheat, because nobody checks whether the cost exceeded what they promised (pages 17-18).

As cynical as I may appear in this article, I maintain a firm belief in value-based care and an unblemished hope that the health care field can find a fair, accurate way to reward its providers and insurers. The problems uncovered by this study, in the state that stands at the forefront of health care in the U.S., may point us to solutions.

About the author

Andy Oram

Andy Oram

Andy Oram writes and edits documents about many aspects of computing, ranging in size from blog postings to full-length books. Topics cover a wide range of computer technologies: data science and machine learning, programming languages, Web performance, Internet of Things, databases, free and open source software, and more. My editorial output at O'Reilly Media included the first books ever published commercially in the United States on Linux, the 2001 title Peer-to-Peer (frequently cited in connection with those technologies), and the 2007 title Beautiful Code. He is a regular correspondent on health IT and health policy for He also contributes to other publications about policy issues related to the Internet and about trends affecting technical innovation and its effects on society. Print publications where his work has appeared include The Economist, Communications of the ACM, Copyright World, the Journal of Information Technology & Politics, Vanguardia Dossier, and Internet Law and Business.