The Lights and Darks of Transparency, in a New Book about Health Care

The health care field is notoriously inscrutable, leading many reformers to hope that we can lower costs, improve communications, and inform policy by wrenching open health care’s closeted areas. These hopes became the focus of the latest anthology coming out of the Petrie-Flom Center of Harvard Law School, titled

Transparency in Health and Health Care in the United States. I have not read this book yet, but I attended the launch at the Harvard Law School and took down notes from the lively exchange of ideas there.


Transparency can boast several victories. Numerous services such as Clear Health Costs are used by thousands of patients-cum-consumers to find procedures at reasonable prices. A recent New York times exposé of aggressive overpricing and litigation (by no means a unique case) is leading to changes just by revealing to the wider world a situation that was already well-known among the local suffering population. Dr. Ameet Sarpatwari of Harvard Medical School, a contributor to the book and a panelist at the Petrie-Flom Center forum, labeled journalism an important form of transparency.

By now, calls for transparency are aired everywhere. One of the federal legislative efforts most likely to advance would eliminate surprise medical bills. A large percentage of the many state efforts to lower medication costs use transparency as a lever. The FDA routinely adds to its regulations for reporting errors by medical devices. Many reformers hope that insight into costs and operations will bring hospital charges closer in line, help doctors adopt the practices of their more successful colleagues, provide data for research into new treatments, address the disparities between races and genders, lead to more consistent IRB rulings, and more.

Transparency can disappoint, though, because people often don’t take advantage of it. For instance, a cost comparison site set up by the state of New Hampshire has been consulted by only one percent of its residents. This statistic was mentioned by the panel’s moderator, Elena Fagotto of the Project on Transparency and Technology for Better Health. Fagotto did not speculate as to the reason for such a low usage rate. Do residents not know about it? Are they just lazy? Perhaps relatively few procedures are amenable to this kind of shopping around, or perhaps insurance and institutional barriers prevent patients from acting as consumers. Or perhaps other considerations–such as trust in a known medical provider–take precedence over costs.

But the panelists also warned about potential negative effects of opening up information. Most obvious is the risks to patient privacy: every advance in sharing patient data leads to some chance of it being combined with other data to reidentify a patient (the mosaic effect) or otherwise misused.

The costs of collecting data are also non-trivial. Regulations and payers ask for more and more data year after year, in the hope (rarely satisfied) of improving medical care or public health. The number of boxes to check and fields to fill on forms is pushing doctors past the limits of their patience. The time required for this data collection plays out, naturally, not ony as increased burn-out but as increased medical costs.

I wonder whether the burden on health care providers could be avoided by getting patients to self-identify with conditions, providing their data to regulators and researchers directly. In some ways, patient self-reports are less reliable and accurate–for instance, rare is the patient who can provide the ICD-10 code for her condition. (On the other hand, how accurate are ICD-10 codes anyway? And how many providers are padding their bills by “upcoding”, perhaps as a response to payer austerity?)

In some ways, I suggest, patient self-reporting may be more accurate than reports from providers. The patient may have insight into herself that the doctor hasn’t taken time to learn, or may be more honest with a computer form than with the clinicians interviewing her.

Beyond privacy and cost, several other problems exist with transparency.

First, it can become just another weapon of control in hands of rich and powerful institutions. A notable example of this trend is the tools used by employers to snoop on employees and job candidates, bringing their influence into our private lives in a way reminiscent of the nineteenth and early twentieth centuries, when employees were fired for “immoral” behavior. In other words, information alone does not change power relationships.

One form of imbalanced power is majoritarian disregard for the interest of minorities. By that rather abstract phrase, I mean that when we better understand the costs and impacts of decisions, we may ride roughshod over the needs of small, disempowered groups. We might say we’re not going to spend a million dollars to save the life of someone suffering from a rare genetic disease. How can such conflicting desires be resolved?

Author and editor Glenn Cohen, Faculty Director of the Petrie-Flom Center, mentioned that many companies invoke the First Amendment (a pinnacle of the movement for transparency) to overturn laws that were meant to protect the public from biased advertising and similar practices.

A second down-side comes when transparency is used to punish or discourage marginalized people. That, at least, was the complaint raised against the U.S. administration’s attempt to put a citizenship question on the census form. Another example raised at the panel by author and editor Holly Fernandez Lynch involves the Environmental Protection Agency. The current (arguably anti-EPA) heads of the EPA announced that they would consider only studies that release the underlying data. The catch is that many crucial studies have personally identifiable health data, which can’t ethically be open to the public. I don’t know whether researchers (at some cost and effort) could anonymize the data and whether doing so would satisfy the new EPA rule. But under the current circumstances, the proposed rule (still under consideration) would eliminate important evidence that could be used to protect the public.

There is one more risk in transparency, raised repeatedly at the panel: that it could become an excuse for not doing anything. What if we learn the true costs of hospitalizations or drug development, and they turn out to justify high prices? Will we have the savvy and political will to change the underlying processes in order to bring these costs down? And does publishing the cost of a $20,000 back surgery make it OK to charge that much? What if the back surgery is not expected to be effective (a situation that perhaps could be addressed through another kind of transparency)?

In short, transparency is useful when it challenges the status quo instead of reinforcing it, and when consumers, regulators, or companies are willing to act on the results of transparency. Before we raise hopes and impose burdens by collecting data, we should make the commitment to push through whatever forms of change that data recommends.

About the author

Andy Oram

Andy Oram

Andy Oram writes and edits documents about many aspects of computing, ranging in size from blog postings to full-length books. Topics cover a wide range of computer technologies: data science and machine learning, programming languages, Web performance, Internet of Things, databases, free and open source software, and more. My editorial output at O'Reilly Media included the first books ever published commercially in the United States on Linux, the 2001 title Peer-to-Peer (frequently cited in connection with those technologies), and the 2007 title Beautiful Code. He is a regular correspondent on health IT and health policy for He also contributes to other publications about policy issues related to the Internet and about trends affecting technical innovation and its effects on society. Print publications where his work has appeared include The Economist, Communications of the ACM, Copyright World, the Journal of Information Technology & Politics, Vanguardia Dossier, and Internet Law and Business.