We are Still Learning the Lesson Charles Babbage Taught Us in 1821

May 11, 2023
Colin Hung
6 Min Read

Back in 1821 when Charles Babbage introduced the world to his Difference Engine, one of the world’s first mechanical computers, he taught us that bad input = bad output. This is a lesson we are still learning today in healthcare. As we leap into the world of artificial intelligence (AI), machine learning (ML), and large language models, we would do well to remember this lesson before relying too heavily on the output of these fantastical technologies.

At the recent HIMSS23 Conference in Chicago, Charlie Harp, CEO of Clinical Architecture – a company that provides solutions for healthcare data quality, interoperability, and clinical documentation – delivered a spotlight session that highlighted the work of Babbage and his important lesson.

Relying Output from Bad Input is Bad

In his presentation Harp recited this hilarious quote from Babbage from the early 1800’s (you have to read it and imagine a posh British accent):

“On two occasions I have been asked [by members of Parliament], ‘Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?’ I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.”

Using dry English wit, Babbage effectively created the concept of “Garbage In. Garbage Out.” This concept is as true today as it was in Babbage’s time and we are in danger of not learning the lesson.

In recent months the world has become enamored with new AI tools like ChatGPT which can perform amazing feats of writing while responding to user created prompts. Over the past few years, AI tools have been helping to improve radiology workflows, direct patients to the most appropriate level of care, optimize clinician schedules, and automated thousands of administrative tasks. Yet, rarely do we ask the question – what was the data that was used to train these AI tools? Is that data representative of the world these AI tools now operate in? Are we confident the data was of good quality?

Harp challenged the audience to think about this during his HIMSS23 presentation.

More Focus on Data is Happening

It was very apropos that Harp presented his own data analysis as part of his presentation. He analyzed the occurrence of the term “Garbage in. Garbage out.” In PubMed and plotted the results over time.

From 1972 to 1998 there is barely a mention of the term. From 1999 to 2022, however, Harp found a steady rise in the use of the term in publications. Interestingly, Harp also plotted the major Health IT milestones on his chart like – MIPPA, MIPS, MACRA, and CURES. You could say that the concern around data seems to be growing as the need for quality data rises through regulations.

This analysis aligns with a message that Healthcare IT Today has discussed with Harp on recent occasions – that quality data is vital to healthcare.

Quality of Patient Data

According to Harp, the biggest determinant of the level of quality of health data is patient data.

In a recent data quality survey conducted by Clinical Architecture, Harp’s team found two interesting results. First was that healthcare organizations felt that their SDOH, Allergies, and Procedure data was the poorest quality vs Demographic data, which was ranked highest quality.

The second interesting result was that overall, organizations are not very confident in the quality of patient data they have collected. Worse, the survey found that organizations have very little trust in data that originates from outside their organization.

And therein lies the paradox. If we ourselves are not confident in the quality of the health data we have collected, then how confident should we be in tools that are based on or trained on that same data? After all, where do the companies that are making the AI algorithms get the datasets they use for training? Makes you wonder.

For the full survey results check out: https://clinicalarchitecture.com/data-quality-survey/

Improving Data Quality

Harp ended his presentation on a positive note by quoting Aristotle, which for accuracy, he used the actual translated quote: “As it is not one swallow or fine day that makes a spring, so it is not one day or a short time that makes a man blessed and happy.” In other words, our desired endpoint does not happen in an instant or with one event. It is achieved over time.

If we want quality health data (and we should, according to Harp) then we need to invest the time and resources to make it so. We have the technology to solve our data challenges, now we need to be willing to commit ourselves to the journey of data quality.

If we don’t then Babbage will have been correct all those years ago.

Learn more about Clinical Architecture at: https://clinicalarchitecture.com/

Clinical Architecture is a sponsor of Healthcare Scene.

Tags

About the author

View All Posts

Colin Hung

Colin Hung is the co-founder of the #hcldr (healthcare leadership) tweetchat one of the most popular and active healthcare social media communities on Twitter. Colin speaks, tweets and blogs regularly about healthcare, technology, marketing and leadership. He is currently an independent marketing consultant working with leading healthIT companies. Colin is a member of #TheWalkingGallery. His Twitter handle is: @Colin_Hung.

   

Categories