Successful examples of AI in health care have reached the news often enough to make it clear that modern algorithms can be valuable. But we don’t want AI to be a black box. So I felt it a great opportunity to talk to Stefan Behrens, CEO of and co-founder of GYANT, a company that makes a healthcare virtual assistant. He told me a bit about how their machine learning algorithms work, and how they can explain how they reached their recommendations. I’ll summarize GYANT’s service in this article and explain their machine learning at a very high level, with minimal technical jargon.
What Front Door Does for Providers and Patients
On May 13, GYANT reported metrics on the adoption and success rate of its COVID-19 Screener & Emergency Response Assistant (COVID-19 SERA). This is an interactive screening tool, integrated into a health care provider’s system, that helps patients determine whether they should come in for treatment. The following screenshot shows a sample question-and-answer screen, which is asking the patient about symptoms and other high-risk factors.
The tool is currently in use at 16 health systems and payers. You can read more about it at the links I’ve provided; what I’ll focus on here is more general offering or which this new service is one particular application. The parent service–a “care navigation and triage tool”–is called Front Door.
Front Door does more than the simple question-and-answer chatbots offered by some other health care apps. In some ways, Front Door is part of a trend in health care to let smart tools do some of the routine work of a clinician.
The patient’s interaction with Front Door starts with a visit to the health care provider’s website or patient portal. The patient enters a request, which Front Door interprets through natural language processing. Sometimes it can direct the patient immediately to the right resource–for instance, connecting them with an appointment scheduling system, leaving their doctor a request for a prescription refill, or just providing an answer to a question without involving the staff. Other times, Front Door goes through a short dialog with the patient before deciding what to do.
Striving to be a comprehensive engagement tool, Front Door has to integrate with the provider’s computer systems at several levels: the scheduling system, the patient portal, and the contact center. A particular strength of Front Door is to take information from the patient’s electronic health record, which has historically been a black box. The integration allows Front Door to personalize its recommendations better by pulling information about the patient’s past medical history, medications, and allergies.
Based on all the available information, Front Door can usually recommend the appropriate course of action to each patient based on their symptoms and history. Where appropriate, Front Door can usually guess the underlying condition behind the patient’s symptoms and connect the patient to the right in-network specialist.
The machine learning algorithm was trained on hundreds of thousands of notes from primary care and emergency room visits, and later fine-tuned by modeling tens of thousands of telemedicine interactions between real doctors and patients. The algorithm also incorporates knowledge from established medical triage protocols and is closely supervised by GYANT’s medical team and external medical experts.
It usually takes two to three months for GYANT to configure its solution for a particular health care provider by incorporating the specific services and operation of each care venue. But like some other organizations in health care shocked and galvanized by the COVID-19 crisis, GYANT rose to the occasion with COVID-19 SERA, which is customizable and deployable within 72 hours.
Seeing the Forest
GYANT’s machine learning is based on popular algorithms in common use. Looking at how they work will help to explain why these are appropriate for health care.
Think of how a doctor makes a diagnosis. They collect a report of symptoms–for instance fever, chest pain, and headache–along with information from the record about the patient’s chronic medical conditions. They weigh all these, often in consultation with medical literature and expert colleagues, to decide which diagnosis is the most likely fit.
Machine learning also looks at symptoms and chronic conditions, which can be called variables, features, or dimensions. The last term, dimensions, is worth a special look for what it connotes.
Think of having two variables that you plot on a graph. There are X and Y axes, and any change to either variable produces a different point on the two-dimensional graph.
Well, machine learning looks at hundreds or thousands of dimensions. A human doctor can intuitively pick out what’s most important; they’re not likely to think about plantar warts when trying to diagnose chest pain. The computer lacks this intuition, but makes up for it with vast computing power. (The need for mind-boggling numbers of calculations explains why machine learning didn’t take off until the twenty-first century, even though the ideas have been around for decades.)
In theory, what the machine should do is compare the impact of every dimension on every other dimension. What happens when you throw in the headache, versus ignoring the headache? How does this affect your result? The algorithm would consider the plantar warts and throw out the dimension pretty quickly, because it would turn out to have no effect. Eventually, the algorithm would narrow things down to a few relevant symptoms and conditions, as the doctor does, and could choose a diagnosis.
But with hundreds or thousands of dimensions, even modern computers would churn far too long. So researchers have found a lot of clever ways to cut down on processing.
GYANT uses a random forest, based on the principle of divide-and-conquer. The computer systems run many analyses simultaneously using the same algorithm, but each of these runs uses a subset of the dimensions and a subset of the data. This is done repeatedly, looking at results and choosing the most promising combinations of dimensions for the next round.
Random forests are very popular because they achieve good results quickly. Behrens says that they have another appealing benefit: because they keep track of which dimensions (symptoms) are most important, they can tell the doctor why they made the decision by surfacing the most predictive dimensions. In other words, the models are explainable and accountable, crucial traits in disciplines like health care where life-and-death decisions are being made. (Front Door doesn’t offer diagnoses, but can direct patients to the resources most likely to help them.)
Learning on the Job
AI promises better results than conventional computer programs because it can learn and improve as it goes along. Yes, other systems can sometimes improve too. For instance, spam blockers adapt to new types of spam through Bayesian filtering, which is based on an idea discovered by the 18th-century Thomas Bayes. But learning is a central feature of AI.
Behrens told me that Front Door has two layers of learning. The first layer is a simple feedback loop based on asking the patient, “Did you find what you were looking for?” Also, clinicians can look at questions to find out what patients are interested in, and add new educational materials or other capabilities.
The second layer helps the processor become more sophisticated in its core job of triaging and directing patients to resources. This layer analyzes the patient’s electronic health record to see what happened after they were triaged. By reviewing the diagnosis issued by a provider in a virtual or in-person visit, GYANT can compare the outcome to the predictions of its AI model and use this feedback to get better and better over time.
As a central repository of data about questions from many clients, GYANT can look across its whole range of health care providers to see what works and what doesn’t. The company often advises a client to change what it says to a customer, because the change has proven effective for other clients. At the same time, Front Door can be adapted to the different cultures and living conditions that providers deal with.
To provide a backstop to the machine learning system, Front Door has a second, simpler engine that can be fed rules by the provider and can make judgements based on those rules. This is another model for AI called an expert system. It assures the clinicians that Front Door will recognize certain obvious things and not let a high-risk patient fall through the cracks.
GYANT also continues to work on patient engagement. Behrens says they are trying to avoid bureaucratic forms or notices and to make the experience “delightful” for patients. This is also what we all want: AI with a human face.