Buyer beware: predictive models might not work the way you expect
Hospitals spend a lot of money on algorithms to predict and prevent sepsis cases and save lives. But these tools are largely unproven.
The predictive models exist largely in a gray market. Without approval or clinical evidence requirements from a regulatory body like the Food and Drug Administration, healthcare executives must rely on data from mostly a market of private vendors.
A new study shines light on this problem. The second-ever randomized control trial on a predictive sepsis model was published this week from safety-net hospital MetroHealth in Cleveland. Some patients with sepsis were caught early and received treatment faster because of pharmacist intervention coupled with a sepsis model from the country’s largest EHR company, Epic.
But the study comes with one big caveat that experts in health informatics warn of: these results shouldn’t be used as evidence that this model works in any other setting other than at MetroHealth.
“The reality is, I don’t think we’re all totally confident that this is a slam dunk,” said Dr. Yasir Tarabichi, director of clinical informatics for research support at MetroHealth. “And that’s why we did the study that we did; we don’t think this is obviously beneficial and everybody should use this.”
The model was deployed in half of about 500 participants in the emergency department. Some patients with sepsis were flagged earlier, and a pharmacist and doctor then came together to decide on the validity of the alarm. If the decision was to treat, the pharmacist then rushed an antibiotic to the patient. These patients received antibiotics quicker, and experienced reduced readmissions and improved days of life at home post-discharge. The findings were published in the journal Critical Care Medicine.
“If you’re the pharmacist, and [a doctor] has ordered an antibiotic, which is something that happens all the time in the emergency room, how do you know which one has the highest urgency, because everything is urgent in the emergency room,” Tarabhichi said. “You need to know this is slightly different.”
But the important thing about these predictive models is that their promised outcomes are dependent on how they’re used. Whether a doctor is notified by pager, or the alert is sent into a health record, or a nurse has to communicate the alert, or whether the model is deployed in an ED vs. an ICU – shapes how well predictions work.
“It’s useful in certain circumstances and in certain ways, and if you understand how it’s useful, then it becomes a tool that you can use,” said Gregory Briddick, the sepsis program coordinator at SUNY Upstate University Hospital in Syracuse. “It doesn’t matter what model you have, every one of them that I’ve seen so far has its limitation and strengths.”
But the issue is that the weaknesses are largely not communicated to hospitals deciding which models to buy. Some vendors have funded studies to measure how models work best at detecting sepsis, in a similar way that prescription drugs or medical devices are developed and tested. But that largely hasn’t happened. Hospitals have to rely on marketing materials and company data.
“Patients may be appropriately concerned when vendors are building predictive AI models behind closed doors with datasets and training processes that are not open to public peer review or regulatory approval,” said Dr. Jonathan Chen, assistant professor at the Stanford Department of Medicine. “This happens as the work is conducted under the gray area label of ‘quality improvement’ as opposed to research.”
The RCT from MetroHealth was funded entirely by the health system, without support from Epic. But increasingly experts in the field are calling for that kind of investment from companies.
“The big theme here is that you need more evaluation, not just for sepsis tools, but in general, predictive technologies need more rigorous evaluation,” said Dr. Suchi Saria, professor of machine learning and healthcare at Johns Hopkins. “People are building and deploying models without any framework for rigorous evaluations or ongoing monitoring infrastructure.”
Other studies have tried to get at the question of whether models work as advertised. That includes a June validation study from the University of Michigan, which looked at both inpatient and ED patients, and measured how accurate the model was in flagging sepsis. It found that the Epic sepsis model didn’t perform as well as the company claims.
“As a healthcare executive, you’re going to be presented with lots of information by lots of different vendors; my main takeaway message is that you have to do some inhouse verification to see that the model works here [in your hospital],” said Dr. Karandeep Singh, assistant professor of learning health sciences at the University of Michigan and an author of the JAMA study. “I think very few people in that position actually know the various ways in which these numbers can shift or not mean what you think they mean.”
With the potential for sepsis models to serve as a population tool and save lives, some in the field are pushing to make information technology and AI companies more involved.
“We need to start adapting our concept of an IT vendor as directly involved in health services, and directly involved in diagnosis and treatment,” Sendak from Duke said, adding that his team developed a mock label that vendors could use. “So some of the similar frameworks we have for procurement of drugs, procurement of devices, we need to start adapting to use for these types of tools.”
The team from Duke is working with industry partners to develop what this might look like, and to envision the creation of governance committees that use hard data to make purchasing decisions of models like it does with other products.
For its part, Emily Barey, Epic’s vice president for nursing in response to questions about the potential of the FDA or another regulatory body becoming involved with its products said they have some of the best research institutions as customers.
“I think the dialogue is really what we’re after,” Barey said, adding that Epic is soon holding a forum with customers to discuss sepsis. Whether the company might fund outside studies on its products, she said, “I think we’ll have to see what comes out of the forum. We take our cues from our customers at this point. So I don’t want to comment any further on whether or not we would go down that path.”