The problem with the Boeing 737 MAX automated control system is continuing to dominate the news these days. We all know that faulty sensors produced incorrect information that caused a control system response to override the pilot and push the nose of the plane down repeatedly during the takeoff. The news of 346 lives lost in the two airplane crashes punctuated the news over several weeks. The pilots in the two aircraft were either helpless or clueless to prevent the tragedy that occurred. At that moment the airplane appeared to take over all decision making.
The healthcare analogy is obvious. A faulty sensor attached to a critically ill patient in the ICU registers an abnormally high reading and automatically activates an IV pump, pushing a preset dose of medication through the tubing into the patient. Among other things, the patient’s blood pressure drops to dangerously low levels and activates alarm bells at the bedside and at the central control center in the unit. Nurses rush to the bedside where previously routine care is now emergency care. You know the rest of the story.
Like many of the developments in our complex healthcare world, information technology tends to be “hyped” beyond practical and safe expectations. In one sense this exuberant enthusiasm is helpful in producing further advancement. On the other hand, we risk removing the on-site human mind from the situation like the 737 MAX, resulting in tragic accidents. In the ICU a trained nurse simply looking at the patient would be able to tell if the blood pressure was really spiking as the faulty sensor indicated. The message here is that we should not be in a hurry to remove the personal element from healthcare.
Giving a perspective on AI in healthcare, Ashish Kachru recently writing in Forbes sums up his enthusiasm by noting that while “the use of AI at any given time will never outpace the incredible capacity of the human mind to perform insanely complex tasks, … (it) will spur us to think and act in ways that are more sophisticated once we are relieved of certain lower-complexity tasks.”
Multiple industry observers have noted that AI-generated predictive analytics have a powerful role in managing the complex health populations that cost the most. This early identification and stratification of this population then allows us to focus the necessary resources on those people, potentially reducing medical tragedies. For example, 30-day hospital readmissions continue to burden the healthcare system. Furthermore, limiting our view to 30 days simply limits the outcome. Keeping track of that at-risk patient over the next 120 days can improve outcomes and reduce cost.[ii] A current AI tool at the Children’s Hospital of Pittsburgh has been able to predict with 79 percent accuracy which patients are at most risk for hospital readmission.
While AI is proving its value in population management, there is still room for healthy skepticism in the acute management of patients. Airplane pilots and nurses have a measure of situational awareness that a computer may not fully comprehend. AI can inject a warning; however, we may want real, well-trained human intelligence to make our decisions in the cockpit or the bedside into the foreseeable future.
[ii] Salient-Trinity Health analysis presented at NAACOS 2019
