Healthcare AI: From Strategy to Execution

Where does healthcare AI most commonly fail? The honest answer is that there is not just one failure point. While on The Signal Room, Brian Sutherland, AI product manager and advisor who built one of the first member-facing intelligent assistants at Humana, was asked if he believed failure could be attributed to a lack of alignment in leadership, designed workflows, or the operating model. His answer was that it could not be attributed to just one. It is all of these, and it is all of these that makes the gap between AI strategy and execution inflexible.

This article occupies that gap. A good idea reaches the top of the organization, and by the time it reaches the people who must implement it, the idea has gone through what Sutherland compared to a game of telephone — a long, and in some cases irrecoverable, translation. Combine that with a technology that is still relatively new, where the fields and the positioning are still evolving, and a workflow that is already beginning to show its age, and the gap between the deck presentation and the functioning system becomes enormous. Bridging this gap is less about how quickly the system can be built and more about how everything must align to the system.

The Scale of the Vision Exceeds the Scale of the Structures

The first issue with the way in which Sutherland sees the world is a mismatch of scale. Here, a pilot is undertaken with genuine ambition, but the system is designed to provide an answer quickly and not to build trust. Sutherland carefully stocked the numericals and presented them as his own interpretation. For example, a pilot is likely to cover about 20% of real use cases and he describes that as success. The issue is that trust cannot be built in such a fragmented way. You cannot serve just 1 out of 5 cases well and then ignore the other cases and expect the users to put their faith in the system. The remaining 80% should be somehow managed, if not fully implemented, and a strategy put in place for what will happen when an external case is presented.

This is why so many pilots appear to be robust but lack any real momentum across the enterprise. The demo is focused around building excitement. The demo does not answer the more difficult design problem of what the system is meant to do with the cases it was never designed to address, and that is where trust is earned or lost.

Become the Operator Before You Automate the Work

At Humana, Sutherland used a discipline of spending time in the call center before building a member-facing assistant. This gave him a sense of the real frustration of the workers and where technology would be helpful or unhelpful. Real member experiences helped him decide what should and should not be automated.

The story points out the importance of being a part of the current reality. In the absence of a current picture of the work, designers and implementers are likely to automate the process based on workflows they remember, rather than on the existing workflows, leading to even more unnecessary friction. Chris Hutchins, from his own experience of visiting clinical placements, showed additional evidence for this, as he said that the process of work changes more rapidly than our understanding of it, and that there are friction points in the current reality rather than the past.

Governance That Enables, Not Governance That Stalls

Governance has a bad image as something that slows down progress, but Sutherland shows it differently. In the right context, it stimulates rather than hinders progress, and determines how scaling should be done.

The key factor is how you implement discipline. Governance should inform processes and should not result in “death by committee.” The goal is for a function to have confidence that the difficult aspects of a function are understood and are allowed to work on other aspects. Governance helps outline boundaries that will allow for potential scaling. This will also aid in the absence of some control. According to Hutchins, the concepts of the “control cycle” of AI, with the understanding that control is a continual cycle, should be applied. He explains that control interacts with a learning model, and therefore governance should also be seen in that context. Something that is approved today is based on assumptions that may have changed in six months. Therefore, governance should be a continual operational loop and should not be seen as just a point in time.

Blind Spots and Resistance to the Identification of Them

The relevance of perspective is due to the fact that blind spots are, ironically, aspects that no one seems to be observing. Sutherland emphasized that if someone is unknowingly secluded in a blind spot, and works in isolation, they will unknowingly work against blind spots that are unknown to them. Room for many perspectives makes it more difficult to overlook gaps. Not all gaps are equally troubling. Some are not troubling at all. Sutherland talked about silent costs that are late to surface. These include the damage to one’s brand and the costs of losing the customer lifetime value which are late to surface or are not immediately apparent. These costs emerge after fines and penalties, which should be of immediate concern and are legally unavoidable.

There are human factors in this system. When a blind spot is discovered and tested, the common reaction is to resist. This resistance is usually not due to a big ego, but because the commitment to a personal plan has been made, and surfacing the blind spot requires adjusting those commitments while dealing with the social and professional fallout this may cause. Sutherland believes that best practice anticipates this type of scenario. For example, be clear about how quickly you plan to launch, and then follow up with a plan to address things that are critical to the overall design of the system.

Where the Human Has to Stay in the Middle

Not every single experience needs to be automated. Sutherland drew the line at how personal the experience is. The more personal the experience, the more apprehensive a person will be to hand it over to a machine. Automating the purchase of a pair of sneakers is fine, but automating the experience of a patient is not, since there is a patient’s health at stake. In this example, the system has to rely on a human being. While a system may be able to mimic the art of an empathetic statement, a person on the other end of the system will be able to respond to a crisis in a way a model cannot.

Some of what the machine cannot pick up on is the hidden nuances observed only in humans, as described by Sutherland — the subtle changes in rhythm and pitch that express unarticulated feelings of unease or frustration, the ‘irksome’ silence that may present itself during a conversation. The medium strips all of those signals out, and during high-risk situations, you do not want to guess if someone is experiencing an adverse emotional response — you want to know. The lesson on execution is designing intentional pauses, or points in the system where judgement can be applied, in the workflow. AI is like a branching structure that is waiting to receive one of a limited set of answers; if you give it a surprise answer (the fourth), and there is no check in place to guide it, it will, in his blunt terms, just create something.

The AI is a Junior Employee Set-Up

The most useful concept Sutherland introduced was that you should think of the AI as a junior employee and take the AI out of the AI. You would train it, set expectations for the first month, two, and three, give more training as the rules shift and add more people to help it, and conduct constant support and oversight of it. Most importantly, it will never pass the junior stage.

Perhaps the most galvanizing part of the framing concerns mistakes. Assume the system will make mistakes, and to be clear, will repeat mistakes. Then, design the system to respond. He cites unintentional disclosures of protected health information as one of the predominant failures in healthcare, and the discipline in healthcare that is already known is a defined policy to identify, self-report, and contain, within the healthcare system, which prevents a single disclosure from becoming many. He was careful to differentiate the anxiety the framing can cause from the framing itself. Referring to AI as a member of staff is not a claim that it is a human or that it will be replacing one, but is a way of reminding us that it is artificial, is in a state of continuous development and will always need oversight. It is his opinion that the organizations that build their frameworks and their governance in this way, will be the ones with longevity, at least until the next disruption.

How Hutchins Approaches Strategy to Execution

The strategy-to-execution gap is the condition we meet most often at Hutchins Data Strategy Consultants: a defensible AI ambition that thins out as it moves from executive approval toward the frontline. Our work concentrates on the parts that decide whether a pilot becomes a production system — designing oversight that informs rather than stalls, getting close enough to the real workflow to place AI where it belongs, building the trust mechanisms and human-in-the-loop pauses that high-consequence settings require, and standing up the ongoing supervision an evolving system needs. This sits alongside our healthcare AI consulting practice and the responsible AI frameworks that make deployment defensible. These themes run throughout The Signal Room podcast, where practitioners describe what execution actually takes once the strategy is approved.

Authoritative sources

• NIST AI Risk Management Framework
• WHO — Ethics & governance of AI for health
• FDA — Good Machine Learning Practice for Medical Device Development
• U.S. Department of Health & Human Services — HIPAA
• Agency for Healthcare Research and Quality