Scaling Healthcare AI: The Trust Infrastructure Underneath

If scaling AI requires a useful analogy, think of a launch pad. That was how Amit Shivpuja, director, data & AI enablement for Walmart, framed a Signal Room conversation on hidden infrastructure of trust. If AI is rockets, the data is the launch pad. If you get the launch pad infrastructure right, the rockets can be launched. If you get it wrong, you will not scale, just repeat the same unreliable rockets launches.

The line that resonated with him described the stakes as garbage in, garbage squared out. The cost of poor input is not a constant drain as the system scales; the effect is a compounding cost. The first launch on an organization's part builds credibility, and the system may go unnoticed for a long time. The trust will fade if the same unsound foundation is relied upon to serve an entire organization. At Hutchins Data Strategy Consultants, the most noticeable gap in a deployed system is between a working demo and an actual scaled system: the demo proved the model, and the rollout exposed the data.

Nothing Scales Without Trust

The issue of trust is something that has to be worked on alongside the building of AI and it is the critical factor in the success of the AI.

Current generative tools have a certain problem. These tools are easy to use, and since there is almost no resistance, people start to trust them more than they should. This is a problem because there is almost no resistance to losing trust either. A few incorrect answers, either because the tool is hallucinating or due to the incorrect training data, and people lose trust quickly. Building trust relies on the three things Shivpuja identified: transparency concerning the data actually available, accountability to ensure the question "who addresses the bad output after deployment?" has an answer, and the ability to provide justification for an answer. None of these components are present in the tools by default. Each of them must be incorporated into the tools at least to ensure that the tools can hold the weight of increasing demand.

Bring the Frontline In Before You Build Anything

One of the most relevant aspects of the conversation was related to the timing. Almost all organizations have the tendency to operate the way Chris Hutchins described the situation in the healthcare sector. The tendency is to first create something that looks extraordinary because the creators believe that the target audience will eventually show up. Chris Hutchins terms this tendency the "field-of-dreams approach," and it looks more honestly like if you build it, they won't come.

Shivpuja's solution was to involve both the subject-matter experts and the eventual users as soon as possible, as context exists primarily with them, and to gather the requirement and the exact need and possible business impact before building anything to have that baseline to compare against later. Two things follow. You design against a target rather than a guess, and the people who were part of the journey become the program's biggest champions, which is what a pilot needs to become an enterprise.

Trust can be lost here before a single line of output is generated. People will not rely on a capability that they do not understand the purpose of, and no amount of polish at launch recovers that.

Address the Fear by Showing the Value

The scaling of AI in any organization hits a human concern, that being the automation of the job. Shivpuja was open that the impact eventually on employment was and still is a real debate with the opposing camps. His way of easing the fear was working on the value, by showing it concretely. A task that took ten hours of manual effort now takes one. The manual error that came with it disappears and the person is left with time.

Somewhere in the discussion, leaders get stuck figuring out what the time they regained is actually for. There will be tasks that they wanted to start, and there will be challenges they wanted to address. Naming that turns a threat into a bargain. This also means that the workforce needs a baseline of data and AI literacy. It is not technical fluency for everyone and not to the same deep level, but to know what a model is and what it is not. AI's time, he argues, should be reinvested to avoid losing in-house expertise along the way.

Build Trust with Communication

Since trust takes time to build and can slowly disappear, some of the structures are monitoring for the signs. The read on the warning signs was unglamorous and direct. He watches reactions to prototypes, how people actually use it and whether the feedback on the first version is good. The right relationships are an additional channel. This is because the people who trust them say when something is not working.

For this to work, contact must be sustained. He was emphatic that this is not a fly by or fire and forget. An agent, in particular, will drift and, at some point, do something it was not meant to do. Someone has to be there to see and respond to that. This was related to a larger issue of trust in institutions, where members of an organization do not trust the leadership or some members of the organization, which only increases the cost of the absence of a consistent, ongoing communication which is necessary to highlight concerns and issues in a timely manner.

What Leaders Should Carry Into the Transition

When told to imagine and communicate to leaders navigating such transformations that are comparable to the shifts in the internet of the 1990s, Shivpuja shared a brief checklist. Trust but verify. Leaders should trust their technology and teams, but should make sure that technology and teams are doing the work they said they are doing. This matters more here than in past technology waves, because the tool in some cases is building the tool, with its own automated capabilities.

Correctly establish the data foundation, as the can has been kicked down the road one too many times, and even if data correctness were ideal, there are still biases and structural issues from business data shaping that would still require addressing. Also, keep a human in the loop as indispensable, for verifying and for representing the people for whom these systems are ultimately built. None of these are checkboxes for go-live. Rather, these are the boundary constraints for where a deployment can grow responsibly.

How Hutchins Approaches Scaling Healthcare AI

Our work tends to begin at the launch pad rather than the rocket. Before a healthcare organization scales a model across service lines, we assess whether the data foundation underneath can actually carry the weight, and whether the transparency, accountability, and verification needed to keep users' trust are in place to grow with it. That includes the unglamorous work of data governance and data quality, and the organizational side — bringing frontline experts in early and building the AI literacy that lets people use these tools well rather than fear them. These themes run throughout The Signal Room podcast, where practitioners describe what it takes to move from a working pilot to something an enterprise can rely on.

Authoritative sources

• WHO — Ethics & governance of AI for health
• NIST AI Risk Management Framework
• FDA — Good Machine Learning Practice
• Agency for Healthcare Research and Quality
• HIMSS