We Can Do That
Eighty-five percent of AI pilots fail, and not because the model was wrong. Notes from the data side on the debt beneath the tech debt, the commitments that lock health systems in, and what it looked like when data quality finally got treated like an asset.
First published in The AI Health Pulse. Also on LinkedIn.
Eighty-five percent of pilots in the US, no matter the industry, are failing. The reason is not the one people reach for. They're not seeing any ROI because they cannot even get implementation to the point where they are testing anything of value. They have that black hole piece: they do not understand the data.
The most noticeable gap in a deployed system is the distance between a working demo and an actual scaled system. The demo proves the model. The rollout exposes the data.
The debt under the debt
In healthcare especially, they've got so much data technical debt they cannot even begin to touch AI solutions. The technical debt is disguising and hiding the data technical debt, because they'll spend tens of millions on the tech but won't deal with the data quality.
If you're growing by acquisition, you're inheriting a matrix as ugly as anything you could imagine. Every time you acquire another company, you've got another one of those matrices to figure out how to connect.
The only thing they normalize is the stuff that runs the business. So why is it we can't fix the pieces that actually touch people in healthcare?
The commitments that lock you in
The big investments with AWS, Google, or Microsoft feel like safe bets, so they overcommit. They sign seven to eight year deals convinced they're getting solutions, and what they got is cloud computing and cloud storage.
Then the parade begins, all these people knocking on your door trying to get you to give them a use case. You have no solutions. Your team is still going to have to do all the same work tomorrow that they're doing today.
The contract side compounds it. You're throwing contract dollars at something that's not building any core competencies into your DNA. Every time something gets finished, all the know-how walks out the front door, and your team can't support it. Then you end up contracting longer term for people who don't know your business and aren't going to learn it. Increasingly the people I meet are lawyers who understand contract law and can help them untangle the bad contracts they've got in place.
Rather than going in trying to sell them a piece of technology, I try to understand where they are and what their major challenges are. Most every time, they start talking about the commitments they've made and how they're in a really tough spot.
Calling the bluff
Whenever you're looking at a solution, your IT guys stand up and say, we can do that, we don't need to pay all that money. You're taking valuable resources off the lights-on strategy, reducing support to your existing people, and half-assing something you're not even capable of delivering at scale.
If I'm a CEO and my IT guy says that, I ask one question. Do you have a demo for me anytime soon? Because if you could have done this and you haven't, that's a problem. But CEOs don't land on that conclusion. They just get taken advantage of.
These IT teams are delusional. What have you built at scale? If you built something that worked, you got lucky. If you built something that scales, you caught lightning in a bottle, because that doesn't happen.
Start with the risk, then find the yes
I think about it in terms of AI readiness. You can have all your data in great shape, but if your cyber isn't top-notch, you've got risks you don't even know about. So the very first thing is a risk assessment. See what exposure you already have.
The first thing I do is help them understand where they are, because usually the issues are not the ones they think they have.
I find them the areas they can say yes to, because they do have data in solid enough condition to trust for certain things. They're not usually the ones they want, but those are the ones they can get while we figure out the steps to get the other data sources ready.
The space between the verticals
I advocate for a fractional chief AI and analytics officer, because most organizations aren't ready for a full-time one but don't have anybody who understands the overlap between finance, operations, and clinical. They have experts in each vertical, but none understand enough about the others to see where things get intermingled and unintentionally put things into models that should never get there. That space in between the verticals is usually the area that needs the most focus.
Finance is one of the first places I look, because nothing goes well if you don't have the finance people on board with a methodology they can stand behind. They're the first ones I have to get on board.
Most organizations have what I call a data club instead of a data governance. They waste time bringing high-level people together to make decisions. After two or three meetings those people go away, and the only thing they end up doing is explaining why they can't do anything. It's gridlock.
Vet the people before you commit
I want to know who I'm dealing with, and I want to know what capabilities are there, because I'm not going to gamble and put something in front of anybody unless we know for sure it's really solid.
I can't gamble on things that are foolhardy. One guy going hard after a pitch said he'd send me all the details on his technology, and he sent me a document that was one hundred percent generated by AI.
One of the best things I do is knowing when to yield and let someone who actually knows what they're doing step up.
What it looked like when the data became the asset
About ten years ago I was hired into a health system in New York as chief data and analytics officer. The team was already eighty people, so it was not that anyone had decided to invest in this work. They were primarily people who dealt with the technology. Data architects, BI developers, a few visualization specialists. The one area they were primarily responsible for was the clinical space, and in particular the logistics of the whole emergency services division. My team supported around fifteen hundred databases and backend systems, a medical school, a nursing school, twenty-three hospitals, and over nine hundred outpatient facilities.
What changed it was not something we bought. The CMO put a PhD-level physician informaticist in with my team, and nothing shipped until it passed his sniff test. We were swimming upstream quite a bit. There was a lot of resistance to standardizing things and getting the noise out of the pipeline, because everyone has a little pet thing they want, and they did not appreciate that they were going to have to talk to my physician informaticist. He challenged the status quo every time. What does this answer change? What decision are you going to make differently with what you are asking me for? They did not like that. But we took the noise out of the pipeline. Before that, the output was crap, no one trusted it, and they should not have.
Then it got tested. Within twenty-four hours of hearing that the first COVID case hit, my team already had everything in place. We had already built a real-time currently admitted patient index across twenty-three hospitals. We knew exactly how many beds we had. You have a catchment area of around six million people, so you do not get to have multiple environments and hope they connect. You have to have one. That index let us run efficiently, and our command centers were up without missing anything. We pulled details from the workforce safety division so we knew where the most modern HVAC systems were, because wherever we had those, we needed a place to isolate people who were not COVID positive. We had all of it hooked up in dashboards overnight, and within three days we were working off them.
None of that was an AI project. It is what became possible once the data quality was finally treated like the asset it was. That is the work nobody wants to fund, and it is the only reason the hard thing was ready when it was needed.
You already have governance. You just may not realize you can manage it and control your AI.
Continue reading from Hutchins Data Strategy
What the CISO Now Owns → https://hutchinsdatastrategy.com/the-ai-health-pulse/what-the-ciso-now-owns
The Committee Nobody Staffed → https://hutchinsdatastrategy.com/the-ai-health-pulse/the-committee-nobody-staffed
On the Signal Room podcast
The Hidden Reason Hospital AI Keeps Failing | Angel Mena, MD → https://signalroompodcast.com/episodes/hospital-ai-pilots-fail
Healthcare Data Readiness and AI Adoption: Why 85% of Organizations Aren't Prepared | Ratnadeep Bhattacharjee → https://signalroompodcast.com/episodes/data-readiness-ai-adoption
The Dark Side of the $50B AI Medical Boom | Lorraine Fernandes → https://signalroompodcast.com/episodes/dark-side-ai-medical-boom
More from Chris Hutchins
Subscribe — one email gets you The AI Health Pulse, the Signal Room podcast, and a free chapter of Beneath the Signal → https://hutchinsdatastrategy.com/free-chapter
Book Chris to Speak → https://www.chrisjhutchins.com/
Learn about our work to improve mental health around the world → https://continuamindhealth.com/
Do you have a podcast? We offer a Free growth assessment from PodcastPull.com → https://podcastpull.com/
Read on BeeHiiv → https://aihealthpulse.beehiiv.com/
Read on Substack → https://aihealthpulse.substack.com/
Tags: AI Health Pulse newsletter · healthcare AI ROI · data technical debt · AI readiness · AI pilots failing · cloud overcommitment · build versus buy · fractional chief AI officer Hashtags: #HealthcareAI #DataStrategy #AIinHealthcare #HealthIT #DigitalHealth
One signal a week. No noise.
Join healthcare leaders reading The AI Health Pulse every Monday.
Facing a challenge like this in your own system?
See how we approach healthcare AI consulting and data and analytics strategy, or book a call.