AI Literacy in Healthcare: The Gap Between Hype and Understanding

The biggest gap people encounter when large organizations implement AI is not a technical gap. On Signal Room, physician executive Dr. Steven Labkoff described leading an AI rollout for a major life sciences company. Labkoff said that the vast majority of leaders — senior or mid-level managers or administrators — did not understand what AI could do. Only a handful of executives understood it. The technology was being advanced and developed at a faster pace than the executives' understanding of how to implement it.

Labkoff argues that this is the biggest oversight in the risk of AI in healthcare. The discussions focus on the governance and safety of AI, which are critical, but there is usually little discussion on literacy, which is one of the largest gaps in healthcare AI. As a consultancy, we continuously encounter the consequences of this gap when we see clients adopt tools for reasons that don't create value.

AI is an Information Technology

Some of the challenges we face on the client side are a direct result of the coverage AI receives. The novelty of AI leads clients to believe that AI is something entirely new and different, which removes the critical lens through which technology is analyzed. Labkoff shows that clients should consider AI for what it is — an information technology — and apply the same critical reasoning they would have used for other technologies.

He refers to the internet as the parallel example. The hype in the early-to-mid 1990s was similar. It was said that it would change everything in an instant. However, it wasn't until the mid-2000s that Amazon and eBay launched and began to thrive. By that measure, AI is only in its infancy in that stage, and its potential is yet to be fully realized. The lesson is not skepticism for the sake of it; it is having the patience to stay away from wishful thinking to make sound judgment. The wins may not even be in the areas of focus. Labkoff suspects that the value will be in the mundane things that humans consistently perform at a higher level. Things, such as paperwork, that systemically grind humans, and not in the more eye-catching things, such as AI's ability to diagnose rare diseases.

The Danger of a Persuasive Wrong Answer

Labkoff is most concerned with AI's over-persuading ability. He is concerned that dependency and irrational trust will worsen an already strained healthcare system. In his view, the systems already have incredible capabilities to achieve extraordinary things, even though today's systems will only ever get better. The hazard is the ability to persuade. Today's most sophisticated AI systems have impressive capabilities, yet experts routinely find significant flaws in the systems. The systems use language that is so fluent and flattering that a knowledgeable user can be easily misled unless they are intentionally and consciously critical of the information.

He cautions against democratising data without the contextual framework to interpret it. There are citizen scientists, as well as professionals in data science and adjacent fields, who can handle and interpret data ethically. His concern is the general public, who unintentionally consume data and make incorrect assumptions. He mentions a personal example of a friend who, upon reviewing his laboratory blood work, was convinced that he had celiac disease, despite his labs being negative. A clinician interprets the result in the context that the general public lacks, and this context is the difference between data and a harmful assumption.

This is the argument he uses to state that powerful tools do not replace the need for professionals. He is truly unsure whether a general model could replace the need for clinical expertise and is certain it could not at the present.

Not All AI Is the Same

A specific literacy failure of AI Labkoff cites is the tendency to generalise all types of AI. An example of this is large language models, as opposed to the types of AI based on machine learning that are being used successfully in radiology and other types of pathology. AI vision tasks have been shown to be better than humans at performing certain tasks. They are different kinds of technology that exist for different purposes, have different limitations, and should be evaluated in different ways.

He likens it to the early internet when people learned the difference between a brochure, a search engine, and a shopping site when they understood that they weren't the same even though they were all called 'websites'. What healthcare has reached now is understanding of its own 'AI' as one in the same and having that kind of understanding, however imprecise it may be, is exactly what will make a board say yes to sanction it or a clinician place their trust in the tool because of a classification it belongs to, rather than its deservedness for the purpose.

Understanding of Data, Not Just Models

According to Labkoff, the absence of understanding of data in relation to AI is beyond just how data is viewed; it goes even beyond the absence of understanding of data in relation to AI itself. He refers to the same discipline as a chemist who exteriorizes the results of a purified compound and pertains to what he refers to as a crystalline state, clean, complete, and veracious. Anything less than that will yield a model with a serious structural deficiency.

He claims that even the most advanced companies fail to comprehend what their real product is. Companies operating in the life sciences think their product is a pill. In reality, it is a trove of structured data like safety and trial data that is delivered to regulators. You never send the FDA a pill. According to him, companies that have internalized that data is the product are outpacing companies that have not.

The solution to the problem he describes is investment in context. He explains the situation in which one large employer's fifty data analysts included only a few who had observed clinical trials. He sent the majority of them to clinical trials because, in his opinion, one cannot fulfill the analytical function properly without the context of the clinical data. Some programs provide this kind of context-building literacy; he mentioned a fellowship that provides clinical training to executives, and the point is relevant elsewhere as well: literacy is an area of capability that an organization funds, not a baseline assumption that it can be present among its members.

How Hutchins Approaches AI Literacy

Our work treats literacy as foundational rather than optional. We help leadership teams build the understanding to evaluate AI on its merits — to distinguish the kinds of AI, to read a vendor's claims critically, to know when a confident output deserves a second look, and to give analysts and clinicians the data and clinical context that make their judgment reliable. This is the human counterpart to data quality and responsible AI governance: the best controls in the world fail if the people operating them cannot tell a sound answer from a persuasive one.

These conversations are at the heart of The Signal Room podcast, where clinical and data leaders discuss what real understanding of AI requires — and what happens in its absence.

Authoritative sources

• WHO — Ethics & governance of AI for health
• NIST AI Risk Management Framework
• NEJM AI
• JAMA Network
• ONC — HealthIT.gov