Healthcare AI Security: Why the Breach You Can't See Is the One That Hurts

Healthcare leaders should be concerned about a particular type of security failure that is more subtle, rather than the more flashy failures. The breach no one sees is more concerning than the breach that sets off the alarms. Guman Chauhan, speaking in a Signal Room discussion, said the worst situation is when there is a breach no one knows about. He then explained that the lack of an alert does not mean there is no breach. It usually means that the observers are not looking in the right place for the right signals.

As AI technology is being rapidly integrated into healthcare, that distinction is important. Anitha Mareedu, a guest speaker at Signal Room and a cybersecurity practitioner who has worked across network and endpoint defense, explains the paradox. With AI technologies, tasks that would normally take hours can be completed in a matter of minutes. However, this increased speed requires more oversight and monitoring. We, at Hutchins Data Strategy Consultants, consider this question to be of the utmost importance: how will AI be protected? We believe it is imperative to treat this question with the same importance as the implementation of AI itself.

AI Expands What Needs to Be Defended

Integrating any AI system within a healthcare environment carries the potential for that system to gain access to sensitive information, and increasingly the ability to take action. Mareedu discussed the level of access control discipline that needs to accompany the decision to provide access to systems. With the rapid development of technology, systems can be afforded capability very quickly. When access control is not managed, a risk surface has been created that no one is watching.

There are varying levels of importance assigned to confidentiality, integrity, and availability across sectors. The burden in healthcare falls heavily on the confidentiality of patient records because of the protection of patient information under HIPAA. An AI threat model that is defined generically does not address this. The way the AI systems of a bank and a hospital are controlled is very different, and an AI deployment has to be secured against the threat model healthcare actually faces.

Attackers Do Not Break In. They Log In.

The reason that incidents of undetected breaches are so prevalent is because modern intrusions do not resemble the historical definition of an attack. Chauhan explained that, rather than deploying malware, attackers use valid credentials to gain access, then move laterally and exfiltrate data while performing legitimate actions with legitimate access. Service accounts that never expire and administrative credentials allow an intruder to operate in an identical fashion to a trusted administrator, with no anomaly and no alert.

He argued that automated alerts stand in for the truth in the security industry, and that is a troubling trend. When dashboards are calm, executives believe that everything is operating within the intended parameters, and, unfortunately, cybercriminals know this better than anyone. They design their attacks in order to remain undetected and to stay within the thresholds that a busy security team expects to see. He cited the Colonial Pipeline, SolarWinds, Capital One, and MOVEit as incidents in which authentication was successful and security tooling assumed that everything was operating normally, while trusted-looking access was used to do the damage. In his experience, and unfortunately, many organizations do not know that they have been breached; they learn it from banks, customers, regulators, or law enforcement.

Most of the Problem Is Not Technology

The automatic reaction to a breach is to install better technology. However, in his view, based on what he has seen, this is not correct. He believes that technology is responsible for about 30 to 40 percent of the detection issue. Most organizations have already acquired the high-end tools. The larger responsibility rests on the processes and the people. This includes unproven or untested workflows, unclear escalation paths, an unexercised response to incidents, and analysts who are reluctant to respond because they are fearful of chasing a false positive, while leadership quietly prioritizes uptime over fast containment.

You have to look out for alert fatigue. In situations with a high volume of alerts, a patient attacker can easily stay below the threshold of attention. Chauhan's approach to combating alert fatigue is to cut the noise rather than increase it. He has seen organizations decrease alert volume and even improve detection after eliminating rules that produced no actionable events in a year. He proposes the shift from volume-based alerting to a risk-based question: does this activity meaningfully increase business risk? That reframing is what turns a flood of signals into something a human can act on.

Visibility Over Perfection, Reality Over Assumption

Two of Chauhan's principles provide tremendous value when incorporated into any healthcare AI security program. The first is that visibility is more important than perfection. A breach that you can see is damaging and unavoidably painful, but a breach that you cannot see is devastating. You cannot protect assets you do not know you have or logs you are not collecting, and a striking number of breaches trace back to a system that was assumed to be internal and was in fact exposed.

The second principle is that defenses must be verified in the field, not assumed. His image for it was that ships are not built to stay in harbor; you do not learn how strong your defenses are by leaving them unstressed. Independent, external validation shows reality instead of assumption: which systems are actually public-facing, which old credentials still work, and whether the alerts you are counting on actually fire during an active exploitation. For AI systems specifically, which sit on sensitive data and may act with autonomy, that validation is not a luxury. It is how you find out whether the security you believe you have actually exists.

How Hutchins Approaches Healthcare AI Security

Our work treats security as part of the deployment decision, not a review bolted on after go-live. We help organizations scope the access an AI system genuinely needs and lock it there, align controls to the confidentiality burden healthcare actually carries, and invest in the process and people dimensions — escalation, tested incident response, and alert hygiene — that determine whether a breach is caught or missed. And we push for validation against reality rather than assumption, because an untested defense is just a hopeful one.

This is continuous with data governance and the responsible AI oversight that decides whether a system is safe to run at all. As agentic AI gives systems more autonomy, the cost of an unseen compromise only rises.

These conversations run throughout The Signal Room podcast, where security practitioners and leaders describe how breaches actually happen — and why the quiet ones are the ones to fear.

Authoritative sources

• U.S. Department of Health & Human Services — HIPAA
• NIST AI Risk Management Framework
• ONC — HealthIT.gov
• HIMSS