Patient Advocacy in Healthcare AI: A Seat at the Table

A mother carrying her son's routine care team's notes about a rare metabolic condition walks into an emergency room while traveling. The clinician on duty does not read the notes. After treating what he thinks is a likely stomach bug due to low glucose and providing standard care, he sends the family home. Hours later, the child is unresponsive. A second hospital, who takes the time to read the notes, calls the original doctors, and provides the care that is needed.

This comes from Amanda Roser, who came to The Signal Room to present a rare perspective on healthcare, that of a parent of a child with a rare disease. As someone from a marketing background, she does not come from a medical background. However, after five years of her own son's care coordination between endocrinology, genetics, nutrition, and metabolic specialists, she is, in her own words, an expert in one thing — her son. Her perspective has opened her and others' eyes to the gaps in healthcare systems, and to the AI now being built on top of them.

The Caregiver Is an Uncredited Member of the Care Team

Roser describes acquiring a job title no one offered her: caretaker, record keeper, and the person who has to push specialists to communicate. When her son was diagnosed in 2021, she presumed the health system was son-centric. She came to see it differently — each specialty operates within its own silo, exactly as it should for deep expertise, but nobody owns the space between those silos. That space is where the family ends up living.

This is an informally assigned, unbound role with no specific placement in the workflow. Yet, the coordination is real and constant, and when it breaks down the patient suffers. Roser directed her comments to caregivers and health professionals. Caregivers should be considered in the same light as health professionals, and their input should be treated as more than what is clinically presented.

The Translation Burden Is Invisible Work

Roser learned there was almost no published information about her son's specific genetic disorder, by her account, a single clinical journal article. She read it, and spent late nights, researching and familiarizing herself with the acronyms and diagrams that she had to compare to high school level biology. The learning curve, she says, was real, and the stakes were her son's health.

Learning a new vocabulary is usually not considered to be part of caregiving. However, it is a significant part of a family being able to advocate for themselves. When Roser did not understand the language, she learned it by writing private blogs, a caretakers guide, and LinkedIn posts. Other caregivers may also benefit from clinicians showing them the few existing rare disease patient communities and organizations. She credits small groups specific to a condition and an organization that focuses on her son's condition for making the climb easier. Without them, she would have been climbing the mountain alone.

The System Forgets, So the Family Repeats Everything

Roser states that a major frustration of the continuous cycle of complex care is the repetition of a complete medical history for each new appointment. At times, a physician would ask extremely specific questions related to a developmental milestone such as when a child took a first step or first food, and the caregiver must recall laboratory work that was done four or five years ago. Roser says she is not a machine and cannot remember everything perfectly, thus information must be omitted, not because she is careless, but because there is no place to hold all the information.

She identifies a part of the problem as being structural. There are many disparate ways that information arrives: a note from an external lab arrives, a result that is either faxed or emailed, or a note left in a section of a portal that a different clinician never accesses. The information is presented in such a way that a new clinician cannot easily integrate it. Her proposed solution is simple and specific. She proposes an executive summary in the clinician portal, which would be a prioritized summary of the last several years of notes and documents, available to clinicians in summary form, so as to optimize their time while allowing them to access the information in greater depth, if necessary. She is careful to say that she is not requesting a twenty-page document of the medical history. She is requesting something which is commonplace for other complicated fields: a one-page summary.

Patients Are Already Using AI — Quietly

One of the most interesting parts of this conversation is that this particular story of artificial intelligence adoption is coming from patients and not the healthcare system. Over the last couple of years, Roser has trained a general AI tool on her son's day-to-day reality, including but not limited to: what healthy days look like, what sick days look like, what his lab results showed yesterday, and what his lab results showed four years ago. She uses this tool to help analyze clinical notes and lab results, as well as to prepare for an even more challenging task of convincing a physician to move outside of the traditional clinical path.

Roser utilized an AI tool to analyze labs while remaining cognizant of the limitations of the AI. During a five-day hospital stay due to physician shortages, Roser described feeding labs to the AI tool and asking it to synthesize beyond the obvious. She recalls a physician encountering the tool's output and being impressed by the alignment of the AI's conclusion to the physician's clinical reasoning. She describes the output as suggesting a possible option to pursue. Two things stand out from this story. One, she describes the AI tool as stating modestly that it is not a physician, and she appreciated that boundary. Two, she came away convinced that most clinicians do not yet realize patients are using AI this way. She hopes that clinicians perceive AI as an adjunct and ultimately carve out space for AI in an overburdened but high-functioning system which balances the current resources without compromising the system's standards of care.

Design for the Operator, Not Just the User

Roser makes a constructive distinction that families do not simply use the healthcare system; they operate it on a daily basis. In this light, they are more like system operators than passive end users — operators know exactly where the process is breaking, as they run it under real conditions which the designers cannot see. She cites two examples from her own life to illustrate this point: a very close call with a badly handled emergency dismissal; and a three-week administrative hold that turned out to be the fault of a single word. She called what she wanted a "transfer of care." However, the system was interpreting it to mean "second opinion," which was a request that was routed down a slower path. A physician later explained that the fix would have been one direct call to her endocrinologist. She learned this late, as nothing in the process was designed to tell her this.

These examples illustrate some of the typical consequences of constructing a system based on specialties and handoffs, as opposed to a system designed for the end user. Coupled with an accelerating AI build on top of this system, the same blind spots are likely to be automated and scaled. Those absorbing the costs are best placed to identify the flaws. Roser noted that patient advocates are underutilized and suggested the existence of a patient advocacy panel at a recent conference was the type of opportunity the field needs in abundance.

How Hutchins Approaches Patient Advocacy in Healthcare AI

We start from the same premise Roser names: the people who live where care actually happens hold knowledge a design team cannot get any other way. When we help healthcare organizations assess whether their data and their AI initiatives reflect real conditions, the caregivers, coordinators, and advocates living with the workflow are among the first voices we want in the room — because they can point to the gaps an internal team has stopped seeing. That work connects directly to designing clinical AI for the conditions patients actually have and to the governance and oversight that keep AI accountable to the patients whose data it uses. These themes run through The Signal Room podcast, where practitioners and the people inside the care experience describe what responsible, human-centered design takes in practice.

Authoritative sources

• WHO — Ethics & governance of AI for health
• NIST AI Risk Management Framework
• ONC — HealthIT.gov
• Agency for Healthcare Research and Quality
• U.S. Department of Health & Human Services — HIPAA