Operationalizing AI Ethics in Healthcare: Ways of Working

Asha Mahesh stated that ethics is not something you write about and then forget. She said this during a Signal Room conversation recorded at an AI conference in Las Vegas. As a life sciences technologist with a background in ethics and privacy, she repeatedly brought the conversation back to actions and away from documentation. For Mahesh, ethics is about the way you work and what you actually do when you build and deploy AI in a clinical environment. For us at Hutchins Data Strategy Consultants, this is important because the gap between published principles and daily habits is where most 'responsible AI' efforts die.

Deliberately, this article remains constricted in focus. There is extensive documentation about the formal assembly of AI and the supporting people like committee members and the review and monitoring boards. This more human question is once that structure is established, how does ethical AI become the way this particular team operates on a Tuesday afternoon?

Ethics Lives in Behavior, Not Papers

Mahesh argues that responsibility is built, not declared. Whether consciously or not, organizations can operate in direct contradiction to their stated values. Values that are framed and hung up in a conspicuous place have no influence on the choices of individuals, unless those values are translated and carried into work by the people. She used as an example her company a genuine lived values system as opposed to framed values that are neglected, showing once again that it is the lived values system that is of importance as opposed to the values that are framed.

The mechanism that best describes the practice of culture, followed by what is described as recognition. Within businesses, to make ethical practice something that is long lasting, this recognition must happen. It showcases the practice in practice to the people, regardless of whether you think it might sound soft. You must consider the alternative practice of what happens in the majority of businesses, where what gets valued and recognized is what gets the most done and most completed. Recognition tells the people what the organization values, and what the purpose is of the practice.

This model of practice conveys a burden of leadership. For a team to bring ethics into the scope of their job, the people above them must exercise this ethics consistently. Without this, anything said and done regarding ethics turns to façade.

Start From the North Star, Not the Technology

When teams become enamored of the technology, ethics start drifting away and become abstract. Mahesh was frank in stating that, like herself, many in the field gravitate toward the data and the tools because they are interesting. And in the absence of clear purpose, a team will sprint toward an attractive option without regard for the end user. It is a pull toward technology that lacks purpose, and the North Star is the counter to that pull.

It seems to her the real test of technology is its intended use. Is it serving a need and who does it serve? This is not a philosophical question. It's the question that determines the viability of a project and should be asked in the preliminary stages of a project, not in a post-design review. She described this pairing of science and what she called 'heart' as acknowledging the technical component of a problem, but judgment at the engineering stage is critical to value the human element of a problem.

An example she provided of purposeful work was the work her teams did using data and machine learning to determine the most optimal locations for vaccine trial clinics. Apparently, this work enabled the completion of trials in a much shorter time span and with a smaller number of trial participants. Despite the lack of external validation of this work, the rapid progress is attributed to the collective understanding of the purpose of the work and not the technology used.

Adoption Fear is Also an Ethical Concern

The most productive part of the discussion was centered around fear. When a scientist or clinician hears AI is being developed for a task they have been mastering for the last ten years, they assume what the AI will do. Mahesh attempts to address that fear and describes a framework called "what's in it for you, what's in it for me."

The plan is to show the tool's benefits from the point of view of the person you are assisting and not to show them what the tool can achieve in the abstract. Mahesh shared an example of the clinical development lead and how she reassured him that nothing will replace the extensive medical school training and practice, as AI will help ease the burden of other less demanding, time-consuming tasks. Mahesh claims overselling the AI and claiming it will resolve all the problems will only reinforce people's skepticism, as overselling the capabilities of the AI is not hard to see through.

Mahesh's approach to adoption fear aligns better with the concept of ethics in practice, as opposed to explaining away the fear of adoption. When introduced to a fearful and resistant team, a tool is most likely to be subverted, worked around, and distrusted, and all of these result in an unsafe practice. Honest and person-focused adoption is not an ethical add-on, but is in fact an ethos framing a practice.

Human-Centered Design, and the Limits of Automation

Mahesh evaluated the human-centered standard for designing products. This means with every product designed, we should consider if the product will resonate with a real person at the point of use. However, Mahesh evaluated the limits of human-centered standard designing. Mahesh gave the example of a hospitalized patient. Most of the surrounding processes of the patient's experience in a hospital room can be, and in many cases, should be automated. However, even the presence of a hospital staff member (especially a nurse) who holds the patient's hand and tells the patient that everything will be okay is of critical importance (and is where positive outcomes, in this case, are greatly influenced). This presence should not be automated.

Many believe that making something more human-like (for example, an AI system) is always a good thing. Most of the time, this is not the case. Care (and good design) is not about creating a simulation in the design. It is where we (the humans) use our own discretion in a moment and understand that the moment should remain human.

Keep the Human in the Loop Actually Awake

Some of the best advice surfaced towards the end of the discussion. In AI diagnostics and clinical decision support, teams usually state there is a human in the loop who is a clinician reviewing the output prior to the decision. Mahesh described the regulators' concerns for this mental model of comfort: An individual can come to rely on a system to the point of believing that the system will always be correct, and in doing so will stop doing the work of the system and will stop looking for how the system might be incorrect. So the regulators were asking what preserves the human's responsibility to do the thinking of the system.

The question changes "human-in-the-loop" from check-the-box compliance to review design. Oversight only works if the reviewer remains a check and not a stamp, and expecting the reviewer to be passive is the inevitable breakdown of a system that is correct most of the time. Designing for sustained attention, uncertainty, and what to confirm, and creating friction, is a work design question, not a policy design question. It's the type of detail that is bypassed entirely in an ethics statement but will give the indication of whether the ethics is being practiced.

How Hutchins Approaches Operationalizing AI Ethics

Our work starts where the ethics statement ends. We help healthcare and life-science teams convert principles into the daily practices that make them real — the adoption conversations that lower fear honestly, the human-centered design standards that decide where automation stops, and the oversight habits that keep a human reviewer genuinely engaged rather than complacent. That work sits alongside the structural side of responsible AI in healthcare and the AI literacy that lets staff know when to question, override, or escalate a system's output. These themes run throughout The Signal Room podcast, where practitioners describe what ethical practice looks like when it has to survive contact with a real working day.

Authoritative sources

• WHO — Ethics & governance of AI for health
• NIST AI Risk Management Framework
• FDA — Good Machine Learning Practice for Medical Device Development
• Agency for Healthcare Research and Quality
• NEJM AI