Building Ethical Intelligence in Healthcare

Typically, the term that comes to mind around this is governance. Unfortunately, it is loaded with images of committees and audits. That is the real issue. The actual concept is a lot more dynamic and engaging. Good oversight is not about filling out forms. It is the mechanism through which trust circulates within an organization. Good oversight is the mechanism through which decisions made and actions taken in one area of the organization can be trusted to be accepted and relied on in all other areas of the organization.

For the last several decades in healthcare, oversight meant committees, audits, and a series of gates you had to pass through to get anything done. That model was built for a different, slower time. AI moves far too quickly for that model. One of the many ways AI is different is that it trains a model and then gets applied to populations that it has never been tested on. Several months later, a review committee finds out what has happened. Oversight is needed, and should be modeled as participatory and dynamic rather than an afterthought.

That is what ethical intelligence means. It means that rather than slowing the organization down when building AI products, it enables the organization to build AI products quickly and make ethical decisions at the same time. In that sense, oversight is not the thing slowing you down. It is the steering wheel that allows you to build AI products quickly and ethically, while still following the rules and not causing harm.

From Control to Clarity

The instinct for control is a list of prohibitions. A more useful instinct is clarity, a common understanding of what the ideal outcome is that is incorporated into each step of building the system. Ethical reasoning is expected to be incorporated into each step of the construction process that includes the selection of a dataset, the selection of an algorithm, and the interaction of the tool with the end user.

It is important to maintain the distinction the way the law does. The law defines what is legal. It does not define what is ethical. Ethical intelligence helps bridge the gap the law intentionally leaves open. A tool can be 100% legal and still be the wrong tool to create. Ethical intelligence is the connective tissue that runs between the data and the decision, and it does the work the law cannot.

There is a change when this becomes part of a culture rather than a system that people have to endure. AI does not just become safer. It becomes better. The same thought that encourages AI to not bring about harm also encourages AI to stay on its intended path.

Three Layers

Looking at this on three different levels is beneficial because most organizations excel at one of these levels at the most.

The first level is structural. These are the systems and policies that define the boundaries. While they are important, by themselves, they are unchangeable, just another binder on a shelf.

The second level is behavioral. This is the sort of culture that is developed, and the systems that are put in place to help teams act when the supervising authority is not present. Structure says what the rule is. Behavior decides whether anyone follows it.

The third level is contextual, and this is the level most organizations are deficient in. This level contains the sort of systems that allow supervision to change on the go as the intended goals change. An oversight system that was put in place in the spring is already outdated by the fall. A contextual organization stops reacting to problems and starts anticipating them, preventing harm while it is still a possibility, rather than an incident.

Who Owns It?

Most organizations have not addressed what is perhaps the most practical consideration. Who owns the outcome? When AI-assisted choices lead to undesirable outcomes, the truthful response in most organizations is that no one knows. Responsibility gets distributed across the tech group, the operations group, and a committee that meets once a quarter. Ethical intelligence proposes to designate an owner prior to failure. The absence of a clear owner is a decision to allow the gaps to determine outcomes. In such scenarios, responsibility simply evaporates.

Design, Not Defense

The deeper shift is building ethical consideration in from the product conception phase through the entire design, rather than bolting it on at the end. When engineers, ethicists, and clinicians occupy the same space, the question is no longer will this pass an approval gate. It is whether this is actually the right thing to do, and whether it is done the right way.

Fairly assessing the tools we have at our disposal requires that we accurately describe what is actually present in these systems. Each data set contains value judgments regarding what data was collected and what was omitted. Each model optimizes for one type of tolerable failure over others. Bias is what is inherited by default, and purpose is what is intentionally placed.

The core issue is that we can always build it. The question is whether we ought to. The answer to whether we can build this gets easier every year. More important is how we build it, and that is part of the mission of Ethical Intelligence, to answer how we should build it.

Why It is Worth the Time

Healthcare cannot sustain another experiment with AI. We need more people able to weigh the rush of technology against ethical standards. More than anything, we need people able to shape the technology in a positive direction. The systems and data we create will either be ethical and compassionate, or they will be devoid of ethics and compassion. We just need to make the right decision when we build these systems and technologies.

This oversight may appear slow, but it actually builds trust. Ethical reasoning and deliberation definitely create the trust you need to make the technology safe to use. Ethical Intelligence will create the technology and the trust you need to successfully integrate AI in healthcare.

Christopher Hutchins Founder and CEO, Hutchins Data Strategy Consultants