The Population the Model Cannot See

In 2019, Science published a study that should be a prerequisite for any buyer of population health software. Researchers analyzed an algorithm health systems used to identify patients that required additional care. From the surface, the tool appeared to be an unbiased method of selection. It was not. And the bias was not from the inclusion of race in the model. It was from the outcome that the model was built to predict.

The algorithm used health care costs as a proxy for health care need. Because of the historically lower health care costs for Black patients with the same level of need, the algorithm assumed Black patients were more health care needy and thus classified them as requiring additional care at a significantly lower level. Patients with the same level of risk were not equally at risk. The study determined that the flaw accounted for more than a 50% decrease in Black patients receiving the care that they needed.

The bias in the model was drastically reduced when the researchers and the sponsoring company, as one example, changed the model from predicting future costs to predicting active health care needs. The lesson is that the population health model is not doomed. What is the most important and significant decision for the entire model is actually done at the earliest and quietest stage when the developer determines what the model will predict.

COST AS A PROXY

Population health systems operate with proxies. Because there aren't any systems that can identify health need directly, they have to rely on something that they can quantify, and in healthcare, cost is the easiest to quantify. Whenever a team chooses a stand-in, they make a choice on whose need they are prioritizing, and they usually don't realize it. Choosing a proxy looks like a technical decision. In reality, the statement is made on what the organization holds important, and it is presented in a language that most executive's don't understand.

As software starts to autonomously take action, this becomes even more important. It's one thing to have a risk model that prioritizes outreach to patients. In contrast, if a system is making outreach a reality and is allocating a limited budget to that outreach, it has made the proxy reality for patients. If a proxy of a model is incorrect, the negative impact is that a patient who was in need of outreach, did not receive it, and the model justified that by determining that another patient who was in need was a higher priority.

I learned this the hard way. A team built a forecast for a newly acquired service line. The math and the model were both solid, and they were still wrong. It failed to account for the way the payer contracts in that region were different from our previous contracts, and the forecast was too optimistic. When the actual numbers came in, the difference was significant. The lesson, however, was not about math. We built the model for the business, not with the business, and the people who were aware of the regional nuances were not in the room.

This is the pattern of failure in most population health tools. A technically sound and contextually weak model is built because the clinicians and local staff who know the population were considered as recipients of the model rather than contributors to it. A model that knows the math but not the context, will be wrong in all the important places.

What Sits Beneath the Dashboard

Every population health platform contains the data that populates it. The Agency for Healthcare Research and Quality (AHRQ) has long demonstrated the inequities in data, finding that the sickest and most socially complex patients have the most disorganized data, and that people who need support the most often remain data-dead. A model that is designed to capture only the present will systematically exclude the absent. Spend millions on the highest-end model, give it incomplete data, and it will confidently produce recommendations for a population it is largely blind to.

Data gaps are often driven by inequitable social conditions. For example, a patient who has to travel long distances to see a physician and who is unable to afford the price of a visit has a sketchy data record. Most models interpret a sketchy data record as someone who is acting healthy. Therefore, a model designed to assist the underserved will end up targeting the patients who are easiest to reach. Without someone looking for that pattern on purpose, no dashboard will capture it.

Money raises the stakes. Value-based care means the same software allocates care managers to different locations in the system's limited budget. A model that misreads need doesn't just miss a patient. It sends limited resources to the wrong place and reports success while doing it. This is done because the patients that were ignored didn't create costs which would have identified them. The math looks clean because of the blind spot which is invisible to the math. This is why the proxy should get more attention than the model. A vendor can look very accurate compared to the target of their choosing and still be optimizing for the wrong thing, and no accuracy metric will prove otherwise.

This portion is unsightly, and no demo will cover it. It is also where the true value is. Before a population health tool can be trusted, a system must have the means to locate the patients in its data whom are most underserved, and locate the hidden beneficiaries in its data. That is the true data strategy, and we spend the majority of our time with health systems, because at the end of the day it is most important.

What Leaders Have to Build

Start with the goal before you use the tools. Consider what the model is actually predicting. Whom does it represent? Who is missing? Don't accept a vendor answer that hides the proxy behind a score. While the model is being developed, not after it has been deployed, include clinicians and staff members who are familiar with the community. After the model has been deployed, look for absences in the output. Were the flagged patients truly patients of concern? Which patients did the model not flag?

None of this will significantly slow the development of a truly good program. This is the difference between a tool that extends a health system's reach and one that makes it self-perpetuating. The health systems that truly derive ongoing benefits from population health software view the model as the easy part and the data and goal as the real challenge. Models are a population sort that takes mere seconds. The decision of who belongs in that population, and most importantly the decision to include marginalized and vulnerable populations, is the part that takes judgment, and should be sustained and maintained.

Background and References

This edition references the 2019 study published in Science on population health algorithms and racial bias, research from the Agency for Healthcare Research and Quality on data quality and underrepresented populations, and the value-based care initiatives that drive population health programs. This edition builds on the work featured in issue 11, The Most Dangerous Bias in Healthcare AI, issue 47, What the Model Inherits, and issue 13, From Prior Auth to Predictive Care.

Christopher Hutchins Founder & CEO, Hutchins Data Strategy Consultants