AI Pilots Win: Why small AI tests succeed while full programs often fail

I have experienced many mergers, and I have learned there is no point in trying to anticipate the surprises that come with each new merger. However, I have observed that many teams believe that if the two organizations have similar systems, the initial stages of the merger will be simple. The reality quickly disproves this belief as teams learn to work with the combined data of the two systems.

This is not only true for systems, but also for EHRs. There may be similar EHRs in two organizations, but there are also many different ways to operate a health system that are not visible at the outset. These will become apparent when you start merging units of the organizations and when you begin to exhaust the more obvious methods of process standardization.

There are limits to the integration of differing systems and structures, and you will not be able to unify differing personnel and service lines. I have experienced exactly the same things in similar organizations that had comparable service mixes and sizes. There will be a wide array of outlier performance metrics across the sites, and the data quality issues will only add more complexity to the situation.

Initially, the metrics that are the most visible, such as length of stay and readmissions, will be the first to raise a warning flag. That is not to say that these are the only metrics that will draw attention. But because they are the first to be visible and because people have been trained to recognize when they operate normally, lack of performance will prompt a discussion that few people are ready to have at that stage.

Normally, the first response of the team is to reconcile the discrepancy, which is perfectly understandable, as one manually goes through the calculations, checks that nothing is wrong with the data pipeline, and finds that reconciliation is necessary, and while it can be helpful, it does not address the real problem, which is not with the numbers, but with the nature of the work.

One of the incidents that I can recall very well is one in which we continued to reconcile a number and it took us a very long time to realize that we were looking at different ways of operating. It was not really an error, but rather different ways to approach the patient flow that most people thought of as the standard operating procedure.

Once this happens, everything changes and you are no longer involved in fixing broken things, but in understanding things that were built intentionally. This is a whole different challenge, especially since the builders are around and can explain the rationale.

Understanding the challenge here is a lot more difficult to explain because people outside of the challenge usually have no idea how to explain integrating systems to someone who has no experience of it. If anything, outside of the challenge, it seems integrating systems should resolve most of the challenges. In reality the opposite is true and most of the time is spent describing the discrepancies of the systems and the actual work is very little.

You can tell this happens based on the different ways people begin to start responding to the data. This may or may not be a significant initial response, but questions to a greater degree, validating data and even some hesitation to verify a given number. The feeling of momentum is really not a product of technology.

I have seen teams react to this problem by trying to over-standardize. There is an inclination for individuals to try to simplify the distinctions and say, pick a definition, standardize everything to that definition, and then we can get back to progress. While I appreciate the objective to restore faith, it is an attempt to remove the process of whatever little context might be left. This also causes further complications that are even more challenging to identify and address later on.

During past integrations, there were moments everything lined up and looked great on the analytics, and the people closest to the work were telling us what was going on was not what they were experiencing. It must not be a great feeling since you have lost both sides. There is no variability and there is no faith.

What worked in this situation was not getting more technical, but getting out of the data and into the operations. This is apparent in hindsight, but does not feel so when you are in the thick of it trying to keep things moving, and required spending time with teams that did not expect to be integrated at this level.

Such conversations do not have a formal structure. Neither do they end quickly, and they expose a host of undocumented issues. These include informal decisions that people make every day, but which inevitably produce consequences.

Once you note those patterns, you start realizing that the differences are not random. That is when the conversation changes, not by any formal announcement, but by a changing focus from attempting to resolve discrepancies to comprehending the functional realities of the organization.

The absence of visible changes does not imply that a section of the work is unimportant. This is the crucial section that determines whether the integration is successful. This section will allow you to not have to continually deal with the same problems every time something new is introduced into the work.

Since the bulk of the preliminary work done prior to the first acquisition is just project management, I no longer look at this as a project. The moment an acquisition takes place, it functions like a project that is perpetually ongoing. Each acquisition contributes still another unmanageable layer to the organization.

Some of the organizations I am familiar with seem to become more adept at handling a particular complexity over time. This is not a case of the complexity becoming less, but rather, the organization has learned the different ways to deal with the complexity while maintaining confidence in the data. An example of this confidence is shown in the speed at which they go from, this does not match the expected outcome, to, this is the reason it does not match, without the process becoming an extended discussion.

What you have learned from the experience is the confidence to deal with the complexity of data without having to rely solely on tools and architecture, which is something that most organizations do not possess.

Christopher Hutchins Founder and CEO, Hutchins Data Strategy Consultants