The Procurement Trap

A health system with 540 beds signs a three-year deal for an AI Operator for the prediction of sepsis post an eight-week pilot at one of their hospitals. Vendor Board materials showed a 30% decrease in mortality at four hospitals where the product was not yet implemented, and this inspired the Board to approve the contract (the sites were probably omitted to avoid model drift). The contract is worded per standard healthcare IT contract. An EHR (like a model) is defined in the contract as a code change or user interface change. 18 months post deployment of the tool to four hospitals, model drift resulted in a predicted mortality reduction of -6% and still decreasing. The Board was approving the deal, as there were no alternatives presented.

The decision of the Board to procure the tool 22 months back is what the health system is bound to for the next three years.

What the Pilot Phase Obscures

By design, pilots succeed. Vendors design the pilots to their preferences, controlling the cadence of implementation, the scope and extent of integration, the refresh of training data, and the criteria to determine success. Reference calls are made to early adopters. Those early adopters have deployed the product so recently that they are unable to recognize the drift phenomenon. The datasets that are utilized in the pilots are pre-cleaned. With respect to the data and clinical workflows, there is a decoupling. The workflow changes are not incorporated in the pilot and will not occur during the modeling exercise. The duration of pilots is kept short to avoid drift.

What is not disclosed during the pilot is how the long-term needs of the local patient population will be addressed. One of the most prominent concerns in the Coalition for Health AI Blueprint, Draft 1.0, is the contrast between expectations and day-to-day reality caused by poorly structured pilots. The AHRQ patient safety reports mention the lack of post-deployment assessment and recommend contract stipulations to include both monitoring and evaluation. Most health care organizations end up signing the pilot contracts, which limit the monitoring and evaluation to the pilot phase.

Reference calls also cannot solve the problem. Vendors create reference lists with systems at the beginning of the deployment. These systems are less probable to have gaps. When reference calls are made, the system that is likely to fill procurement gaps is the one that is the most developed, and is most likely the one not on the reference list. For a new AI procurement team, it is usual to reach out to other vendors that have their first AI systems in the active state to conduct due diligence. It can be safely assumed that the two vendors are outside of the renewal cycle.

Un-negotiated Terms of the Contract

The standard one-time-configured-runs contract language for healthcare IT, EHR, and similar systems has been modified for AI. Contract language for AI models should include and define certain clauses. This would include who gets to decide the timing of significant changes that would justify an update, the funding for these changes, the performance evidence the vendor will provide, and how this performance will be most adequately measured. The right to cease remedy must be contractually agreed. The performance related Service Level Agreements must withstand the cycle of renewal.

The ONC HTI-1 Final Rule requires predictive decision support systems to make their training data sources known. This gives buyers the opportunity to form better questions around the purchase. However, simply disclosing the information does not solve the problems associated with the contracts. In situations where the HTI-1 disclosure has been adopted within the evaluation frameworks, the training data questions that vendors used to ignore are now to be answered, and it is expected that the answers will necessitate changes to the contracts. Most evaluation frameworks created by procurement teams do not as of yet include the HTI-1 disclosures.

There is a separate, but related issue, that the vendors maintain all of the intellectual property for any training-related improvements that they make. The vendor of the health system retains the improvements to the model, and other clients must pay to access the improved model. The health system whose data was used does not access the improvements. This is in relation to the augmented intelligence policy of the AMA. While there are the oversight concerns of AI vendors and the IP issue, these contracts remain unresolved by policy as of 2026.

The Renewal Math

Renewals occur 18 to 24 months after the signing of the initial contract. Currently, model drift has reached a level where the vendor considers it acceptable, while the health system does not. The inclusion of model outputs to the clinical teams has resulted in the integration cost. The first contract has no cost associated with undoing the work.

Vendors know the costs well. Renewals are done with the notification of operational costs, or the next higher tier has a greater functionality. Often, the health system renews on the terms the vendor sets, with the alternative being a costly Integration Write Off. Most systems renew.

The ECRI Institute has stated that the greatest risk for 2025 is the implementation of AI in health care. The most likely cause of this will be the lack of management during the lifecycle of a contract, especially during the renewal phases. The risk manifests across the geography and bed capacity of a health system. The main reason for this is the reliance on standardized contract templates. The work published by the Robert J. Margolis Institute on AI Committees illustrates that the most important work done by the committees that survive the longest is vendor management.

The Procurement Management

The terms of reference conclude the discussion. After 18 months, the board will have little to no recourse. CAIOs and the legal team, along with the CFO, CISO, and the procurement team, own the contract terms. The five AI Readiness Diagnostic questions become procurement questions. What is the halt authority of the contract? Where is the local validation of the contract assessment clauses? Who will own the IP improvements in three years? What will the exit costs be at the 6, 12, 24, and 36 month marks? What is the pricing for the renewal timeframe at the time of contract signing?

The AHA Trustees Toolkit on AI in healthcare indicates that vendor selection should be included on the 2024 board agenda items; however, most procurement teams have not integrated these questions into their evaluation standard. Within the Coalition for Health AI, Blueprint v1.0, procurement gaps are addressed with a shared evaluation framework that employs a multi-system, pre-purchase evaluation as a counterbalance to the vendor-led reference calls. Vendors promise a 30% reduction in sepsis mortality, and the promise is validated with each iteration of the pilot. The renewal occurs in 18 months with new pricing, unexpected integration costs, and offers the potential for procurement to begin the pilot. Transformational change occurs with the presence of the CAIO, completion of the readiness diagnostic, the procurement office in agreement, and an optimized AI procurement contract.

The board is taking the renewal with the understanding that it is the least unfavorable choice. The changes to procurement for 2026 will influence our understanding of the options available for 2029.

Context and Sources

This edition is informed by the Coalition for Health AI Blueprint (Draft 1.0 and v1.0) on pilot evaluation and shared evaluation methodology, AHRQ patient safety reports on post-deployment assessment, the ONC HTI-1 Final Rule on the disclosure of training data for predictive decision support systems, the AMA policy on augmented intelligence and the oversight of AI vendors, the report of the ECRI Institute on the Top 10 Health Technology Hazards for 2025, the Duke University Robert J. Margolis Center for Health Policy regarding AI committee membership, and the AHA Trustees Toolkit on the implications of AI for health care in regard to vendors. Related editions: The Chief AI Officer Mandate and The AI Readiness Diagnostic.

Christopher Hutchins Founder and CEO, Hutchins Data Strategy Consultants