Most healthcare organizations have moved past the question of whether to adopt AI. The question now is why the investment keeps falling short of the return — and what's getting in the way.

Ramani Narayan · July 2026 · 6 min read

The demos keep getting better. At every conference, every vendor briefing, every internal proof-of-concept, the clinical AI is impressive. It surfaces the right information. It drafts a coherent note. It spots a care gap that a physician would have missed. The roomful of clinicians nod.

And then six months later, the same organization is running the same manual processes it was running before, with a pilot report buried in a shared drive somewhere and a lingering question about why the thing that worked so well in the demo never made it into the workflow.

This is the adoption gap. It shows up consistently enough across enough health systems that it is worth treating as a structural problem rather than a failure of individual deployments. Three forces drive it. They are related but distinct, and each one has a specific technical root cause that a vendor pitch deck rarely addresses honestly.

We've spent two earlier posts on why the return on generative AI is so hard to pin down. In the first, we argued that the benefits of GenAI are remarkably easy to overestimate — they are felt locally, prediction by prediction, while the headline number is booked in aggregate. In the second, we made the opposite case: that the cost and effort of delivering AI as a production system are just as easy to underestimate, because AI infrastructure behaves nothing like the predictable, fixed-cost software it is usually procured alongside. The adoption gap is what happens when those two errors meet a real care environment. An overestimated benefit and an underestimated cost collide in production, and the deployment stalls. The three forces below are the specific shapes that collision takes.

The AI isn't the problem. The AI has never been better. What's failing is the infrastructure between the AI and the care environment — and that failure has three very specific faces.

1 Pilot-to-Production Decay

Every healthcare AI initiative starts the same way: a defined use case, a motivated clinical champion, a vendor who has done this before. The pilot runs. The metrics look good. The readout goes to leadership. And then someone asks the question that kills more healthcare AI deployments than any technical failure: “How do we scale this to the rest of the organization?”

The answer, in most cases, is to rebuild it. Not because the AI didn't work, but because the infrastructure it ran during the pilot — a curated dataset, a custom integration, a hand-tuned prompt pipeline — was never designed to generalize. Each new use case requires a new integration. Each new department requires a new configuration. The thing that worked in cardiology requires a different setup in oncology, a different one in primary care, and a different one for each EHR version.

There is a second reason pilots fail to survive contact with production, and it has nothing to do with integration plumbing: the benefit was overestimated in the first place. As we argued in our ROI series, the value of GenAI is easy to overstate and hard to measure — errors are felt locally, prediction by prediction, while the win is claimed in aggregate. A pilot compounds the problem, because it is run under near-ideal conditions: a curated cohort, an engaged champion, and a use case chosen precisely because it flatters the model. The measured lift reflects those conditions, not the messy average of everyday care. When the same tool meets the full patient population, the real benefit regresses toward something far less impressive than the demo promised — and a scale-up justified on the pilot's numbers no longer clears the bar. Pilot-to-production decay is therefore not only an engineering problem. It is also the distance between an overestimated benefit and what the tool delivers once the flattering conditions are gone.

Taken together, these are the pilot-to-production decay. The pilot delivers proof of concept. What comes next is a project — and that project costs more, takes longer, and, too often, must justify itself against a benefit that was never as large as the pilot suggested.

2 Uncontrolled Cost of Tokenmaxxing

Most clinical AI deployments send too much data to the model. The logic feels right: more context should produce better answers, so give the model everything it can see — the full record, all the notes, every lab going back fifteen years. In practice, this turns every use case into an escalating inference bill while making the outputs worse. Models perform better with focused, relevant context. Flooding the context window with irrelevant content dilutes the signal and inflates the cost simultaneously.

Tokenmaxxing is the most concrete evidence we have that GenAI cost is genuinely hard to contain — not as an abstraction, but line by line on the invoice. In our ROI series, we argued that AI infrastructure is fundamentally difficult to characterize and scale: unlike traditional software, where the cost per transaction is stable and easy to forecast, compound and agentic AI systems turn every request into a variable-cost event. Tokenmaxxing is that unpredictability made painfully specific. Every token of context carries a price, so a decision as innocuous-sounding as "just include the whole chart" silently becomes a per-call tax that scales with every patient, every clinician, and every use case at once. Cost stops tracking the value delivered and starts tracking raw data volume — and in healthcare, data volume is effectively unbounded. An organization that cannot govern its context cannot forecast its own inference bill, which is exactly why so many deployments that pencil out cleanly at pilot scale become financially indefensible at production scale.

There is also a HIPAA dimension that rarely surfaces in this conversation. The minimum necessary standard requires access to PHI to be limited to what the specific task requires — not what the system can reach, not what permissions allow. Sending a patient's full historical record to a third-party API for a question about their current blood pressure isn't just expensive. In many architectures, it isn't compliant. Governing context isn't cost optimization. It's a compliance requirement that most deployments haven't solved.

3 No Sovereignty Over Data Assets

Most health system leaders can articulate this one as a feeling before they can articulate it as a technical problem. Patient data, clinical knowledge, care pathway logic — it's all leaving the perimeter and going into systems the organization doesn't control, with limited visibility into what happens next. The vendor has a BAA. The data is technically protected. But a BAA is a legal instrument, not a governance architecture. It doesn't answer ”what data went into that recommendation, where did it go, and who can audit it?”

The deeper issue is strategic, not just regulatory. A health system that feeds its clinical data into a third-party model is renting AI capability. One that builds its own data layer — with provenance, residency controls, and the ability to incorporate institutional care pathway logic — is accumulating it. The difference compounds over time in the same way it did with the EHR: slowly at first, then decisively.

Why All Three Point to the Same Root Cause

Pilot-to-production decay, tokenmaxxing, and loss of data sovereignty look like different problems. They have different symptoms, and they surface in different conversations — the first in engineering, the second in finance, the third in legal and compliance.

But they share a root cause: the absence of a governed, normalized, ontology-grounded data layer that sits between the raw clinical record and the AI applications consuming it.

Without that layer, every AI initiative starts from scratch — pilot-to-production decay. Without that layer, context can't be scoped efficiently — tokenmaxxing. Without that layer, data provenance and residency can't be enforced — sovereignty failure. The three forces are symptoms of the same architectural gap.

What “Convergence” Actually Means

The frame that cuts through all three problems is convergence: the integration of the application layer, the AI layer, and the data layer into a coherent architecture rather than three independently procured capabilities bolted together after the fact.

In most current deployments, these three layers are procured separately. The EHR vendor provides the data. A clinical AI vendor provides the model and application. A cloud infrastructure vendor provides the compute. Nobody is responsible for making the three work as a governed system. The data layer doesn't scope context for the AI layer. The AI layer doesn't enforce the governance policies that apply to the data layer. The application layer calls whatever APIs are available and hopes the result is compliant.

Convergence means the data layer is designed to serve the AI layer: normalized, annotated, scoped, and sovereign. The AI layer is designed to consume the data layer: grounded in what the data layer can verify, bounded by what the data layer is allowed to share. The application layer is the beneficiary of both: fast to build, cheap to run, auditable by design.

The organizations that close the adoption gap will not be the ones that found the best model or the most capable vendor. They will be the ones that built — or partnered to build — the right foundation. That foundation is not glamorous. It is invisible when it works. But its absence is the reason most clinical AI investments are still waiting to pay off.

This is the first post in a four-part series. The next three go inside each force separately — what it looks like in practice, why it persists, and what it takes to resolve it. Next up: pilot-to-production decay.

ThetaRho (thetarho.ai) builds clinical AI infrastructure for healthcare organizations. RISA is our clinical intelligence platform — HIPAA-compliant, AICPA SOC certified, and live on the athenahealth Marketplace.