From pilots to performance: what it really takes to scale AI in healthcare

AI is the most discussed topic in healthcare right now. It is also, by some distance, the most misunderstood.

I sat down with Dr. Lacy Knight, System VP and Chief Health Informatics Officer at Piedmont Healthcare, and Sara Vaezy, Chief Product and Technology Officer at Chartis, for an honest conversation about what AI transformation actually looks like on the ground with all its complexity, its promise, and its risk.

Four things stood out:

1. A coherent market story isn’t a communications exercise. It’s a scaling tool.
2. Start with the problem, not the technology.
3. AI is non-deterministic. Build for failure, not just success.
4. Scaling AI is an organizational challenge, not a technical one.

1. Start with the problem, not the technology

“Don’t try to cram a solution into a problem you didn’t bother to examine,” warns Sara.

The organizations getting AI right are not starting with the technology. They are starting with the problem, and the distinction matters enormously.

In an environment where a minimum viable AI product can be built in a matter of weeks, the temptation to lead with a solution is enormous. It is also one of the most reliable paths to wasted investment and organizational fatigue. The discipline required is to define the operational, clinical, or financial challenge first, then ask what AI can do about it.

Sara shared a useful framework for doing this: AI is best applied to tasks that are dull, difficult, dirty, or dangerous – the 4 D’s. And within those tasks, it can create value in five distinct ways: by assisting, augmenting, automating, amplifying, or accelerating – the 5 A’s. Together, these give leaders a practical lens for identifying where AI can genuinely move the needle.

2. Build for failure, not just success

“It’s predictably wrong in unpredictable ways. The only way to be comfortable with it is for everyone to become experts at the tools, to know where to push the limits and where it might falter,” says Dr. Knight.

The organizations that scale AI safely are the ones that plan not just for what will go right, but for what will go wrong.

Unlike traditional software, generative AI doesn’t produce the same output from the same input every time. That makes it powerful. It also makes it dangerous if left unmonitored. The answer isn’t to avoid it. It’s to build guardrails, verification processes, and kill switches in from the start, and to treat 24/7 runtime reliability as a baseline requirement, not an afterthought.

Sara cited a real experience that makes the point clearly. A tightly scoped AI tool for patient navigation was designed exclusively to handle medication refill inquiries. It provided a technically clinical response in only two instances out of millions of conversations. It was rare, unpredicted, and quickly caught by an active monitoring framework. But it happened. And in healthcare, unpredicted behavior carries real consequences.

3. Scaling AI is an organizational challenge, not a technical one

“Without the CEO being not just bought in, but driving it. I would go so far as to say it’s doomed to fail,” explains Sara.

The biggest barrier to scaling AI in healthcare isn’t the technology, but the organization.

Every minute of every employee’s day in a healthcare system is already accounted for. Asking people to engage with new, imperfect, non-deterministic tools on top of everything else requires what Dr. Knight called “heroic effort” – a discretionary commitment that can’t be mandated, only inspired. The organizations that find it are the ones where leadership has made clear the work matters, and where early wins have built enough credibility to sustain momentum.

Ambient documentation tools are the clearest current example. They primarily deliver wellness and retention benefits rather than in-year financial returns, which makes the ROI case harder in the short term, but no less real. Physician burnout drives attrition. Attrition drives capacity constraints. Capacity constraints drive access failures. The long tail of value is significant. But realizing it requires an organizational commitment that goes well beyond deploying a tool.

4. A coherent market story is a scaling tool

This is where many healthcare organizations stall. The transformation is real. The results are emerging. But without a clear, coherent narrative that connects what the organization is doing to why it matters, stakeholders – clients, partners, investors, employees – are left to draw their own conclusions. In a market full of noise and competing claims, that silence is rarely interpreted favorably.

The organizations that scale AI most effectively treat their market story as part of the transformation itself. Not something that comes later, once the results are in. A story that builds internal alignment and external belief simultaneously, giving leaders, clients, and stakeholders a shared understanding of where the organization is going and why it will get there. When that clarity exists, momentum builds faster, trust compounds, and the path from pilot to performance becomes significantly shorter.

The organizations that will define the next era of healthcare aren’t just the most innovative. They’re the most coherent.

The potential of AI in healthcare is genuine and significant. But potential only compounds when it is paired with the discipline, the governance, the organizational coherence, and the market clarity to realize it.

That work starts not in the lab, but in the boardroom.

About the author: Jason Hutt, Head of Strategy and Healthcare Lead

Jason has over 20 years experience combining insight and creativity to craft compelling brand strategies that drive growth and inspire audiences across healthcare.

Connect with Jason