Why AI pilots
stall

Why are successful AI pilots rarely scaled up to production?

AI pilots stall for causes that are never technical: the pilot only answers whether something is technically possible, it runs outside the real work process, it has no owner for the phase after, it leaves out the cost of upkeep, and it avoids the organisational change needed. A pilot demonstrates that something is feasible, not that it's worth doing. What works well in practice is recording, before the pilot starts, the question it needs to answer, the stop criteria, the scale-up owner, the upkeep estimate and the change required, all on one page.

Can AI pilots run better in the real working environment than in a separate one?

Yes, for AI pilots the countermeasure is: a smaller scope in the real environment beats a larger scope alongside it, with one team, real data, the real process and real integrations. That setup is slower in week one and considerably faster by month six.

Does the scale-up owner for AI pilots need to be known before the pilot starts?

Yes. The scale-up owner for AI pilots is appointed before the pilot begins: someone from the business, with budget and a seat on the pilot steering group. If that person can't be found, that's itself an important finding.

Can AI pilots go ahead if nothing in the organisation is allowed to change?

Yes, but the business case for AI pilots is then calculated without the savings that change would require. Often a real, smaller value remains, and that's a more honest starting point than a promise that can't be delivered later.

Source: article by Dave Willemstein. Updated: October 2026.

The pilot paradox

Pilots are designed to reduce risk. Start small, learn, then scale up. In practice, they often do the opposite: they create the illusion of progress without anything actually changing. An organisation with seven successful pilots and zero applications in production is further from its goal than an organisation with one application that's actually running.

The causes are strikingly consistent. Five of them, and none are technical.

Cause 1: The pilot proved the wrong question

Most pilots answer the question "can this work technically?" These days, that answer is almost always yes, which means the pilot is neither informative nor decision-useful.

The questions that actually matter: does this cost less than it delivers at a hundred times this volume? Do the people who have to work with it actually want to use it when nobody's watching? Can we maintain it with the capacity we have?

Countermeasure: before you start, write down the one question the pilot needs to answer, and phrase it so that "no" is a possible answer. If you can't picture what a negative outcome looks like beforehand, you're not designing a pilot, you're putting on a demonstration.

Cause 2: The pilot ran outside the real work

To move fast, a pilot often gets set up alongside the existing process: a separate environment, a hand-picked group of enthusiastic users, a standalone dataset. That gets you a result quickly and tells you nothing about scalability.

What then goes wrong at scale-up: integration with the source system turns out to be three months of work, the average user doesn't have the context the pilot group had, and the real data is messier than the test dataset.

Countermeasure: a smaller scope in the real environment beats a larger scope alongside it. One team, real data, the real process, real integrations. That's slower in week one and considerably faster by month six.

Cause 3: Nobody owns the next phase

The pilot has a project lead. Their job is done at delivery. The business hasn't planned any capacity, reserved any budget, or set any target that this application fits into.

What happens then: the application keeps running in pilot form, unmaintained, until it breaks or the licence expires. Nobody decides anything. It just evaporates.

Countermeasure: name the scale-up owner before the pilot starts, not after. That person sits in the business, has budget, and sits on the pilot steering group. If you can't find that person, that's itself an important finding: apparently there's no place in the organisation where this value lands.

Cause 4: Upkeep wasn't factored in

AI applications need ongoing maintenance: quality monitoring, retraining or recalibration, tracking model versions, a test set that stays current. During the pilot phase, the project team does this implicitly. After delivery, nobody does.

This is why applications that worked fine in month one are quietly performing worse by month eight, without anyone noticing.

Countermeasure: include the upkeep cost in the business case, expressed in hours per month and assigned to a team. And during the pilot, run at least one real maintenance action (a retraining, a model update), so you know what it actually costs.

Cause 5: The organisation wasn't allowed to change

This is the deepest cause, and the one talked about least. An AI application that saves time means people do their work differently. Sometimes a role shifts. Sometimes a check moves from one department to another.

Pilots are often deliberately designed so these questions don't come up, because that makes things move faster. At scale-up, they all surface at once, and it turns out the business case only adds up if staffing or the division of work actually changes. That conversation was never prepared for.

Countermeasure: name at the start which organisational change is needed to realise the value, and check whether there's support for it. If the answer is that nothing is allowed to change, you can still run the pilot, but calculate the business case without the saving that change would require. Often a real, smaller value remains. That's a more honest starting point than a promise you can't keep later.

A one-page pilot contract

What works well in practice: before you start, write down five things on one page, signed by the sponsor and the scale-up owner.

• The question this pilot answers, with a measurable threshold for "successful".
• The stop criteria: at what outcome do we stop, and who decides that.
• The scale-up owner: name, and the department where the value lands.
• The upkeep estimate: how many hours a month does this cost after delivery, and who supplies them.
• The change required: what has to be different in the work for the value to be realised.

This page takes an hour to write and prevents the scenario where, a year later, you have seven successful pilots and nothing in production. The biggest effect isn't in the document itself, but in the conversations you have to have to fill it in. That's exactly where the blockers become visible, while there's still time to fix them.