Why AI pilots fail

Stalled pilots are usually explained by ownership, unclear success measures, unready data and workflows that never change, though no study ranks these causes. Here is what the studies behind the 95% figure actually measured.

By The FDE Blog editorial team·Published 23 October 2026·Updated 23 October 2026

The evidence we could verify points more to organizational causes than to a model that cannot do the task, though respondents in the NANDA survey did rate model output quality concerns the second-highest barrier and no study ranks the causes. Four explanations recur across the sources: nobody owns the result once the demo ends, success is never defined in business terms, the data and systems are not ready, and the way people work never changes. The first has the thinnest direct evidence, as we explain below^1,5,6. The headline claim that 95% of AI pilots fail is real but narrower than it sounds. It comes from one preliminary MIT NANDA report that rests on 52 interviews, a 153-person survey and a review of public initiatives¹.

We read the MIT NANDA report itself rather than the coverage of it, then checked it against other sources: an IBM survey of CEOs, Boston Consulting Group's AI value research, a peer-reviewed survey of published machine learning deployments, an interview study of data practitioners and a UK National Audit Office report, plus PostHog's public handbook for one practitioner example. We loaded every source on October 23, 2026. The NANDA and IBM figures date from 2025. BCG has since published a 2026 index, which we include, and IBM's CEO study page now carries a 2026 edition whose figures we could not verify⁷.

What does the 95% figure actually say?

The source is The GenAI Divide: State of AI in Business 2025, published in July 2025 by MIT's Project NANDA. Its methodology combines a review of more than 300 publicly disclosed AI initiatives, interviews with 52 organizations and a survey of 153 senior leaders, and the report labels its findings preliminary¹.

The 95% is the flip side of the 5% in one chart. For embedded or task-specific GenAI tools, 60% of organizations had investigated them, 20% had piloted them and 5% had reached "successfully implemented," which the report defines as tools that users or executives said had a marked and sustained effect on productivity or profit and loss¹. Three details change how to read that.

• The report states the figure in three ways. It says 95% of organizations are getting zero return, says just 5% of integrated AI pilots are extracting millions in value, and later refers to a 95% failure rate for enterprise AI solutions. Those mix two units (organizations and pilots) and three thresholds (zero return, millions in value, a marked and sustained productivity or profit and loss effect)¹. If the chart's bars share a base, 5 of the 20 that piloted is one in four, so the chart would imply a failure rate among pilots nearer 75% than 95%. That is our arithmetic, not the report's.
• General-purpose tools look very different, but were not held to the same test. In the same chart, 80% had investigated tools such as ChatGPT or Copilot, 50% had piloted them and 40% had implemented them. The report states its success definition only for task-specific tools and says the general-purpose ones mainly raise individual productivity rather than profit and loss, so 40% against 5% is not a like-for-like comparison¹. The report locates the problem in custom and vendor-sold enterprise tools.
• The authors hedge, and so should you. They say the figures are directional, rest on interviews rather than company reporting and may define success differently across organizations. The summary also counts eight sectors where the body counts nine, which is consistent with its preliminary label¹.

None of this makes the report wrong. It is a rough picture of one slice of the market, custom enterprise GenAI tools as described by executives and users at the organizations interviewed.

How do the other failure numbers compare?

Studies count different things, so their numbers cannot be averaged into one failure rate. The table shows what each one measured, in the source's own terms.

The three 2025 studies that report rates agree on direction: only a minority of organizations report AI at scale or at the return they expected. BCG's 2026 index, with a different sample and its own definitions, describes almost half of companies as generating value, a reminder that these numbers move with the definition and the year. They do not agree on a number, and we would not force them to.

Why does nobody own the pilot after the demo?

The closest thing the studies measure is sponsorship and change management. NANDA surveyed executive sponsors and frontline users across 52 organizations and asked them to rate common barriers to scaling on a 1 to 10 frequency scale. Lack of executive sponsorship and challenging change management each scored about 6.5, read from the report's chart. That made them the two lowest of the five barriers charted: unwillingness to adopt new tools led at about 9, with model output quality concerns and poor user experience at roughly 7.5 and 7¹. IBM's CEO survey points a similar way on collaboration: CEOs named lack of collaboration across organizational silos among the top barriers to innovation, a broader question than AI pilots². The UK National Audit Office warned of risks to value for money if government does not settle which department has overall ownership and accountability for its AI adoption strategy, which is a point about a national strategy, not a single pilot⁸.

What the organizations that did succeed did differently is more telling. NANDA says its better buyers let budget holders and domain managers surface problems, vet tools and lead rollouts, with executive accountability on top, and partnered with vendors through early failures, treating deployment as co-evolution¹. The survey of published deployments makes a similar point about handoffs. It says researchers and software engineers often work on the same code and share concerns about model inputs, outputs and metrics, so including researchers in the whole development journey, with a shared codebase and code review, was seen to bring long-term benefits in delivery speed and quality despite a slow start⁵. No source we found counts "no owner" as a cause directly, so this one rests on adjacent evidence.

Our read: if the pilot's sponsor is a budget line rather than a named person who answers for it in production, we would worry that the demo is the last milestone.

Why does success never get defined?

Two sources describe measuring what is easy instead of what the business needs. NANDA asked executives to allocate a hypothetical $100 across functions, and sales and marketing took the largest share, which the authors attribute partly to outcomes being easy to measure, such as demo volume or email response time, while finance, procurement and legal gains are real but harder to put in front of an executive. The same report says success definitions may differ between the organizations it studied¹.

The deployment survey has the sharpest example. It says an increase in model performance often does not translate into business value, as Booking.com discovered after deploying 150 models into production, and it names a failure of proxy metrics such as clicks to convert into the business metric, conversion, as one reason Booking.com highlights. Defining better metrics, it says, takes understanding from modeling, engineering and business angles⁵.

A practitioner version of the fix is in PostHog's public handbook. It says every engagement its forward deployed engineers take on needs a measurable outcome, a rough timeline and a shared definition of done, written up in a short brief that the customer confirms before work is committed. A deliverable that doubles in scope becomes a new quote, not an informal extension⁹. Those are engagements for PostHog's own customers, not specifically AI pilots, and the handbook describes its own process, so this is a practice to borrow, not evidence that it works.

Is the data really ready?

The evidence is indirect, but it points to data as a common weak spot. Sambasivan and colleagues interviewed 53 AI practitioners in India, East and West Africa and the USA and define "data cascades" as compounding events with negative downstream effects from data issues. Of those interviewed, 92% reported at least one cascade⁶. The study focuses on high-stakes areas such as health and conservation and measures data problems, not pilot outcomes.

Closer to enterprise work, the deployment survey notes that finding what data exists and where it lives can be hard in a large organization, and that joining sources with different schemas and conventions is integration work in its own right⁵. In IBM's 2025 survey, 68% of CEOs called an integrated, enterprise-wide data architecture critical for cross-functional collaboration, and half acknowledged that the pace of recent investment had left their organization with disconnected, piecemeal technology². The first shows what CEOs value, the second what they say they have. The UK National Audit Office named legacy systems and data access and sharing as fundamental barriers to AI adoption in government⁸.

One tension to flag. NANDA's barrier chart has no data option, and its authors say the core barrier is learning, not infrastructure, regulation or talent¹. Silence in a chart with no data option is not evidence that data is fine.

Why does the workflow never change?

In our view, a pilot that runs beside the real process tests the model, not the change. NANDA's evidence here is about tools that do not fit the work more than about workflows that never change. It asked what prevents broader adoption for mission-critical work, and the four reasons it charts are that the tool breaks in edge cases, cannot be customized to the team's workflow, needs too much manual context each time and does not learn from feedback. The authors call this the learning gap: most systems do not retain feedback, adapt to context or improve over time¹.

The deployment survey points the same way in older machine learning, on the interface side. It reports that most doctors could not use an AI diagnosis tool productively in rural Chinese clinics, with one main reason being a design that did not account for screen sizes or the other software in the clinic. It also reports that the developers of Atlanta's Firebird fire-inspection system found the best way to avoid resistance from users, who were replacing a pen-and-paper method, was an interface that presented results the way fire officers and inspectors found most useful and clear⁵. The UK National Audit Office is the source that speaks to process change directly: it says achieving large-scale benefits is likely to need significant changes in business processes and corresponding workforce changes, not only new technology⁸.

How can you tell early that a pilot is stalling?

This table is our synthesis of the findings above, not a validated checklist. Each question can be asked in the first week.

What does this mean for forward deployed work?

Several of these failures sit in the gap between a model that works in a demo and a customer's production system. In our view that gap is where forward deployed work earns its keep, though whether a particular role actually owns it varies by employer, as the postings we compared in is forward deployed engineer a good role show. If you are weighing the work, do forward deployed engineers travel covers the onsite side.

What do we still not know?

Quite a lot. No study we found tests why pilots fail against a comparison group. Every figure here comes from interviews and surveys of executives, users and practitioners, or, for BCG's 2025 numbers, a method its page does not describe, and the studies define success differently, so the relative weight of the four failure modes is unknown. Comparisons of building against buying are confounded, as NANDA itself warns¹. The deployment survey predates generative AI, so it shows which failure points are old, not how common they are now⁵. We could not read the figures in IBM's 2026 CEO study, so the IBM numbers describe 2025. Read this article as a map of where to look, not a forecast.

How we edited this. The FDE Blog editorial team wrote and edited this note. Each factual claim carries a numbered footnote to the source listed above. How we source