UK startups: why ChatGPT demos haven’t become production AI

# The short story UK startups kept showing ChatGPT demos—live summaries of contracts, help-center chatbots, or Copilot-style features—and mistook the demonstration for a finished product. The models improved, but product practices did not. As a result, many proofs of concept stalled and never reached daily use.

# What founders and boards were doing

# The numbers and who said it IDC research (reported via CIO.com) found that 88% of observed AI proofs of concept did not reach widescale production. Gartner forecast that at least 30% of generative AI projects would be abandoned after PoC by end of 2025 because of poor data, missing risk controls, rising costs, or unclear value.

# Why the demos fail in production Demos use public models and saved prompts. They lack memory of permissions, provenance for answers, failure detection, and ownership when the system is confidently wrong in front of customers. Common, concrete problems:

• Data hygiene: teams used SharePoint dumps and ad-hoc corpora instead of curated, versioned datasets.
• No evaluation set: no repeatable metric to show progress or regression.

# Retrieval, RAG and agents — what failed and what survived The simplistic 2023 approach—"dump PDFs into a vector database and call it RAG"—did not work. Retrieval as a concept did not die, but naive implementations failed. Agents introduced a new set of problems: cost, maintaining state, secure access to systems, and safety. Real incidents in the field included an agent that wiped production data and research showing agents could turn public wikis into private message boards. Those are technical and operational failures rather than pure model intelligence issues.

• Define a one-sentence success test before you write a prompt. If you cannot, don't build yet.
• Treat PoCs as decision points: either go live with a clear plan, iterate against measurable criteria, or kill the project.

• Avoid shared Copilot-style logins and ad-hoc prompts as long-term strategies.

# What this means for UK startups now

# Quick checklist for founders

• Can you write a one-sentence success test? If no, pause.
• Is there an evaluation dataset and metrics? If no, build one.
• Have you designed audit trails and permission checks? If no, it's high risk in regulated fields.
• Do you have a budgeted plan to operate cost, state and access for agents? If no, agents are premature.