Stopping a stalled pilot
Illustrative example built on simulated data. Not a client engagement.
The situation
A simulated utility launched an agent-assist pilot in customer service: AI suggests answers, agents send them. Six months in, the pilot is neither scaled nor stopped. The team keeps asking for one more quarter.
The evidence
The original business case assumed a 30% reduction in handling time and a knowledge base that was current. The re-score compared both assumptions with what the pilot actually recorded, and added one question the business case had skipped: which answers touch contract terms and pricing.
What the assessment showed
Measured handling time fell by 6%, not 30%. Two in five knowledge articles were out of date, so agents rewrote most suggestions. One in four enquiries touched contract terms, which raises the oversight requirement. The suitability score fell from 66 at launch to 41. The cause of the stall was not the technology. It was the knowledge base, and no one owned it.
The decision
- Defer agent-assist in customer service, pilot stopped
The pilot team of three moves to the next candidate in the portfolio.
The record states what was assumed, what was measured and why stopping is the defensible decision.
What would change it
The pilot is reassessed when the knowledge base has a named owner, at least 85% of articles are current, and a four-week controlled test shows at least a 15% reduction in handling time.