02A different kind of project

A mirror held up to the organisation

Within weeks, our rail maintainer's team runs into questions nobody planned for. The same procedure exists in three versions, and the engineer who could have said which is right retired last year. A service bulletin contradicts the manual it was meant to correct. Some supplier documents are confidential and not every technician may see them. Whole families of older engines are barely documented at all.

The AI project created none of these problems. Experienced people simply knew how to work around them. A system that answers questions makes them visible, because it needs to know which answer is true and who may receive it. What was approved as a technology project turns out to be a diagnosis of how the organisation manages its knowledge, and the findings call for decisions no technical team can make alone.

No specification, and no end

In a classic project, each requirement passes or fails, and when everything passes, the project is delivered. Here, nobody can list every question a technician will ask, and the system will be right most of the time, never all of the time. Quality is not a box that gets ticked. It is a rate, which must be measured and which can drift.

Nor does the project end at delivery. Engines are modified, procedures revised, experts leave. Unless the system follows, its answers grow stale, and a stale answer delivered with confidence is worse than none. Every question has a running cost, and so does measuring quality. Running the system is not the tail of the project; it is a large part of it, and the budget must say so.

Little experience to lean on

When a company approves a new ERP or a payment engine, it relies on others' experience without thinking about it: reference costs and durations, known risks, people who have done it before. GenAI projects don't have that yet. Few organisations have taken one to stable, everyday use, and fewer still have shared what it really cost.

That leaves two tempting responses, both wrong. The first is to wait until others have worked it out; for our rail maintainer, the experts are leaving now. The second is to "just do it and see": start a pilot, show a demo, and hope value appears. That is how organisations end up with pilots that never finish, never scale and are never stopped, because nobody defined success.

Learning in short cycles

If the experience doesn't exist, each organisation must build its own, deliberately rather than by accident. That means learning in short cycles, as Figure 3 shows: build a little, on a narrow scope; put it in front of real users and experts; measure how often it is right; then decide to continue, adjust, or stop.

We lack experience with these projects. We do not lack disciplines for managing what we don't yet know. Agile frameworks such as Scrum were designed for work whose outcome must be discovered step by step, and the chapters that follow draw on other established disciplines too: governance, funding in stages, explicit acceptance of risk, testing every change against what already worked. None was invented for AI. What is new is applying them here, under explicit control, so that short cycles lead somewhere rather than drift.

AI can help with much of this work, including finding contradictions and converting old documents. It cannot decide which version is true, who owns it, or what level of error is acceptable. Fix what only you own, and let AI help with the rest.