11People, roles and skills

Roles first, expert time as a management commitment, one team across both sides, and the skills to build.

The second principle stated in chapter 01 has a practical consequence: the business is not consulted about this system. It runs part of it.

Roles come first

A product owner from the business sets priorities, triages feedback and accepts each increment. Each source has an owner who decides what it means, what is validated and who may see it. Experts define the vocabulary, correct drafts, fill gaps and judge responses. Technology builds and runs the instruments that make their decisions hold (Figure 11). The CTO and the CIO share that work as the first article described, but neither decides in the business's place.

The business decides Technology makes it hold what a term means records it in the contract what counts as validated checks it and records the status how critical a question is applies the stricter path, or points who may see what carries rights all the way to the answer which data may leave the company routes each class of data accordingly when a stage is good enough measures it on the test set what feedback means fixes the system, versions the release
Figure 11. Who decides what

Expert time is a management commitment

Validating knowledge, correcting interview drafts and growing the test set fall on a small number of people who already have full-time jobs. If the work is simply added on top, it slips, and the project stalls without anyone deciding to stop it. Expert time must be budgeted and released by the experts' management, like any other resource, and it is often the scarcest one. Where experts are about to retire, it is also running out. Their engagement is earned the same way as sponsors': by seeing, cycle after cycle, that their corrections make the system better.

One team across both sides

Separating an "AI team" from the people who own the data and the source systems recreates the drift between the two sides that chapter 02 warned against. The walking skeleton needs one team, product owner included, building both sides together.

Own, buy, or bootstrap

The make-or-buy logic applies to people too. What the company will keep for years is owned: the description of its knowledge, the test set, access rights, product ownership. What is commodity or temporary can be bought: specialised document processing, the initial hosting setup, a security review. What is needed early but must be owned later is bootstrapped with outside help and a planned transfer, written into the roadmap rather than assumed. One skill is easily forgotten: interviewing experts well, on which Crawl depends. Hiring often takes longer than several cycles, so a missing skill can block a stage as surely as missing infrastructure.

Engineers need new skills, on top of the old ones

Most tools for preparing knowledge and running evaluations come from the Python world, which many enterprise teams have never used (Figure 12). More fundamentally, these systems are not deterministic: the same question can produce different responses, so "correct" becomes a rate measured on a test set rather than a test that passes or fails. Meanwhile, the established disciplines matter more than ever, because they are what separates a production system from a demo. Learning takes time, and that time belongs in the roadmap. A team asked to build a production system while learning its tools will deliver a demo.

Still essential Software engineering fundamentals clean code, reviews, automation Integration with company systems sources and access rights live there Matters more than before Architecture and interfaces the contract between the two sides, replaceable parts Data and domain modelling describing knowledge with the people who own it Testing discipline regression on every change, now measured as rates Security rights carried from the source to the answer Version and release management a release is model, knowledge, processing and rules Operations and monitoring cost, drift, degraded modes, rebuilding the index Working with business experts they decide what is true and what is acceptable New The Python toolchain most tools for preparing knowledge and evaluating Thinking in probabilities "correct" is a rate, not a pass or a fail Evaluation design test sets, measures, small samples, checked graders Document and data processing scans, tables, cutting into passages, profiling Retrieval techniques vectors, keyword search, ranking Model instructions as code written, versioned, reviewed and tested New threats and costs prompt injection, aggregation, cost per question
Figure 12. Skills for the technical team