07Measuring what matters
The expert test set as the instrument, safety failures counted one by one, and a full run on every change.
Without measurement, every claim about the system is an opinion: it seems better, the demo went well, users like it. With measurement, every change has a number, and every stage decision rests on evidence. Defining and measuring relevance is not a refinement to add later. It is the instrument that makes everything else in this article manageable (Figure 7).
The test set is the instrument
It is a collection of real questions, each with the response experts expect: the right value and its source, the right document to point to, or the right refusal. Refusals matter as much as answers; without them, the test set cannot detect the most dangerous failure. The first version is built at Crawl, from expert interviews and real questions, together with a baseline: how long a technician takes today to find the right torque, and how often they call an expert. It grows every cycle with new questions, expert corrections and field cases. Like the knowledge base, it is a company asset that will outlive any model or tool.
Measure the two sides separately
For retrieval: were the right passages found, from the right version, among those the user may see? For answers: given what was retrieved, was the response correct and sourced, and was the choice between answering, pointing and refusing the right one? A single overall score hides where the problem lies. When retrieval is poor, a better model will not help. When retrieval is good and answers are wrong, reindexing will not help either.
Count safety failures; never average them
"95% correct" sounds reassuring and may include an unsourced torque value. Safety failures are counted one by one, and the target is zero. One such failure blocks a release, whatever the other scores say.
Run the test set on every change
A new model version, a reprocessed batch of documents, a different way of cutting them, a reworded instruction: each can improve some responses and silently break others. The full test set runs on every change, and a change that lowers quality or adds a safety failure does not ship. This is the first article’s Definition of Done, enforced by the pipeline rather than by good intentions.
Calibrate the graders
Experts cannot judge thousands of responses on every change, so grading is partly automated, often by another model. That grader must itself be checked regularly against expert judgement, or it produces confident numbers that mean little.
Then add the field
Once real users arrive, the test set is joined by usage signals: questions asked, responses flagged, pointers followed, refusals that turned out to be wrong. Chapter 09 covers how these flow back into the system.