09Living with it: feedback and upgrades
Feedback routed to the side that can fix it, upgrades nobody chose, and releases versioned as a whole.
Going live is not the end of a project. It is the beginning of a product's life, and for a system serving knowledge that lasts decades, that life is long.
Feedback is designed in
Users need a simple way to say that a response was wrong, incomplete or unhelpful, at the moment they notice it. Those signals go to the product owner, a business role, who decides what they mean and what to do about them, like any other item in the backlog.
Route feedback to the right side
Most reports fall into one of two kinds (Figure 9). Either the knowledge is wrong or missing: the manual is outdated, the figure was never recorded. That goes to the owner of the source, because no change to the software will fix it. Or the knowledge existed and the system failed to use it: the right document was not found, or the response ignored it. That goes to the team. Telling the two apart is half the value of feedback. Mixing them sends engineers to fix documents and experts to fix software.
Turn every confirmed case into a test
Once an expert has confirmed what the right response should have been, the case joins the test set, so the same failure cannot return unnoticed.
Don't rely on feedback alone
Users report only a fraction of what goes wrong, and rarely the confident wrong answers they did not notice. Experts therefore review a regular sample of real responses. It is the only way to find the failures nobody flagged.
Expect upgrades you did not choose
Model providers retire versions on their own schedule. Documents change every week. Components receive security updates. Over the life of the system, each of these will happen many times, and each is a change that goes through the test set like any other.
Version the whole release
What users receive depends on four things together: the model, the state of the knowledge base, the way documents are processed, and the rules that govern responses. A release is that combination, versioned as one. Without it, nobody can say after an incident which system answered, or roll back to the last version that worked.
None of this is new. It is ordinary product management, with a backlog, releases and a business owner, and the Scrum articles on this site describe how it runs.