AI Feedback: Turn Production Failures into Reviewed Tests

Reviewed and updated October 8, 2026.

Production feedback is useful when it becomes a reviewed test case with a clear expected behavior. Automatically adding every transcript to training data or an evaluation suite can preserve mistakes, expose sensitive material and make results difficult to compare.

The decision is which observations deserve a controlled change to the system, the test set or neither. Treat model, prompt, retrieval and tool changes as releases. A feedback process should explain why a change was accepted and how its effect was checked.

Move from observation to an approved case

Capture the failure category and system version with the minimum information needed to investigate. Confirm permitted use, retention and access before reusing production content. Redact confidential and personal material, then have a domain owner decide whether the event was a model defect, a source problem, an unclear requirement or a valid refusal.

The NIST generative AI profile provides voluntary risk-management actions across the lifecycle, including evaluation and monitoring. The review workflow below is an engineering proposal for applying those ideas; it is not a NIST certification process.

An illustrative complaint-to-test cycle

Suppose a user reports that an assistant cited the old reimbursement policy. This is a proposed feedback example, not a measured improvement. First inspect the source version used. If the current policy never reached the index, fix ingestion rather than rewriting the prompt to guess the latest policy.

1. Create a redacted case containing the question, applicable policy versions and expected citation.
2. Have the policy owner approve the expected answer and the conditions for abstention.
3. Add variants with an expired policy, a missing current source and a user without access.
4. Reproduce the failure on the recorded old configuration, then compare the proposed change.
5. Check existing cases for regressions before a limited rollout.

Keep the reviewed case in a development suite. Maintain a separate holdout that is not repeatedly used to tune prompts or training. Deduplicate related cases and label their origin so a flood of similar complaints does not appear to be broad coverage.

Review the evaluator as well as the system

A user rating is a signal, not a complete label. A satisfied user may miss an incorrect citation; a dissatisfied user may be reacting to a correct denial. Model-based graders also need calibration. The MT-Bench and Chatbot Arena judge study documents biases including position and verbosity in the evaluated judges. It does not establish that a particular enterprise grader is reliable.

Sample both successful and failed interactions for review. Compare grader decisions with domain labels, inspect disagreements and version rubric changes. Separate an intentional expansion of scope from a repair: the former may need new permissions and acceptance criteria.

Release with a recorded reason

Keep the approved change, evaluation evidence, owner and rollback configuration together. Monitor the affected failure category after release. If it recurs, investigate rather than automatically training on the new complaint. Continuous improvement still depends on source owners, reviewers and an operating process.

For defining that process, see AI roadmap and enablement. Request a project review with representative failures, the current feedback route and the person who can approve correct behavior.