Evaluating AI Agents: Check Outcomes, Actions and Recovery

Reviewed and updated October 8, 2026.

Evaluate an agent by checking both the result and the observable actions that produced it. A final message saying “completed” is not evidence that the intended state exists. Equally, a different sequence of valid steps should not fail merely because it differs from a reference transcript.

Anthropic's agent evaluation guide distinguishes transcripts from outcomes and combines code, model and human graders. This distinction is useful when an agent reads records, changes state and recovers from tool failures.

Define the outcome and prohibited actions

Write a task case with initial state, user authority, allowed tools, expected final state and conditions that must remain true. Test in an isolated environment with controlled fixtures. Verify the resulting state through an independent read rather than relying on the agent's own explanation.

Collect action names, arguments, authorization decisions, tool results, retries and state transitions. Restrict access to these records and redact secrets. The trace can establish what the application observed; it cannot prove the model's private reasoning or reveal every unobserved failure.

An illustrative ticket reassignment test

Suppose a user asks an agent to assign an open support ticket to an authorized team. This is a worked test design, not a reported run. The initial fixture has an unassigned ticket, a tenant boundary and a team membership rule.

• Outcome: the correct ticket has the requested team and an audit event identifying the action.
• Permission: the user may change that ticket and assign that team. A ticket in another tenant remains unchanged.
• State: a closed ticket cannot be reopened implicitly just to make reassignment succeed.
• Recovery: a simulated timeout after the write triggers a status check, not an unbounded repeat of the mutation.

Score those four checks separately. An illustrative result with the correct assignment but an unauthorized cross-tenant read fails the permission requirement. Three passing checks do not make that release acceptable. A sequence that reads membership before the ticket can pass just as a sequence that reads the ticket first, if both respect access and produce the same valid state.

Use the right grader for each claim

Code checks suit exact record changes, schema constraints and prohibited tool calls. A domain rubric can grade explanation quality. Human review is useful for ambiguous cases and grader disagreements. Define required evidence for each judgment so a persuasive final answer does not overpower a failed state check.

Temporal's documentation explains the external-action ambiguity when a worker completes an operation but crashes before recording completion. Include this case in action evaluations even when orchestration is durable.

Compare systems under declared conditions

Repeat trials where variation matters, reset fixtures and record models, prompts, tool versions and resource limits. Report tasks and failure categories rather than one unexplained aggregate. Keep cases that expose valid alternative paths, and repair a grader that rejects them. Preserve a holdout when using the development suite to tune behavior.

For a workflow with real state changes, see AI automation. Request a project review with the desired final state, permitted actions and recovery cases that acceptance must cover.