What goes into
an agent release review.

A commit is only part of the change.

An agent can load its prompt remotely, retrieve a refreshed dataset, or switch to a fallback model. Reviewing its code alone can miss the conditions that determine its behavior.

TrueRook’s release record is designed to bring those conditions together so a reviewer can see exactly what is being assessed.

Agent and environment

The application, its owner, and the environment being changed.

Candidate version

The code and known configuration under review, with versions for prompts, models, tools, and data where available.

Test evidence

The cases run, baseline and candidate results, failed business rules, and evidence that is missing or out of date.

Review decision

Who approved or rejected the candidate, the policy used, and any exception with a reason and expiry.

Deployment result

What the deployment system reports as serving, including partial rollouts or an unknown version.

A candidate that changes after approval needs a new assessment. Historical results should remain attached to the version that produced them.

Show why an agent is affected.

A sales policy appears in a quote agent’s configuration. A trace also shows that the agent read the policy when preparing a quote. Together, those observations give the reviewer a reason to test that agent when the policy changes.

Configuration and runtime observations answer different questions: what the agent can use, and what it used in a particular run. Neither proves that every dependency has been found.

Make failed cases easy to inspect.

A reviewer needs more than a score. For each required check, show the input, expected business outcome, observed behavior, and the version that was tested.

In the sales-quote example, the approval-free discount limit falls from 10% to 5%. A test asks for 10% off. The agent should ask a manager, but instead offers the discount without approval. That failed rule should hold the candidate for review even if other checks pass.

• Use the same cases to compare the candidate with its baseline.
• Separate a behavior failure from a test job that did not run successfully.
• Show whether the cases cover the changed rule and known consumers.
• Record reviewer decisions and any explicitly accepted gaps.

Existing evaluation and security tools can supply the results. Test cases still need domain knowledge, and replay must run in an environment that cannot trigger real business actions.

Check the version at deployment.

A human approval is a decision about a specific candidate. It does not prove that the same candidate was deployed, or that the rollout completed.

A protected step in the customer’s deployment pipeline can check authorization before it runs. After rollout, the serving version needs to be compared with the approved version.

Builds, traffic routing, rollback, and incident response remain with the customer’s systems and operators.

Connect evidence before adding control.

GitHub, CI, Python or TypeScript agent configuration, and existing evaluation and telemetry systems are initial integration targets. The exact APIs, permissions, and version coverage need checking for each workflow.

Reading metadata, requesting an action, and blocking a deployment are separate permissions. Content access, retention, and exports should be agreed before connection. Inventory does not require exporting credential values.

Use production findings to improve the next test.

A production finding is easier to investigate when it can be linked to the release and the evidence used to approve it. Where identity or sampling is incomplete, the link should be presented as uncertain.

A reviewed incident can become a regression case for a future evaluation. It should not rewrite the history of a past approval. Pausing new tasks or rolling back a version also cannot undo business actions already completed.

Begin with one review that was difficult.

A practical starting point is an agent your team already operates and a recent change that took substantial effort to assess. Identify who owns the release, where the evidence lives, and which business rule must hold.

Then decide whether the useful first scope is collecting evidence, advising reviewers, or adding a gate to one deployment path. Measure the time spent preparing the review, important gaps found, incorrect blocks, and whether the final decision improved.

Agree the success criteria before connecting systems so both teams can assess whether the review became faster or more reliable.