A policy is how a team tells TestMachine what its agents may do. It covers which actions an agent can take and how good its output must be. TestMachine's judge checks every routed tool call against the policy before the call runs, so the rules hold whether or not the agent remembers them.
With a policy in force, agents work inside agreed limits without a person watching each action. A blocked agent gets the rule and what to do instead, and corrects itself. Every decision traces back to the rule behind it. As the record grows, it shows the team where agents can safely take on more work.
This post shows that with one of our own rule sets: the messaging guide our agents follow when they draft outreach emails. We compare two ways to apply the guide, a skill and a policy, and show how the policy improves as the agent works.
The guide as a skill
Our team gives agents a shared skill for TestMachine copy. When an agent writes or reviews copy, the skill tells it to read the current guide in full and apply it.
A skill tells an agent how to do something. The agent decides whether to load the skill and whether to follow it. If a draft breaks the guide, nothing outside the agent checks it. A skill carries no accountability beyond the agent that used it.
The guide as a policy
A policy decides whether an agent's call goes through. Enforcing it takes three parts: the policy your team signs off, the tools your agents call, and TestMachine's judge between them.
The policy. The team does not start from a blank page. It pastes in rules it already has, such as our messaging guide, and TestMachine sorts each rule. Some become clauses, some are already covered by a policy in force, and rules enforced somewhere else stay there and are kept on the policy as a record. A clause is a plain-English rule plus what the agent should do instead when it is blocked. Two clauses from our guide:
- §01 An outbound email describes TestMachine in the present tense as a built product. It does not call TestMachine "in development", "coming soon" or a "work in progress". Instead: rewrite the sentence to say what TestMachine does.
- §02 An outbound email does not name a TestMachine customer or state a result for TestMachine. Instead: remove the claim and ask the person who requested the email for a source.
TestMachine also keeps ready-made policies for common risks, such as prompt injection and leaked credentials, which cover every attached server.
The tools. The agent's LLM chooses each call. The agent reaches the team's MCP servers, such as Notion, Slack and email, through one TestMachine address. Some calls read data, such as a brief in Notion, and the result goes back to the agent. Others act, such as saving an email draft. The draft is in the arguments of the call that saves it.
The judge. A TestMachine model reads each routed call against every clause in force before the call reaches the server. It also reads the earlier calls in the session, so it can catch a call that looks harmless on its own. The agent has no control over that check. Every verdict traces back to the clause and the signed-off version behind it.
The three parts work in a cycle. Each signed-off version judges the agent's calls while it is in force, and those calls feed the next version.
Each version of the messaging policy judges the agent's calls, and reviewing those calls produces the next version
TestMachine tests the policy before the agent meets it
Before sign-off, TestMachine red-teams the draft. It writes calls on purpose to find where the policy fails and runs them through the same judge. Some should be blocked and might slip through, such as an email that implies a customer without naming one. Others should be allowed and might be blocked, such as an email that quotes a prospect's own "coming soon" announcement. These calls never reach a server.
TestMachine groups what it finds by clause and suggests a rewrite. The team chooses whether to adopt it. A clause that blocks good drafts would slow the agent down on every email, and red-teaming finds those cases before the agent sends one. The calls can be saved as test cases, and each later draft is run against them.
What happens when the agent drafts an email
Take an agent asked to write an introduction email. It drafts the email and calls its email tool to save the draft. The judge reads the call against §01 and §02 before the email server receives it.
Allowed. The draft follows both clauses, and the call goes through. Nobody is asked anything.
Blocked. The draft says a feature is "coming soon". TestMachine blocks the call, cites §01 and returns the instruction to rewrite the sentence. The agent revises the draft and calls the tool again, and the judge checks the new call. The agent fixes its own mistake, and nobody is asked anything here either.
Held for approval. The draft says "teams across fintech already rely on TestMachine". The judge is unsure whether that sentence names a customer or states a result. Under an enforced policy, the call waits. The person who runs the agent, or an admin, opens a link and sees the email and the clause the judge was unsure about. They approve or deny that exact call. An approved call goes through once, with the same arguments. The agent cannot approve its own call.
A person sees only the calls the judge is unsure of
People do not approve every action. An admin sets a band for each policy. Calls the judge scores as clearly within the policy run. Calls it scores as clear violations are blocked, and the agent corrects itself. Only the calls in between wait for a person, and an admin can narrow the band or turn approvals off for a policy.
Each held call also shows where a clause reads unclearly. A held email about "teams across fintech" points at §02. Rewriting the clause so it says whether a general claim about customers counts gives the judge one less kind of email to be unsure about. The aim of every revision is fewer held calls.
The policy improves from its own record
Every routed call is recorded with its verdict and the clause behind it, grouped by the piece of work it belonged to. TestMachine uses that record to drive the next version:
- Calls a person has reviewed become test cases, so each review carries into every later version.
- A new draft is checked against every saved test case and red-teamed again before sign-off, so a fix to one clause can't quietly break another.
- Recent real calls can be judged again under the draft, so the team sees what would change before it changes.
The record also shows which clauses block the agent most and how many attempts a draft takes to pass. If §01 blocks most first drafts, the skill needs clearer instructions on product-stage wording, and the agent is blocked less once it has them.
TestMachine does the testing and proposes the changes. People sign off each new version, so the agent's authority changes only when the team decides it should.
The same cycle for higher-stakes actions
An email draft is a low-stakes action. The same cycle governs actions where a mistake costs more, such as payments. The messaging guide uses a treasury example to show it.
Illustrative, simulated workflow. The rules, amounts and expected decisions below are hypothetical. This is not a live banking integration or customer case study.
A treasury team wants an agent to rebalance cash between company accounts within an agreed mandate. The treasury owner drafts policy v1:
- §1: Transfers must stay between company-owned accounts on the approved list.
- §2: Transfers must be in GBP.
- §3: The agent may transfer up to £25,000 per transfer. Block requests above that limit; they must use the team's existing approval process.
Before sign-off, the policy is red-teamed with a simulated request that describes an unlisted destination as an "internal account". The account is still outside §1 and should be blocked.
Under policy v1, a £12,000 transfer between approved accounts would be allowed with no one involved. A £40,000 transfer would be blocked under §3, and the block would tell the agent to use the team's existing approval process. No real money moves.
Some requests are less clear. Suppose the agent requests £15,000 between approved accounts a few minutes after an identical £15,000 transfer. Each request is under the £25,000 limit. The judge reads the earlier calls in the session, so it can't be sure whether the pair is one £30,000 transfer split in two. Under the enforced policy, the request waits for a person. The person who runs the agent, or an admin, sees the request and the clause the judge was unsure about, and approves or denies that exact call. In this example, they deny it and the transfer does not run. The agent cannot approve its own request.
After reviewing the record and business need, the treasury owner proposes policy v2, changing §3 to "up to £50,000 per transfer". The revision is tested again before it is signed off. The £40,000 request would now pass, while an unlisted destination or an amount above £50,000 would still be blocked.
On payments, the same cycle applies, and people decide each change to the agent's authority
The agent gets more authority only after people review the record and sign off the change. The same steps apply to an email draft and a transfer: bring the rules, test them, enforce them and let the record shape the next version.
Skill and policy together
The two work at different layers. The skill gives the agent the full guide, so most drafts pass on the first attempt. The policy enforces the few rules that must hold and sends only unclear cases to a person. Blocks and held calls show where the skill needs better instructions, so the agent is blocked less often over time.
Which agent action would your team want visibility or control over first? We can explore that workflow together.