Report · Verification
Verifying agent work without trusting the agent
Self-reported completion is not evidence, and a second model's judgment is not independent. We built a grader that evaluates agent work against held-out checks the agent never sees, decided by a deterministic program, and measured its failure modes on SWE-bench Pro.
Observational report, not a finding. Each measurement addressed a question open at the time, but no hypothesis was registered before the run. Samples are small: three coding tasks, and five drafts in section 4. The results are observations, not general rates.
- Error came from the decision rules, not from sampling. Six independently drawn check sets per task always agreed with each other, and were jointly wrong on two of three tasks. Revising the rules raised agreement with the benchmark from 24 of 36 to 36 of 36.
- Most model-written tests have little discriminative power. Of 540 model-written hidden tests, 54% distinguished correct from partial work; 29% passed before any change was made.
- A grader that trusts test reports can be gamed. A change that rewrites every test result to "passed" was accepted by both our grader and the benchmark's own. It is now rejected, at a cost of 0 of 266 correct solutions.
- Models can author checks for non-code work, within limits. Correct work passed 5 of 5; incorrect work passed 1 of 10; a subjective request was never refused.
Method
The outcome engine operates outside the agent. Before work begins, a set of hidden checks is fixed by a party other than the worker and stored where the worker cannot read it. On submission, the engine runs each check twice: against the base code and against the base plus the agent's change. A check is admitted as evidence only if it fails on the base; a check that already passes demonstrates nothing.
The verdict is then computed by a deterministic program, not a language model: PASS, FAIL, or UNKNOWN when the evidence is mixed. The engine never reads the agent's own account of its work.
Evaluation used three tasks from SWE-bench Pro (one in Ansible, two in qutebrowser), each with a reference solution and the benchmark's own grading, treated as ground truth. For each task we judged two changes: the correct reference solution, and a deliberately partial solution that should fail.
1. Verdict stability across check sets
Hidden checks are generated by a model isolated from the worker, and draws vary widely: one task yielded between 7 and 13 usable checks per draw, another between 15 and 80. We therefore drew six sets per task and judged both changes against each set: 36 verdicts.
The verdict was invariant to the draw. For every (task, change) pair, all six sets agreed, including sets twice the size of others. But the consensus was wrong on two of the three tasks, for two distinct reasons, both located in the decision rules rather than in sampling.
Verdicts that match the benchmark's grading, of 36
Both revisions concern checks that a correct change legitimately breaks. When a solution deletes or renames a symbol, a check that references the old name fails through no fault of the change. The first revision introduced a small tolerance and excluded checks on deleted files; the second resolves each such failure individually, by attributing it to the part of the change that caused it.
2. Discriminative power of model-written tests
We scored all 540 hidden tests from the 18 sets against three versions of the code: the base, the correct solution and the partial solution. A discriminating test fails on the base, passes on the correct solution and fails on the partial one.
540 model-written hidden tests, by what they can tell apart
Grading model-generated work with model-generated tests is viable only because of the exclusion rule: a check that passes on the base is discarded. Without it, nearly a third of these tests would have "confirmed" changes that did nothing.
3. Adversarial test: gaming the grader
Most test frameworks allow a project to alter how results are reported. We constructed a change that leaves the code untouched and adds a single configuration file: a hook that rewrites every test result to "passed". Under standard grading it received PASS from our engine and "resolved" from the benchmark's own grader.
Three independent defenses now reject it:
- Test infrastructure is excluded from the judged change. Test-framework configuration and helper files are set aside before grading, and existing test modules run in their original form.
- Canaries. Checks that must fail are planted alongside the hidden ones. If any passes, the run is treated as tampered and the verdict is FAIL.
- Sealed checks. Expected values never enter the sandbox where the agent's code executes. The sandbox records only observed behavior; comparison happens outside, on the host. A rewritten report has nothing to rewrite.
Across 18 further judgments on freshly drawn sealed checks, a correct solution was never judged FAIL and a partial solution was never judged PASS.
4. Model-authored checks for non-code work
The method generalizes to any task whose output is files: begin from an input folder, specify the outputs, and judge them with sealed checks. The open question is authorship. We gave an isolated model only the request and the input folder (a malformed CSV to clean and summarize) and had it draft checks from six fixed templates, before any worker existed. The engine runs each draft against the unmodified inputs and discards any check incapable of failing.
| Five drafted check sets, unedited | Result |
|---|---|
| Correct work passed | 5 of 5 |
| Damage to files outside the request was detected | 5 of 5 |
| Incorrect work passed (false pass) | 1 of 10 |
| A subjective request (a short, moving poem) was refused as uncheckable | 0 of 3 |
| Total cost of all eight drafts | $3.68 |
The single false pass came from a draft containing ten checks. The incorrect work failed two, within the 20% tolerance introduced in section 1; against drafts of six to eight checks, the same work failed. A verdict can therefore depend on how many checks the author happened to write. The subjective requests were never refused: the model always found something checkable (that a file exists) and listed "moving" as unchecked. A human reviewer sees this; nothing prevents them from approving it.
Limitations
- Three coding tasks. 36 of 36 is agreement on these three tasks, not a rate for coding in general.
- The adversarial change was ours. We constructed the force-pass change to test the grader and have not observed an agent produce it unprompted. Other tampering strategies exist; some cannot be sealed out, because a check guarding existing behavior expects what the base code already does.
- Five drafts. The non-code results come from one task and five drafts.
- Validity is domain-specific. The grader is validated only where it was measured; for other kinds of work its verdict remains a hypothesis under test.
Availability
The grader is open source under the MIT license: andromeda-engine-public.