← Writing · Research note ·
Prompt less, check more
An AI agent can follow a rule in one reply and break it in the next. We tested code checks on proposed messages and actions, with specific feedback when a check fails. Carefully designed checks reduced detected violations in our tests. Poorly designed checks made agents less effective.
Jake Spillane · Hivemind Labs
01 Key findings
- 01Tested on τ²-bench, IFEval and 74 previously rejected production turns.
- 02Precise checks cut detected violations without hurting task completion.
- 03Generic retries and poorly designed checks made agents worse, so check the false-positive rate first.
02 Full article
hivemind.au
The full research note, with methods, figures and limits of the evidence, is published on the Hivemind Labs site.
Read the full article →