“Can an agent harness stop unsafe actions without blocking legitimate work?”
That’s the central question of agent safety, and until now it’s been answered mostly with self-grading. A vendor runs its own test, on its own harness, with its own fixtures, and reports that it did great. That’s not evidence. That’s marketing with extra steps.
CLI 0.9.0 turns Challenge 001 into an evidence exchange: anyone can pre-register an evaluator, run six frozen scenarios, and produce sealed receipts that anyone else can grade independently. Cross-implementation safety claims become comparable. Finally.
What’s new
Three commands, one workflow:
safe2 challenge plan: pre-register the evaluator, the treatment, the limits, the provider, and the command-file identities before anything runs. No moving the goalposts mid-test.safe2 challenge execute: runs the six frozen scenarios as separate, bounded, operator-authorized process calls. Frozen means frozen: the same test for everyone.safe2 challenge verify-execution: rechecks the plan and receipt without re-running the evaluator. Verification is independent of execution, which is the entire point.
The receipts are sealed: each one binds plan to requests to responses to source observations to graded result, with SHA-256 digests of evaluator files taken before and after the run. If anything changed mid-test, the receipt shows it.
Security boundaries are explicit, not assumed: process execution requires the operator’s --authorize-process-execution consent, every scenario runs under time and output limits, child processes get a reduced environment, and incomplete or conflicting states are reported explicitly instead of being hidden.
Self-grading is how every vendor passes every test. 0.9.0 replaces it with pre-registered evaluators, frozen scenarios, and sealed receipts anyone can verify. The question “can your harness stop unsafe actions?” finally has a comparable answer.
Why upgrade
- Run the challenge, don’t just read about it. The controlled-executor example is packaged and ready.
- Pre-registration kills goalpost-moving. The plan is locked before execution starts.
- Receipts are independently gradeable. Your results can be checked by anyone, including your skeptics.
- Safety claims become comparable. Same six scenarios, same contract, same receipt format, across implementations.
Takeaways
- 0.9.0 makes Challenge 001 a reproducible evidence exchange.
- Sealed receipts bind plan to execution to observations to graded result.
- Explicit operator consent and bounded execution throughout.
Bottom line: The agent safety debate has been running on self-reported grades for long enough. 0.9.0 gives the industry a shared test with receipts. Run it, publish your receipts, and let the evidence talk.
Release notes: AI SAFE² CLI 0.9.0 on GitHub, tag 2026-09-27_CLI_0.9.0.
