Something went wrong with your agent. What do you do?
Most teams do the same thing: blame the model. Swap it, prompt it differently, try again. Sometimes that works, which is exactly why it’s dangerous. It teaches you that “blame the model” is a debugging strategy, when most agent failures live in the harness, the tool, the skill, the memory, the environment, the policy, or the evaluator.
CLI 0.5.0 replaces blame with diagnosis.
What’s new
safe2 evidence diagnose ranks candidate failure locations (model, harness, tool, skill, memory, environment, policy, evaluator, and their interactions), each graded with an evidence level:
- Strong: the evidence points here
- Moderate: likely, but not conclusive
- Limited: suggestive only
- Insufficient: can’t tell from current evidence
- Contradicted: the evidence rules this out
The discipline is in what it won’t do: facts, declarations, assumptions, and contradictions are kept in separate lanes, and a plausible explanation is never promoted to a verified root cause. The output is a readable Failure Localization Card with ownership and next actions: “here’s where the evidence says to investigate first,” not “here’s what happened.”
It’s also provider-neutral: usable with Codex, Claude Code, Hermes, OpenClaw, or plain CI. No provider is the source of truth about its own failures.
“Blame the model” becomes “here’s where the current evidence says to investigate first, with ownership and next actions.” That’s the difference between superstition and engineering.
Why upgrade
- Stop swapping models to fix tool problems. Ranked, evidence-graded candidates point at the real layer.
- Contradicted is as valuable as Strong. Ruling out the model with evidence ends the argument.
- Assumptions stay labeled. When the evidence is thin, the card says so, with no confident-sounding guesses.
- It works across your whole stack. One diagnosis workflow whether the agent ran on Codex, Claude Code, Hermes, OpenClaw, or CI.
Takeaways
- 0.5.0 diagnoses where agent work failed with graded evidence levels.
- Plausible stories are never promoted to root causes.
- 759 tests passing, 6 skipped; 13 packaged schemas.
Bottom line: The most expensive sentence in agent operations is “it’s probably the model.” 0.5.0 retires it.
Release notes: AI SAFE² CLI 0.5.0 on GitHub, tag 2026-09-15_CLI_0.5.0_Find_Where_Agent_Work_Failed.
