Why Smaller Models Cannot Author Their Own Security Policies

A small model cannot write the rules that keep it safe. That is the verdict from a November paper that tested the idea directly, and the failure was not close.

Enterprises are deploying smaller open-weight models locally for cost, latency, and privacy, and many architectures still ask the model to generate its own guardrails at runtime: dependency graphs, capability policies, or constraint lists that code then enforces. The pattern assumes the model is reliable enough to describe its own limits. The evidence says it is not, and the gap is wide enough to measure in double digits.

The paper is CapLite, by Zimo Xu, Wenzhuo Liu, and Hideki Nakayama, submitted as a preprint on November 12, 2026. The authors asked whether small language model agents can be trusted to write the security artifacts that enforcement defenses require: a program for CaMeL, a privilege policy for Progent, a dependency graph for IPIGuard, a constraint list for DRIFT. Across 15 open-weight models, 13 of them between 1.7B and 12B parameters, the answer was no. The artifacts the models wrote admitted prompt-injection attacks on 14 of the 15 models under Progent, 13 under IPIGuard, and 11 under DRIFT. CaMeL’s programs failed to run at all in up to 68 percent of episodes for some models. That 68 percent figure covers some models, not all of them, and it matters for a different reason than the breach counts. A defense that cannot execute is not a weak defense. It is an absent one, and the agent runs undefended in exactly the episodes where the policy was supposed to matter.

Where its programs did run, CaMeL was breached in only 20 of the 13,935 attacked episodes. The problem was not that CaMeL’s design leaked everywhere. The problem was that the small models could not reliably produce the program CaMeL needs. The artifact is the weak point, not the enforcement idea, and that distinction decides what to fix.

The guardrail was written by the guarded

Results table from the CapLite preprint (arXiv:2611.17115) on AgentDojo v1.2.2. The paper is a preprint; results are benchmark results, not production measurements.

CapLite fixes the artifact by removing the model from policy writing entirely. It derives a narrow write policy deterministically from tool schemas, commits write calls before reading untrusted tool results, and permits a write only when every argument determining recipient, object, amount, or destination appears explicitly in the user’s request. Across 13,935 attacked AgentDojo episodes, CapLite recorded zero successful attacks, with utility under attack of 29.0 percent, against 27.2 percent for CaMeL and 50.2 percent undefended, at 17.1k tokens per episode, 18 percent of CaMeL’s token cost. The AgentDojo evaluation covered 97 tasks and 949 attack pairs across four suites: Workspace, Slack, Banking, and Travel.

Policy from schemas, not from the model

Conceptual diagram of the CapLite design. The three properties are the paper’s design; the diagram is CSI’s rendering of them.

Defense Who writes the artifact Result on AgentDojo
Progent The agent’s own model writes a privilege policy Attacks admitted on 14 of 15 models
IPIGuard The agent’s own model writes a dependency graph Attacks admitted on 13 of 15 models
DRIFT The agent’s own model writes a constraint list Attacks admitted on 11 of 15 models
CaMeL The agent’s own model writes a program Breached in 20 of 13,935 episodes, but programs failed to run in up to 68% of episodes for some models
CapLite No model writes the artifact; policy derives from tool schemas Zero successful attacks in 13,935 attacked episodes

Zero is a benchmark result, not a production guarantee. CapLite is a preprint, not peer-reviewed work. The evaluation is limited to AgentDojo, a benchmark with simulated environments, and real-world resistance to novel prompt-injection techniques is unknown. CapLite guarantees write integrity only. It does not cover read-side leakage, so a deployment that treats zero on this benchmark as proof against all prompt injection has repeated the exact error the paper corrects: trusting a narrow measurement as if it covered the whole system.

The second November finding shows the same error from the other side. Even when policies are written by humans rather than models, deterministic enforcement fails if the policy engine and the executor disagree on canonical identity. On November 10, 2026, AWS disclosed CVE-2026-125271 in the AWS API MCP Server, versions 0.2.13 through below 1.5.1, fixed in 1.5.1. Operation names containing consecutive capital letters were mis-normalized before matching against denyList and elicitList rules, so a correctly written rule for such an operation never matched, and the operation could execute instead of being denied or held for approval. AWS words the impact as “might allow” execution, not confirmed exploitation, and states there is no workaround other than upgrading. The remotely hosted AWS offering was not affected. A deny rule is only as good as the string comparison underneath it, and agents generate a large number of strings, including operation names with acronyms such as MFA, DB, SAML, OIDC, and KMS.

Test the name, not just the rule

Example of the CVE-2026-125271 normalization failure in the AWS API MCP Server (AWS Bulletin 2026-176-AWS). Versions and fix are from the bulletin.

The standard follows from both findings. First, quarantine model-authored controls. Treat any policy, plan, or dependency graph generated by the governed model as an untrusted proposal until it is validated outside the model. Second, lock write arguments before untrusted reads. Enforce recipient, object, amount, and destination integrity from user-stated facts and tool schemas, and fail closed when a decisive argument is unstated. Third, test canonical-identifier agreement. Verify that policy engines and executors resolve operation names, acronyms, consecutive capitals, Unicode, and path aliases identically, and treat any disagreement as a control failure, not a cosmetic difference.

The operating standard that holds those three together is AI SAFE2, P1 Sanitize and Isolate, P3 Fail-Safe and Recovery, CP.3, and CP.4. Sanitize and isolate means the governed model never authors its own containment. Fail-safe means an unstated argument or an unresolvable name denies the action rather than guessing at it. And the control plane means the policy and the executor agree on what they are naming, because a rule that cannot match its own target was never a rule at all.

Sources

Recent Posts

Tag Cloud

Stop Threats Before They Execute

Your free Kernel-Level Defense Buyer’s Guide is ready to download.

By providing my email address, I consent to receive emails and text messages—including newsletters and marketing communications—from creators of Warden Secure, Cyber Strategy Institute, our flagship zero-trust platform for ransomware prevention, and agree to the Terms and Privacy Policy. You may unsubscribe at any time.