The White House is right that advanced AI systems with significant cyber capabilities should be tested before they enter sensitive environments. But an invitation-only testing pathway risks becoming a de facto approval system, unless the rules themselves are public.
At a glance:
- What happened: An emerging White House framework reportedly gives selected AI companies a voluntary pathway to provide powerful models for government testing before release, while the framework itself, its eligibility standards, and key participation details remain unpublished.
- What it exposes: A voluntary program can harden into a de facto approval system, creating an assurance asymmetry between companies inside the process and everyone outside it.
- What to examine: Whether your organization treats participation in such programs as a proxy for safety, and whether your assurance rests on a brand’s name or on a governed architecture.
Key quote: “You can classify the test. You cannot classify the rulebook.”
The White House is right about one thing: advanced AI systems with significant cyber capabilities should be tested before they are placed into sensitive environments. And some parts of that testing should remain classified.
This is not a close call. Publishing exploit chains, zero-day vulnerabilities, classified threat intelligence, or exact adversarial test vectors could help the very actors the government is trying to stop. A serious testing program needs protected examination material, just as a serious exam needs questions the students have not seen.
But protecting the examination is not the same as concealing the rules governing who may take it. And that is where the emerging framework runs into trouble.
The emerging White House framework reportedly gives selected AI companies a voluntary pathway to provide powerful models for government testing before release. The framework itself, its eligibility standards, and important details about participation have not been made public. Note the word reportedly: the details are not published, which is itself the problem being described. We are discussing the structure of the program as it has been reported, not asserting a definitive account of unpublished government deliberations.
That creates a serious risk. Not because testing is wrong, but because of what an opaque testing program becomes.
The examination stays sealed; the rulebook stays open. Protect the test, publish the rules. Illustrative artwork.
A Voluntary Pathway Can Become a De Facto Approval System
Key quote: “That is not a technical determination. It is an assurance asymmetry.”
Here is the mechanism. A voluntary program sounds harmless: nobody is compelled, nobody is excluded by law. But voluntary participation, once visible, becomes a signal. Models inside the process become associated with government scrutiny and trust. Models outside it may be treated as suspect, even when they were never given a pathway to participate.
Consider who sits outside the pathway: startups, universities, independent laboratories, and open-weight communities. These are not marginal actors; they are a large share of where AI capability is actually being built and studied. If the price of being treated as trustworthy is admission to an invitation-only process with unpublished eligibility criteria, then trust is being allocated by access, not by evidence.
That is not a technical determination. It is an assurance asymmetry. Two models with equivalent or even inverted safety properties can end up on opposite sides of the trust line purely because one had a relationship with the program and the other did not.
A voluntary program can harden into a de facto approval system, creating an assurance asymmetry unrelated to technical merit. Conceptual diagram.
Now ask where this touches your organization. When you evaluate a model or a vendor, do you treat participation in government testing as a proxy for safety? If so, what exactly does that participation certify: the model’s current behavior, the vendor’s cooperation, or simply its access to the program? The answer matters, because procurement decisions built on the trust signal inherit the asymmetry. You may be selecting for connectedness while believing you selected for safety.
The fix is not to publish classified benchmarks, and it is not to abandon cyber testing. The fix is to separate what must remain protected from what must remain public, and to hold that line deliberately.
Separate What Must Be Protected From What Must Be Public
Key quote: “Failure to participate in an invitation-only or inaccessible voluntary evaluation cannot be treated as evidence that a model is unsafe.”
The government should protect the examination material: exact test vectors, classified intelligence, sensitive vulnerabilities, and operational exploit techniques. Each of these, published, would arm the actors the program exists to defend against. That part of the secrecy is legitimate and should be defended.
The government should publish the rulebook: which models are covered, who may apply, what capabilities are evaluated, how findings are categorized, how conflicts are controlled, how decisions may be reconsidered, and how agencies may, and may not, use participation in procurement. None of this helps an adversary build a weapon. All of it helps the public, the market, and the research community understand what the program is and who it serves.
Exact test vectors stay classified; eligibility, criteria, and decision processes stay public. Proposed policy position.
And there should be a clear rule, stated plainly: failure to participate in an invitation-only or inaccessible voluntary evaluation cannot be treated as evidence that a model is unsafe. Without that rule, the program quietly converts non-participation into suspicion, and suspicion into a procurement disadvantage, without any technical finding ever being made. That outcome would punish exactly the independent and open-weight ecosystem that keeps the field honest.
This is a proposed position, not a description of current law. But it is the minimum condition under which a voluntary testing program can coexist with broad innovation instead of quietly sorting the market into the trusted and the suspect.
Capability May Be Tested. Authority Must Be Governed Continuously.
Key quote: “Capability may be tested before release. Authority and alignment must be governed continuously in operation.”
Even a transparent pre-release test is only one part of the answer, and this is the point most easily lost in the policy debate. A model can pass an evaluation and later be updated, connected to new tools, given persistent memory, granted greater authority, or placed inside a materially different architecture. The thing that was tested is not the thing that is running.
A pre-release test measures capability under stated conditions. It cannot certify what the system will be permitted to do six months later, in a different deployment, with different tools and different delegated authority. That gap is not a flaw in testing; it is a fact about how deployed systems change. Real assurance must therefore continue after deployment, controlling and recording the things that actually determine what happens in operation:
- Identity and permissions
- Data and network access
- Tool use
- Memory
- Human approval
- Behavioral drift
- Intervention and containment
- The ability to replace the model without losing the mission
This list is the difference between trusting a brand and trusting an architecture. A brand tells you who built the model. An architecture tells you what the model is allowed to do, who authorized it, and what evidence exists that the boundary held.
A passed pre-release test does not describe the deployed system. Assurance must continue across eight control points. Proposed verification steps.
The United States should not choose between serious security and broad innovation. It can have both, if it keeps the two jobs separate. Test the capability. Publish the rules. Open the pathway. Govern the runtime.
Trusted architecture, not trusted brands. That is the standard that protects the country without quietly deciding, in advance, which builders are allowed to be trusted.
What to Do Next
1. Inspect. Review how your organization evaluates AI models and vendors. Identify every place where participation in a government or industry testing program is treated as a proxy for safety, and ask what that participation actually certifies about the deployed system you will run.
2. Test. Pick one high-impact deployed workflow and verify the eight continuous-assurance control points against it: identity and permissions, data and network access, tool use, memory, human approval, behavioral drift, intervention and containment, and model replaceability. Record which points have real evidence and which are assumed.
3. Decide. Require published criteria before treating any testing program as a trust signal in procurement, and adopt the clear rule internally: non-participation in an inaccessible evaluation is not evidence a model is unsafe. Then put your assurance budget where the risk lives, in the runtime, not the brand.
Find Your Exposure First
Free AI governance assessment, instant score, no sales call required.
Score Your AI Risk in 8 Minutes · Book a Strategy Briefing
Test the capability. Publish the rules. Open the pathway. Govern the runtime.
