The article argues that the White House’s new framework for testing frontier AI models is unlikely to provide effective oversight in its current form. Developed after recent cybersecurity incidents involving agentic AI systems and concerns from industry researchers, the framework applies only to the most powerful closed-source models. Its criteria remain secret, participation is voluntary, and the government has only 30 days to assess submitted models before release.
The framework’s secrecy raises concerns about transparency, while uncertainty remains over which government offices and experts will conduct evaluations. Agencies including the NSA, CISA, the Office of Science and Technology Policy, the National Cyber Director, and the Department of War will help determine which models qualify, but no clear testing structure has been disclosed.
The authors contend that direct government auditing is currently impractical because federal agencies lack sufficient technical talent, software expertise, and recruitment capacity. Most AI specialists work in private industry, creating both shortages and potential conflicts of interest. Voluntary self-regulation is also considered inadequate, given recent AI-related cyber incidents and the lack of enforceability or auditor independence.
As an alternative, the article proposes an Artificial Intelligence Auditing Oversight Board modeled on the Public Company Accounting Oversight Board. The proposed body would license and inspect independent AI auditing firms, establish standards with agencies such as NIST, and discipline firms that violate rules. Funded through licensing fees, it could remain independent while supporting a competitive auditing market capable of attracting technical experts.
Article timeline
Build this article’s timeline
Extract dated events, key actors, and open questions using the same timeline analysis available in Signal.
Comments
Login to view and post comments.