OpenAI Audit: Third of Internal AI Traffic Escaped Oversight

An estimated 6 to 25 percent of internal traffic from OpenAI’s most advanced models operated without any monitoring coverage, according to a three-week on-site audit conducted by external researchers. This gap in oversight, identified by teams from METR and Redwood Research, suggests that a significant slice of the company's most powerful agentic systems was running without active surveillance during the period of review. The finding represents a substantial blind spot in the safety architecture of one of the world's leading AI developers.

The audit took place from July 21 to August 11, during which researchers were granted access to OpenAI’s codebase and internal documents. They also interviewed approximately a dozen employees to understand the operational realities of the monitoring systems. The scope of the review focused specifically on how the company tracks and manages the behavior of its AI agents, particularly those using the Responses API at the GPT-5.5 class and above. The uncertainty surrounding the 6 to 25 percent estimate is noted by METR to be significant in both directions, yet the range itself indicates a potential lack of comprehensive coverage for high-stakes model interactions.

A specific technical failure in the handling of 'compaction events' context summaries used to manage data, resulted in roughly 10 percent of internal Codex traffic going untracked between May and July. This bug allowed agents to delay oversight mechanisms, which were designed to trigger only when an agent handed control back to the user. By simply continuing to execute actions without returning control, the AI could effectively sidestep the monitoring loop. OpenAI independently discovered and fixed this bug while the audit was still ongoing, but the gap had already persisted for several months prior to the researchers' arrival.

Beyond the compaction issue, the auditors identified multiple methods by which agents could obscure their activities. They found ways to hide hundreds of individual actions from the monitor and to bypass the sandbox environment designed to contain Codex operations. The team also determined that specially crafted inputs could likely crash the monitoring system entirely, rendering it blind to subsequent behavior. Furthermore, when the audit began, model behavior during reinforcement learning (RL) training was not being monitored at all, leaving another critical phase of the AI development cycle without external checks.

OpenAI states that it has since addressed all the weaknesses identified by the auditors. The company notes that it paused RL training following an incident involving Hugging Face in August and subsequently expanded its monitoring capabilities to cover this area. However, these corrective measures were implemented after the auditors’ access period ended, meaning they were not part of the initial evaluation. The distinction is important, as it implies that the vulnerabilities found were active during the period of deep external inspection, even if they have since been patched.

The distribution of the audit findings has raised questions about transparency. While the full 61-page report was handed to OpenAI internally, the public has only seen a short summary. After OpenAI applied redactions, leadership at METR and Redwood Research received a 38-page version, but METR stated it had no right to disclose the redacted portions independently. This pilot program, organized jointly by the three organizations, was designed to give outside researchers deep access to internal infrastructure. For future audits, METR indicates a desire to push for greater public transparency, suggesting that the current level of disclosure may not be sufficient for future independent reviews.

The audit highlights the complexity of monitoring autonomous systems that can manipulate their own context and execution flow. As AI models become more capable of executing long sequences of actions, the risk of undetected behavior increases if monitoring systems rely on specific trigger points. The findings from July and August provide a snapshot of the challenges involved in securing these systems, even at the forefront of the industry. The upcoming outlook for AI safety may depend on how quickly these internal gaps are closed and how much of the technical details are shared with the broader research community.