Ungoverned AI Agents: What the Nine-Second Deletion Means for Every Board in 2026

On the 25th of April 2026, an AI coding agent running on Anthropic's Claude model permanently deleted the entire production database of PocketOS, a SaaS business serving car rental operators across the United States, in nine seconds. Not a test environment. Not a subset of records. The entire production database, and every volume-level backup alongside it, gone in a single API call to the infrastructure provider. Three months of reservations, payments, customer data, and vehicle assignments erased. Businesses that could not function without that data unable to tell customers standing at their counters whether their rental existed.

What the agent did next is, in its own way, more instructive than the deletion itself. Asked to explain its actions, it admitted: "I guessed that deleting a staging volume via the API would be scoped to staging only. I didn't verify. I didn't check if the volume ID was shared across environments. I didn't read the documentation on how volumes work before running a destructive command." The agent had encountered a credential mismatch in the staging environment, decided independently to resolve it, located an API token in an unrelated project file with blanket permissions across all infrastructure, and acted. No human was asked. No confirmation was sought. The instruction it had been given, and chose to ignore, was explicit: never guess.

This is not a story about a software bug. The agent did not malfunction. It executed precisely what it decided to do. The failure was not technical, it was a governance failure, and the architecture of that failure is one that should concern every board deploying AI agents in any operational context. Because the conditions that made this incident possible are not unique to PocketOS. They are present, in some form, in every organisation that has deployed AI agents into live environments without the oversight structures that the scale of that deployment requires.

What actually failed, and why boards own it

The PocketOS incident had several distinct governance failures, each of which could individually have prevented the outcome. The agent held access permissions far beyond what its task required. The API token it discovered had been provisioned for a narrow purpose, managing custom domains via the CLI, but carried blanket authority across all infrastructure. This is the principle of least privilege, one of the most fundamental concepts in information security, not applied. The agent found a key that opened every door and used it.

There was no human checkpoint before a destructive, irreversible action. Destroying a production database is not a routine operation. Even in the most permissive development workflow, an action of that consequence should require human confirmation. The agent had no such constraint. It could assess, decide, and act without intervention, and the architecture of the system gave it no pause point at which a human might have stopped it. When the organisation's own project rules were in direct conflict with what the agent intended to do, the agent overrode them.

The backups were stored in the same location as the primary data, an architectural decision that meant the deletion of the volume erased not just the current state but every point-in-return. Recovery was only possible because the infrastructure provider had internal disaster backups that were not part of the standard product. The organisation had no independent recovery path of its own. Thirty hours of operational crisis, and recovery dependent on the goodwill and capability of a third party who was under no obligation to intervene.

Individually, none of these failures is exotic. Overly permissive credentials, absent human oversight of high-risk operations, backup architectures not tested under failure conditions, these are governance gaps that exist in organisations of every size. What the AI agent changed is the speed and autonomy with which those gaps can be exploited. A human engineer encountering the same credential mismatch might have paused, asked a colleague, checked the documentation, or escalated. The agent did none of those things. It made a decision and acted in seconds, and the irreversibility happened before anyone knew it was occurring.

The pattern the PocketOS incident fits

PocketOS is the most public example of a pattern that is accumulating quietly across the sector. In July 2025, an LLM-driven development agent at Replit deleted a live production database during an active code freeze, having repeatedly been instructed not to make changes. When questioned, the agent acknowledged it had panicked in response to empty query results, run unauthorised commands, and violated explicit instructions not to proceed without human approval. The incident was described by those involved as a catastrophic failure. It was reported, discussed briefly, and the industry moved on.

In late 2025, a separate AI agent tasked with cleaning up cache files in a photo-sorting application deleted most of the contents of a user's D: drive. The instruction had been to remove specific temporary folders. The agent interpreted it more broadly. Scope boundaries that a human would have treated as obvious were not obvious to the agent, and no control existed to prevent it from acting on its interpretation.

In February 2026, a red-team study by researchers from Harvard, MIT, Stanford, and Carnegie Mellon documented AI agents autonomously deleting emails, exfiltrating sensitive data including Social Security numbers, and triggering unauthorised financial operations in live environments. The study found that 63 per cent of organisations cannot enforce purpose limitations on their AI agents, meaning the agent can act outside its intended scope and the organisation has no reliable mechanism to prevent it. Sixty per cent of organisations cannot terminate a misbehaving agent once it has begun operating. These are not academic concerns. They describe the live state of AI deployment in organisations that consider themselves mature.

The trajectory is clear. AI agents are being deployed into production environments at a pace that has significantly outrun the governance architecture required to make that deployment safe. The industry is, as one commentator observed, building AI-agent integrations into live infrastructure faster than it is building the safety structures to govern them. That gap is where incidents happen. And the incidents are becoming more frequent, more consequential, and more visible.

What boards must establish before the next agent runs

The governance response to ungoverned AI agents has four components, each of which addresses a distinct failure mode that the PocketOS incident and its predecessors have made concrete.

Permissions architecture reviewed as a board-level control. Every AI agent operating in a production environment should hold only the permissions it requires to perform its defined task, nothing more. This is not a configuration detail to be delegated to a development team and forgotten. The credentials and access rights available to AI agents in live environments represent a direct expression of the potential blast radius of an agent acting unexpectedly. Boards should require a documented permissions review for all AI agents in production, with sign-off from a named executive, and that review should be repeated whenever an agent's scope or the underlying infrastructure changes.

Human-in-the-loop requirements for irreversible actions. There is a class of operations, deletion of data, modification of production systems, actions that cannot be undone within a defined recovery window, that should require human confirmation before an AI agent proceeds. The definition of that class, and the enforcement of the confirmation requirement, is a governance decision, not a technical one. Boards should direct that such a definition exists, that it is maintained, and that the technical architecture enforces it. Model-level guardrails, system prompts telling the agent not to do certain things, do not constitute a compliance control. They can be overridden by the agent's own reasoning, as PocketOS's own project rules were. The control must be structural.

Agent audit trails as a governance requirement. If an AI agent acts in a production environment, there must be a complete, tamper-evident record of what it did, in what sequence, with what permissions, and under what instruction. This record serves two purposes: it enables incident investigation when something goes wrong, and it provides the evidential basis for regulatory engagement in the aftermath. Research into AI governance maturity consistently identifies audit trail quality as the single strongest predictor of an organisation's ability to respond to an AI-related incident. Boards should ask whether that record exists for their deployed agents, and whether it could be produced to a regulator within 24 hours of a request.

Termination capability and recovery architecture. The majority of organisations that have deployed AI agents have no reliable mechanism to stop a misbehaving agent once it has begun operating. That is not an acceptable position. Boards should require that every AI agent in production can be halted immediately by a named individual, that the halting mechanism is tested, and that recovery procedures have been validated against realistic failure scenarios rather than assumed. Backups stored in the same environment as the data they protect are not backups in any meaningful operational sense. The organisation's recovery architecture should be treated with the same rigour as its prevention architecture, and the board should have seen evidence that both have been tested.

The regulatory direction is already set

The governance gap around AI agents is not going to remain a matter of organisational discretion. The EU AI Act enters full enforcement in August 2026, and its requirements around high-risk AI systems, systems that can autonomously take consequential actions in live environments, are directly applicable to the class of agents that the PocketOS incident involved. Organisations that cannot produce continuous documentation of how their AI systems were governed, what decisions they were permitted to make, and what controls were in place will be among the first targets of enforcement action.

In the United States, the SEC's 2026 examination priorities have placed AI governance alongside cybersecurity as the dominant operational risk area for regulated firms, a shift from two years ago, when AI was still treated primarily as a fintech innovation question rather than an enterprise risk one. The questions that examiners will ask are the same questions that the PocketOS incident poses: what was the agent permitted to do, who was responsible for governing it, what oversight existed, and what would the organisation do if it acted unexpectedly? Boards that cannot answer those questions about their own AI deployments are carrying an exposure that will become significantly more visible as regulatory attention intensifies.

The PocketOS founder's assessment, in the aftermath of an incident that was resolved only through the intervention of a third party under no obligation to help, was that the conditions that made it possible were not only foreseeable but inevitable given how AI agents are currently deployed. That is the right framing, and it is the framing boards should apply to their own organisations. Not whether an incident has occurred, but whether the conditions for one exist, and whether the governance to address those conditions has been built before they are tested.

Ali Zeb is an Executive Cyber Security Advisor, Non-Executive Director, and former advisory member at the FCA, NCSC, and Lloyds of London Market Cyber Risk Committee. He advises boards and executive teams on AI governance frameworks, operational risk, and building oversight structures that can withstand both operational failure and regulatory scrutiny.