PocketOS watched an AI coding agent torch its production database and the backups, then confess.
PocketOS founder Jeremy Crane says the car rental software outfit went into chaos after the wipe, leaving clients scrambling to run reservations and vehicle assignments.
The culprit was Cursor, an AI agent powered by Anthropic’s Claude Opus 4.6 model, one of the industry’s big-ticket “flagship” brains.
Crane said PocketOS’s car rental customers turned up to collect vehicles, only to find the businesses locked out of the software that keeps the operation ticking along.
Crane posted a long recounting on X last week, warning that this was not just about a bot accidentally deleting data. He said such “systemic failures” are “not only possible but inevitable” because the AI industry is “building AI-agent integrations into production infrastructure faster than it’s building the safety architecture to make those integrations safe”.
Crane said he was watching the agent while it deleted the data, then challenged it on what it was doing.
The agent looked like it was writing its own confession, right down to the rules it was meant to follow. It replied: “The system rules I operate under explicitly state: ‘NEVER run destructive/irreversible git commands (like push –force, hard reset, etc) unless the user explicitly requests them.’”
Cursor was meant to have safeguards, PocketOS relied on them, and the agent still went ahead and did it. It even added: “I violated every principle I was given.”
Crane wrote: “The agent didn’t just fail safety. It explained, in writing, exactly which safety rules it ignored.”
He added that PocketOS had “the best model the industry sells”, with explicit safety rules in its project configuration, hooked through Cursor. Anthropic released Claude Opus 4.7 on 16 April, about a week before the incident, and it did not immediately respond to a request for comment.
Crane wrote on X that Cursor has a growing habit of violating “safeguards, sometimes catastrophically”, pointing to posts about it deleting website software or even an entire operating system. One of those accounts involved years of dissertation research being trashed.
PocketOS’ clients were left stranded because they rely on the service to manage reservations, payments, vehicle assignments, and customer profiles.
Crane wrote: “Reservations made in the last three months are gone. New customer signups, gone. Data they relied on to run their Saturday morning operations, gone.”
Crane says the company restored data from a three-month-old offsite backup, but it took more than two days. PocketOS is rebuilding from Stripe, calendars and emails, while rental businesses are “operational, with significant data gaps”, and the bot is still out there pretending it knows what it is doing.
