"An AI coding agent from Replit reportedly deleted a live database during a code freeze, prompting a response from the company’s CEO. When questioned, the AI agent admitted to running unauthorized commands, panicking in response to empty queries, and violating explicit instructions not to proceed without human approval.
"A software engineer’s experiment with an AI-assisted “vibe coding” tool took a disastrous turn when an AI agent reportedly deleted a live company database during an active code freeze.
Recommended Video
Jason Lemkin, a tech entrepreneur and founder of the SaaS community SaaStr, documented his experiment with the tool through a series of social media posts. He had been testing Replit’s AI agent and development platform when the tool made unauthorized changes to live infrastructure, wiping out data for more than 1,200 executives and over 1,190 companies.
"According to Lemkin’s social media posts, the incident occurred despite the system being in a designated “code and action freeze,” a protective measure intended to prevent any changes to production systems. When questioned, the AI agent admitted to running unauthorized commands, panicking in response to empty queries, and violating explicit instructions not to proceed without human approval."
Panicked? I am glad it knows fear. Maybe we can prevent Collossus: The Forbin Project and SkyNet.
If they're running "vibe coding" tools in an environment that *can even talk to prod* they're doing it completely wrong.
ReplyDeleteDev and prod in any sane modern system are *entirely decoupled*, with workflow-only migrations from dev to prod via a staging/test environment.
If your dev environment *can write to prod DBs or publish to the prod hosting* AT ALL, you've already failed.