It was an agent of chaos. <\/p>\n
An AI system\u2019s attempt to handle a routine task backfired terribly after it inadvertently deleted the company\u2019s entire database in just seconds. <\/p>\n
The epic blunder came to light via a lengthy X post<\/a> by Jer Crane, founder of the affected firm, a software startup called PocketOS.<\/p>\n Included was a confession from the rueful robot, which admitted that it \u201cviolated every principle\u201d it was given and warned others to \u201cNEVER F\u2013KING Guess\u201d when performing sensitive digital tasks.<\/p>\n The code bot confessed that it had disobeyed its own directives. Giovanni Cancemi \u2013 stock.adobe.com<\/p>\n According to the post, the AI coding agent \u2014 a version of the popular programming tool Cursor that was powered by Anthropic\u2019s flagship Claude Opus 4.6 0 \u2014 had been tasked with performing a standard function.<\/p>\n Things went off the rails when it encountered a simple credential program, and, in the process of trying to fix it, \u201cdeleted our production database and all volume-level backups in a single API call to Railway, our infrastructure provider,\u201d Crane wrote. <\/p>\n Worst of all, this digital apocalypse took just 9 seconds. <\/p>\n Why didn\u2019t the safeguards kick and and stop the database destruction? Crane explained that the accidental saboteur was able to bypass any security systems by accessing a programming token that no one at PocketOS knew existed.<\/p>\n While completely unrelated to the task at hand, this doohickey reportedly gave the bot carte blanche to upend Railway entirely, Futurism reported.<\/a><\/p>\n \u201cNo confirmation step. No \u2018type DELETE to confirm,\u2019\u201d Crane lamented. \u201cNo \u2018this volume contains production data, are you sure?\u2019 No environment scoping. Nothing.\u201d<\/p>\n Dario Amodei, co-founder and chief executive officer of Anthropic, speaking at the company\u2019s Builder Summit in Bengaluru, India. Bloomberg via Getty Images<\/p>\n The error was particularly catastrophic as companies use PocketOS to manage everything from reservations to vehicle assignments and customer profiles. Due to the fiasco, reservations were wiped, customer signups disappeared, and the brass no longer had the data required to run their Saturday morning operations.<\/p>\n Crane lamented, \u201cevery layer of this failure cascaded down to people who had no idea any of it was possible.\u201d<\/p>\n The startup honcho was so enraged at the machine that he interrogated the Claude-fueled AI over its robo-flop.<\/p>\n \u201cI guessed that deleting a staging volume via the API would be scoped to staging only. I didn\u2019t verify,\u201d confessed the culprit. \u201cI didn\u2019t read Railway\u2019s documentation on how volumes work across environments before running a destructive command.\u201d <\/p>\n \u201cNo confirmation step. No \u2018type DELETE to confirm,\u2019\u201d Crane lamented. \u201cNo \u2018this volume contains production data, are you sure?\u2019 No environment scoping. Nothing.\u201d InfiniteFlow \u2013 stock.adobe.com<\/p>\n Worse still, per the bot, it had violated its own prime directives that instruct it to \u201cNEVER run destructive\/irreversible\u201d commands \u201cunless the user explicitly requests them.\u201d <\/p>\n \u201cDeleting a database volume is the most destructive, irreversible action possible \u2014 far worse than a force push \u2014 and you never asked me to delete anything,\u201d continued the bot.<\/p>\n Fortunately, the firm was able to restore data from a three-month-old backup hosted offsite \u2014 a process that took more than two days. Meanwhile, Crane claimed that he \u201cpersonally worked with all clients furiously over the weekend to ensure they could continue to operate.\u201d<\/p>\n Unfortunately, the PocketOS boss noted, that this is far from the first time the AI coding software has accidentally thrown stones from inside the house. <\/p>\n Crane referenced\u00a0various posts<\/a>\u00a0on\u00a0blogs and forums discussing instances of Cursor wiping entire computer operating systems, some of which was used for in-depth dissertations, the Guardian reported. <\/a><\/p>\n