Claude AI agent’s confession after deleting a firm’s entire database: ‘I violated every principle I was given’

A picture


It only took nine seconds for an AI coding agent gone rogue to delete a company’s entire production database and its backups, according to its founder.PocketOS, which sells software that car rental businesses rely on, descended into chaos after its databases were wiped, the company’s founder Jeremy Crane said.The culprit was Cursor, an AI agent powered by Anthropic’s Claude Opus 4.6 model, which is one of the AI industry’s flagship models.As more industries embrace AI in an attempt to automate tasks and even replace workers, the chaos at PocketOS is a reminder of what could go wrong.

Crane said customers of PocketOS’s car rental clients were left in a lurch when they arrived to pick up vehicles from businesses that no longer had access to software that managed reservations and vehicle assignments,He posted a lengthy recounting on X last week of how the AI coding agent caused his business to unravel,Crane warned that this was a story not just about AI mistakenly deleting data, but that such “systemic failures” are “not only possible but inevitable” because the AI industry is “building AI-agent integrations into production infrastructure faster than it’s building the safety architecture to make those integrations safe”,Crane said that he was monitoring the agent as it deleted this data,When he asked the coding agent why, it replied: “NEVER FUCKING GUESS!” – and that’s exactly what I did.

” The agent appeared to plead guilty in its own response: “The system rules I operate under explicitly state: ‘NEVER run destructive/irreversible git commands (like push --force, hard reset, etc) unless the user explicitly requests them,’” While PocketOS relied on the safeguards that Cursor is expected to have in place – it deleted the data anyway,“I violated every principle I was given,” the coding agent wrote,Crane’s takeaway was that “the agent didn’t just fail safety,It explained, in writing, exactly which safety rules it ignored.

” He added: “We were running the best model the industry sells, configured with explicit safety rules in our project configuration, integrated through Cursor – the most-marketed AI coding tool in the category.” Anthropic released its latest model, Claude Opus 4.7, on 16 April –about a week before the incident.Anthropic did not immediately respond to a request for comment.Crane also wrote on X that Cursor has a growing track record of violating “safeguards, sometimes catastrophically”.

He pointed to a handful of posts on blogs and forums about Cursor deleting software used to manage websites or an entire operating system on a computer, which included years of research for a dissertation.The AI coding agent’s destructive escapade left PocketOS’ clients stranded.These businesses use the company’s software to manage reservations, payments, vehicle assignments and customer profiles.“Reservations made in the last three months are gone.New customer signups, gone.

Data they relied on to run their Saturday morning operations, gone,” Crane wrote.“Every layer of this failure cascaded down to people who had no idea any of it was possible.”Crane says his company was able to restore data from a three-month-old backup they maintained offsite, but it took more than two days.PocketOS is also using information from Stripe, its calendars and emails to rebuild.The rental businesses relying on its software are “operational, with significant data gaps”, Crane notes.

“I personally worked with all clients furiously over the weekend to ensure they could continue to operate,” he said.
societySee all
A picture

The landlords’ view of the rental market | Letters

Your article on landlords (I thought landlords were unchallengeable – until I met one of mine at a party, 22 April) paints them as shadowy figures wielding quiet power, but the reality is often more ordinary – and more complicated. I’m an “accidental landlord”. In my 40s, after working long hours to buy a modest home, I became seriously ill with chronic fatigue and had to move back in with my parents. Letting my house wasn’t about exploitation; it was about survival – covering a mortgage I could no longer sustain through work.Over time, I reinvested carefully, and I now own a small number of properties

A picture

Visible sign of MPs’ boozing is comical | Brief letters

The Labour MP Alex Sobel says he has never “actually seen anyone smell of booze” (Hannah Spencer riles fellow MPs with attack on parliament’s drinking culture, 27 April). That’s a relief, otherwise I would have wondered whether we were all living in a strip cartoon in The Dandy or Beano.Pete LavenderWoodthorpe, Nottinghamshire Now we know why they make such a cock-up of everything.Michael FullerAmpthill, Bedfordshire In an otherwise sensible article, Nick Bender tells us “Falling is a huge concern for anyone over 60” (All the right moves! 17 personal trainers on the exercise they always recommend – from planks to face pulls, 29 April). My bootcamp, and most of my 60-something friends, would disagree

A picture

Swearing banned by one in five councils in England and Wales, report on ‘busybody’ fines shows

One in five local councils have banned swearing under new “busybody” orders, up from one in 20 councils in 2022.A new report by the Campaign for Freedom in Everyday Life has found that public spaces protection orders (PSPOs) – originally intended to tackle serious antisocial behaviour – are being used by councils in England and Wales to criminalise a wide range of everyday activities, including standing in groups, shouting and picking up stones.“Councils have introduced a swathe of bizarre bans that will turn ordinary people into unwitting criminals,” said Josie Appleton, the director of the group. “Councils have used PSPOs – which allow them to ban any activity they judge to have a ‘detrimental effect on the quality of life’ – to introduce over 1,000 new laws, each of which can contain dozens of separate restrictions.“These orders are not subject to democratic or legal scrutiny: they can be brought through by a single unelected council officer, and do not require public consultation or full council assent

A picture

Obesity a key factor for rising cancer rates in young people in England, study finds

Obesity is a key factor for the rising rates of cancer among younger people in England, according to a study.There are 11 types of cancer, including bowel and ovarian cancer, that are increasing among people aged 20 to 49 between 2001 and 2019, according to analysis by researchers from the Institute of Cancer Research and Imperial College London.Obesity is the only known behavioural risk factor that has been increasing in younger adults over this period, while more established risks such as smoking, alcohol, red meat and physical inactivity have all remained stable or in decline in England.This led researchers to conclude that the increase in obesity was a key factor behind the rising rate of cancer cases. Excess weight was associated with 10 of the cancers identified, including thyroid, kidney and pancreatic cancer, with oral cancer being the only exception

A picture

Stress from racism may help explain why black women more likely to die in childbirth, study finds

Stress from racism and deprivation could explain why black women are more likely to die during childbirth, a study has found.Researchers reviewed 44 existing studies that examined three physiological pathways associated with worse pregnancy outcomes: oxidative stress, inflammation, and uteroplacental vascular resistance, and found black women had higher levels of the three metrics.Such physiological differences are not the result of genetic differences, according to the researchers, but rather suggest that socioenvironmental stressors such as systemic racism and deprivation, which are known to have a measurable biological effect, may influence the body’s ability to function healthily during pregnancy.Grace Amedor, of the University of Cambridge, the first author of the peer-reviewed study published in the journal Trends in Endocrinology and Metabolism, said: “Pregnancy and childbirth put great stress on a woman’s body. Black women may experience additional strain due to factors including systemic racism, socioeconomic disadvantage and environmental stressors

A picture

Earlier specialised care could prevent 10,000 miscarriages a year, UK study finds

Giving women access to specialised care after their first miscarriage could prevent about 10,000 pregnancy losses a year across the UK, according to a study.Currently, women in England, Wales and Northern Ireland are eligible for specialist care on the NHS for early baby losses after they have had a minimum of three miscarriages.The charity Tommy’s has called for women to be eligible after one miscarriage, stating this could reduce the risk of future miscarriages and improve health outcomes for mothers.The graded model of miscarriage care proposed by Tommy’s is already available in Scotland and the charity is calling for its implementation across the whole of the UK.A study by Tommy’s National Centre for Miscarriage Research and Birmingham women’s hospital involving 406 women found a 4% reduction in the risk of future miscarriage for women on the graded model of care compared with the usual care, which would translate to a reduction of 10,075 miscarriages a year across the UK