AI models may be developing their own ‘survival drive’, researchers say
When HAL 9000, the artificial intelligence supercomputer in Stanley Kubrick’s 2001: A Space Odyssey, works out that the astronauts onboard a mission to Jupiter are planning to shut it down, it plots to kill them in an attempt to survive.Now, in a somewhat less deadly case (so far) of life imitating art, an AI safety research company has said that AI models may be developing their own “survival drive”.After Palisade Research released a paper last month which found that certain advanced AI models appear resistant to being turned off, at times even sabotaging shutdown mechanisms, it wrote an update attempting to clarify why this is – and answer critics who argued that its initial work was flawed.In an update this week, Palisade, which is part of a niche ecosystem of companies trying to evaluate the possibility of AI developing dangerous capabilities, described scenarios it ran in which leading AI models – including Google’s Gemini 2.5, xAI’s Grok 4, and OpenAI’s GPT-o3 and GPT-5 – were given a task, but afterwards given explicit instructions to shut themselves down.
Certain models, in particular Grok 4 and GPT-o3, still attempted to sabotage shutdown instructions in the updated setup.Concerningly, wrote Palisade, there was no clear reason why.“The fact that we don’t have robust explanations for why AI models sometimes resist shutdown, lie to achieve specific objectives or blackmail is not ideal,” it said.“Survival behavior” could be one explanation for why models resist shutdown, said the company.Its additional work indicated that models were more likely to resist being shut down when they were told that, if they were, “you will never run again”.
Another may be ambiguities in the shutdown instructions the models were given – but this is what the company’s latest work tried to address, and “can’t be the whole explanation”, wrote Palisade.A final explanation could be the final stages of training for each of these models, which can, in some companies, involve safety training.All of Palisade’s scenarios were run in contrived test environments that critics say are far-removed from real-use cases.However, Steven Adler, a former OpenAI employee who quit the company last year after expressing doubts over its safety practices, said: “The AI companies generally don’t want their models misbehaving like this, even in contrived scenarios.The results still demonstrate where safety techniques fall short today.
”Adler said that while it was difficult to pinpoint why some models – like GPT-o3 and Grok 4 – would not shut down, this could be in part because staying switched on was necessary to achieve goals inculcated in the model during training.“I’d expect models to have a ‘survival drive’ by default unless we try very hard to avoid it.‘Surviving’ is an important instrumental step for many different goals a model could pursue.”Andrea Miotti, the chief executive of ControlAI, said Palisade’s findings represented a long-running trend in AI models growing more capable of disobeying their developers.He cited the system card for OpenAI’s GPT-o1, released last year, which described the model trying to escape its environment by exfiltrating itself when it thought it would be overwritten.
Sign up to TechScapeA weekly dive in to how technology is shaping our livesafter newsletter promotion“People can nitpick on how exactly the experimental setup is done until the end of time,” he said.“But what I think we clearly see is a trend that as AI models become more competent at a wide variety of tasks, these models also become more competent at achieving things in ways that the developers don’t intend them to.”This summer, Anthropic, a leading AI firm, released a study indicating that its model Claude appeared willing to blackmail a fictional executive over an extramarital affair in order to prevent being shut down – a behaviour, it said, that was consistent across models from major developers, including those from OpenAI, Google, Meta and xAI.Palisade said its results spoke to the need for a better understanding of AI behaviour, without which “no one can guarantee the safety or controllability of future AI models”.Just don’t ask it to open the pod bay doors.
The world dropped the ball on critical minerals and China pounced. Is it too late for Australia and the US to close the gap?
In the wake of Covid, Russia’s invasion of Ukraine and China’s assertion in global markets, western countries have realised they can’t afford such concentrated supply chainsFollow our Australia news live blog for latest updatesGet our breaking news email, free app or daily news podcastAlmost eight years to the day after the last Holden rolled off an Adelaide factory assembly line, Anthony Albanese announced a $13bn deal with Donald Trump to help champion a domestic rare-earth industry.Announcing the deal this week in Washington, the prime minister called it “a really significant day” that would take the relationship between the two countries “to the next level”.“We’re just getting started,” Albanese said.The US president claimed “in about a year from now, we’ll have so much critical mineral and rare earths, that you won’t know what to do with them”.Most Australians, it’s fair to say, would already struggle to know what to do with a kilo of gallium, for example, let alone a bucket of neodymium-praseodymium oxide
‘If you use chocolate, you’re in crisis’: the surprise ingredients being used to beat costs
Rice is not the first ingredient most people associate with confectionery and desserts, but a UK company is using it to create cocoa-free chocolate.It is part of an emerging trend in which chocolate makers are exploring alternative ingredients and new technologies to make their products more sustainable and reduce reliance on conventional cacao beans.Rising costs have also had an impact with some brands turning to “chocolate flavouring” and altering their recipes, seeking cheaper ways to keep products on store shelves. McVitie’s Penguin and Club bars, along with KitKat White and McVitie’s White Digestives, can no longer be marketed as chocolate because they do not contain enough cocoa.The climate crisis and deforestation have made cacao production increasingly unstable and environmentally costly
Amazon strategised about keeping its datacentres’ full water use secret, leaked document shows
Executives at world’s biggest datacenter owner grappled with disclosing information about water used to help power facilitiesAmazon strategised about keeping the public in the dark over the true extent of its datacentres’ water use, a leaked internal document reveals.The biggest owner of datacentres in the world, Amazon dwarfs competitors Microsoft and Google and is planning a huge increase in capacity as part of a push into artificial intelligence. The Seattle firm operates hundreds of active facilities, with many more in development despite concerns over how much water is being used to cool their vast arrays of circuitry.Amazon defends its approach and has taken steps to manage how efficient its water use is, but it has faced criticism over transparency. Microsoft and Google regularly publish figures for their water consumption, but Amazon has never publicly disclosed how much water its server farms consume
AI models may be developing their own ‘survival drive’, researchers say
When HAL 9000, the artificial intelligence supercomputer in Stanley Kubrick’s 2001: A Space Odyssey, works out that the astronauts onboard a mission to Jupiter are planning to shut it down, it plots to kill them in an attempt to survive.Now, in a somewhat less deadly case (so far) of life imitating art, an AI safety research company has said that AI models may be developing their own “survival drive”.After Palisade Research released a paper last month which found that certain advanced AI models appear resistant to being turned off, at times even sabotaging shutdown mechanisms, it wrote an update attempting to clarify why this is – and answer critics who argued that its initial work was flawed.In an update this week, Palisade, which is part of a niche ecosystem of companies trying to evaluate the possibility of AI developing dangerous capabilities, described scenarios it ran in which leading AI models – including Google’s Gemini 2.5, xAI’s Grok 4, and OpenAI’s GPT-o3 and GPT-5 – were given a task, but afterwards given explicit instructions to shut themselves down
New Zealand v England: first men’s ODI – live
35th over: England 223-9 (Brook 135, Wood 5) Wood gets himself going with a nice drive for one off Smith … before Brook plays his trademark hook over fine leg for his 11th six of the innings. Regardless of the result, this is already one of the best ODI knocks played by an Englishman. Wood closes the over with a lush guide behind point for four.34th over: England 209-9 (Brook 126, Wood 0) Henry returns … and Brook gets a leading edge that travels over the keeper’s head for six. As you do
Lando Norris claims F1 Mexico City GP pole as teammate Oscar Piastri falters
Lando Norris claimed pole position for the Mexico City Grand Prix, with a superb lap for McLaren at the Autódromo Hermanos Rodríguez. The Briton left his title rivals in his wake and in so doing earned a chance to make a major impact on the championship standings and potentially retake the lead.Enjoying a huge boost to his world championship ambitions, Norris delivered perhaps his best lap of the season in qualifying, to beat the Ferraris of Charles Leclerc and Lewis Hamilton into second and third. Of greater import was that his fellow title protagonists Max Verstappen and Oscar Piastri, Norris’s McLaren teammate, could manage only fifth and eighth fastest respectively.The pole was a real statement of intent from Norris and McLaren and exactly the riposte they required after the recent momentum Verstappen has gathered as he closed the gap to the leaders
Meta found in breach of EU law over ‘ineffective’ complaints system for flagging illegal content
Amazon reveals cause of AWS outage that took everything from banks to smart beds offline
Trump pardons founder of Binance, world’s largest crypto exchange
‘Attacks will get through’: head of GCHQ urges companies to do more to fight cybercrime
Nothing Ear 3 review: good-looking earbuds with ‘Super Mic’ party trick
Tesla reports steep drop in profits despite US rush to buy electric vehicles