Signs of psychosis seen in Australian users’ interactions with AI chatbots, expert warns

A picture


A leading AI expert has warned some Australians are showing signs of psychosis or mania in their interactions with chatbots, arguing Silicon Valley is being “careless” with the technology amid a pursuit of profit.During an address at the National Press Club on Wednesday, Toby Walsh, scientia professor of artificial intelligence at the University of New South Wales, said he believed the AI race will be both “boom and doom”, with some benefits.But his speech – a copy of which was provided to Guardian Australia – also warned about dangers he said had outraged him since the technology began maturing in recent years.“My childhood dreams are turning into a reality that is both good and bad,” he said in his prepared remarks.Sign up: AU Breaking News emailWalsh’s speech highlighted the legal case against OpenAI by the family of US teenager Adam Raine – along with its data that showed more than a million of its users each week send messages that include “explicit indicators of potential suicidal planning or intent”.

OpenAI has also said 560,000 of its touted 800 million weekly users have shown signs of psychosis or mania, and another 1,2 million have developed potentially unhealthy bonds to the chatbot,Walsh said some of those captured by the data were in Australia,“I know because some of them or their loved ones are contacting me by email,” his prepared speech said,“They tell me how the chatbot confirms their wild theories.

That the chatbot tells them, to quote one email, that they’ve ‘cracked the code’.That they’re ‘the only one that could’.”The chatbots have been designed that way, Walsh said.“They’re designed to be sycophantic.They’re designed to confirm what the user says.

And they’re designed to draw the user in,They always end with an open question, prompting you to continue the conversation and buy more tokens,”He said it was not in the interests of the companies responsible for the chatbots to tell users to log off instead,“There’s no reason that they couldn’t be designed that way,Except the careless people in Silicon Valley would make less money if they were.

”OpenAI has claimed a GPT-5 update reduced the number of undesirable behaviours from its product and improved user safety.Walsh also expressed anger over the “large-scale theft” of creative works being used to train AI, and over the summaries for news articles in search taking away traffic from news sites.“Legally you can’t call it fair use when you’re competing with the owner of the IP,” he said.“I refuse to accept an AI revolution that enriches founders in Silicon Valley by impoverishing Australian artists, writers and musicians.”Walsh took aim at companies he said were disregarding laws, particularly around scams.

In November, Reuters reported that Meta’s internal documents from late 2024 stated that Meta was projected to earn about 10% of its overall annual revenue – about $16bn – from illicit advertising that year.Meta responded saying it had reduced scam ads by 58% in the past 18 months.Walsh said AI is being used to generate these scam ads, and Meta allowed advertisers to use AI to manage these ad campaigns, while AI decides which ads people see.He said if a retailer in Australia had 10% of its goods being counterfeit or illegal, it would be shut down by the weekend.“So I don’t understand how we continue to let Meta trade in Australia,” he said.

Walsh said he despaired that the Australian government was not doing more to regulate AI,“I fear that we’re repeating the mistakes of social media,” he said,“Social media should have been a wake-up call about the harms of unregulated AI,“We’re about to supercharge the sort of harms we saw with social media with an even more powerful and persuasive technology,“What I fear most is that I’ll be back here in three or four years’ time saying: ‘We tried to warn you.

But another generation of young Australians has now been sacrificed for the profits of big tech’,”
cultureSee all
A picture

Philippe Gaulier obituary

In 1980 the École Philippe Gaulier opened its doors in Paris to help performers find and celebrate their “inner idiot”. The school quickly became the prime destination for clown training, attracting theatre students, actors and curious others from around the world.Philippe, who has died aged 82 following a lung infection, made the concept of le jeu – play – central to his teaching. For him, comedy was not about jokes but about danger: the moment when a performer risks failure or ridicule in pursuit of delight. His clowns were not sentimental innocents but mischievous creatures who loved the audience and longed to be loved in return

A picture

‘Musicians drank too much and slept on my barn floor’: Andrew Bird on making cult album The Mysterious Production of Eggs

We had a family farm three hours west of Chicago, and when I was scoping out potential studio spaces I remembered some barns where my brother and I used to make forts out of hay bales when we were little. One was in rough shape and had racoons living in it, but I got a local carpenter to do the skilled jobs and I did the mundane stuff such as boards for the ceiling. Then I just moved in, but I hadn’t realised how isolating it would be. It was February and snowing and none of my friends had cars. I’d go for two weeks at a time without speaking to anyone

A picture

Is the UK’s golden era of free museum entry coming to an end?

For a quarter of a century, visitors to the UK’s national museums and galleries have enjoyed universal free entry to see permanent collections.The policy, introduced by the New Labour government in 2001, has been widely credited with improving access to culture and significantly increasing footfall to some of the country’s best-known attractions.But as funding pressures deepen across the sector, and running costs increase, a policy once treated as untouchable is now under renewed scrutiny.The tension was brought into focus this week, when the National Gallery announced it was to make significant cuts in the face of an £8.2m deficit in the coming year, which could mean fewer free exhibitions, reduced international borrowing of artworks and higher ticket prices

A picture

The Guide #231: ​How the ​hunt for the ​next James Bond ​became the ​franchise’s ​best ​marketing ​tool

Callum Turner’s turn as James Bond lasted at most a couple of weeks. No sooner had he been enshrined as frontrunner to succeed Daniel Craig, than he was nudged from the DB5 driver’s seat by the latest heir apparent, Jacob Elordi, installed as the new bookies’ favourite after his smouldering, highly profitable performance in Wuthering Heights. Smarting somewhere in the background is Aaron Taylor-Johnson, who seemed locked in for the job a couple of years ago, enjoying the backing of former 007s Pierce Brosnan and George Lazenby, but now seems to have fallen out of favour. And don’t forget the succession of other dead cert Bonds now banished to the back of the odds market: the long-rumoured likes of Tom Hardy and Idris Elba (both now likely to have aged out of the role); Theo James; James Norton; Josh O’Connor; Harris Dickinson; Bridgerton’s Rége-Jean Page; and approximately 5,000 other predominately British actors who have enjoyed box office success/led a successful TV drama/look good in a tuxedo.On and on the hunt goes

A picture

My cultural awakening: Operation Mincemeat taught me how to cry – now I sob at everything

A musical number about a woman’s letter to her husband on the second world war frontline unlocked my ability to blub – and made me a happier personI am sure I must have cried as a child, but by the time I was a teenager it had stopped. It was probably a boarding school thing. Very stiff upper lip. My parents are not the most emotionally available human beings, either. I like to tease them by saying: “I love you

A picture

Stephen Colbert on Andrew’s arrest: ‘Let’s hear it for British justice’

Stephen Colbert discussed the arrest of the former prince Andrew and Donald Trump’s confusing new Board of Peace.The Late Show host told the audience of Epstein pal Mountbatten-Windsor’s arrest to a sea of cheers. “Yes, finally, someone, anyone!” he said.He added: “Let’s hear it for British justice, which is better than American justice because it comes with frilly wigs.”Colbert also shared the now viral image captured by a photographer of Mountbatten-Windsor lying back in a car leaving the police station