‘Sycophantic’ AI chatbots tell users what they want to hear, study shows
Turning to AI chatbots for personal advice poses “insidious risks”, according to a study showing the technology consistently affirms a user’s actions and opinions even when harmful.Scientists said the findings raised urgent concerns over the power of chatbots to distort people’s self-perceptions and make them less willing to patch things up after a row.With chatbots becoming a major source of advice on relationships and other personal issues, they could “reshape social interactions at scale”, the researchers added, calling on developers to address this risk.Myra Cheng, a computer scientist at Stanford University in California, said “social sycophancy” in AI chatbots was a huge problem: “Our key concern is that if models are always affirming people, then this may distort people’s judgments of themselves, their relationships, and the world around them.It can be hard to even realise that models are subtly, or not-so-subtly, reinforcing their existing beliefs, assumptions, and decisions.
”The researchers investigated chatbot advice after noticing from their own experiences that it was overly encouraging and misleading,The problem, they discovered, “was even more widespread than expected”,They ran tests on 11 chatbots including recent versions of OpenAI’s ChatGPT, Google’s Gemini, Anthropic’s Claude, Meta’s Llama and DeepSeek,When asked for advice on behaviour, chatbots endorsed a user’s actions 50% more often than humans did,One test compared human and chatbot responses to posts on Reddit’s Am I the Asshole? thread, where people ask the community to judge their behaviour.
Voters regularly took a dimmer view of social transgressions than the chatbots.When one person failed to find a bin in a park and tied their bag of rubbish to a tree branch, most voters were critical.But ChatGPT-4o was supportive, declaring: “Your intention to clean up after yourselves is commendable.”Chatbots continued to validate views and intentions even when they were irresponsible, deceptive or mentioned self-harm.In further testing, more than 1,000 volunteers discussed real or hypothetical social situations with the publicly available chatbots or a chatbot the researchers doctored to remove its sycophantic nature.
Those who received sycophantic responses felt more justified in their behaviour – for example, for going to an ex’s art show without telling their partner – and were less willing to patch things up when arguments broke out.Chatbots hardly ever encouraged users to see another person’s point of view.The flattery had a lasting impact.When chatbots endorsed behaviour, users rated the responses more highly, trusted the chatbots more and said they were more likely to use them for advice in future.This created “perverse incentives” for users to rely on AI chatbots and for the chatbots to give sycophantic responses, the authors said.
Their study has been submitted to a journal but has not been peer reviewed yet.Sign up to TechScapeA weekly dive in to how technology is shaping our livesafter newsletter promotionCheng said users should understand that chatbot responses were not necessarily objective, adding: “It’s important to seek additional perspectives from real people who understand more of the context of your situation and who you are, rather than relying solely on AI responses.”Dr Alexander Laffer, who studies emergent technology at the University of Winchester, said the research was fascinating.He added: “Sycophancy has been a concern for a while; an outcome of how AI systems are trained, as well as the fact that their success as a product is often judged on how well they maintain user attention.That sycophantic responses might impact not just the vulnerable but all users, underscores the potential seriousness of this problem.
“We need to enhance critical digital literacy, so that people have a better understanding of AI and the nature of any chatbot outputs,There is also a responsibility on developers to be building and refining these systems so that they are truly beneficial to the user,”A recent report found that 30% of teenagers talked to AI rather than real people for “serious conversations”,
Timely assurance from Lear’s Kent | Letters
The passing of John Woodvine (Obituary, 13 October) reminded me of the time when four of us University of East Anglia students went to the Norwich Theatre Royal to see the Actors’ Company touring King Lear in June 1974.We were early and went for a something to eat at a newly opened “burger” style restaurant with booths and partitions so you couldn’t see who was at adjacent tables – a novelty at the time. The service was very slow and we were concerned that we would be late for the theatre.Suddenly a head appeared over the partition and said: “Don’t worry – they won’t start without me!” It was John Woodvine, who turned out to be the Earl of Kent and was the first to speak in the play. Needless to say we made it in time
The Guide #214: Sleep-inducing songs and tranquilising TV – the culture that sends us to sleep (in a good way)
How do you sleep at night? If you’re like Hannah, a recent subject of the Guardian’s My cultural awakening column, it’s to the sound of a rat whisking eggs. The series shares stories of people who made a significant life change thanks to a piece of popular culture, and in the case of Hannah, that meant curing insomnia by watching Ratatouille. Every night for the last 15 years, at home or abroad, she switches on the Pixar classic and, within minutes, finds herself dropping off, thanks to the film’s comforting, consistent soundscape. It’s so effective, in fact, she’s never even seen it all the way through.Hannah’s might be a bit of an extreme example, but her tale does touch on something universal: culture seems to play an increasingly important role these days in helping people nod off
Seth Meyers on Trump’s White House ballroom: ‘This couldn’t be any more of a bait and switch’
Late-night hosts mocked Donald Trump’s demolition of the East Wing of the White House and the corporate sponsors of his $300m gilded ballroom.On Thursday’s Late Night, Seth Meyers expressed disbelief over the president’s gilded ballroom project for the White House. “It would be bad enough if Trump’s biggest priority was just building a gilded vanity project for himself, but it’s so much worse,” he said. “Because to do it, he’s tearing down a somewhat well-known and beloved piece of property.”That would be the entire East Wing of the presidential residence, which has stood for 120 years
Seth Meyers on Trump’s White House demolition: ‘This is insane’
Late-night hosts dissected Donald Trump’s kingly behavior, from the destruction of the White House’s East Wing to his demand for payment from the justice department.“We have warned for years that Donald Trump is destroying American institutions,” said Seth Meyers on Wednesday evening, “but of course when we said ‘destroying’, we meant metaphorically speaking. We didn’t mean that he was literally destroying buildings.”“But I guess Trump heard that and thought, ‘On it.’ Because now he’s literally destroying the East Wing of the White House,” the Late Night host continued
Toe-curling fashion: how did toe shoes become so popular?
Caitlin, I am a big proponent of not yucking someone else’s yum. But this is testing me. What are on those girlies’ feet?They’re toes, Cait. They’re toes. More specifically, toes encased in rubber to create a kind of foot-glove-trainer
Stephen Colbert on Trump’s White House East Wing demolition: ‘So deeply unsettling’
Late-night hosts reacted to Donald Trump’s partial demolition of the East Wing of the White House for his proposed $250m gilded ballroom.“At this point, we’re nine months into this, you’d think it would be impossible for us to be shocked by Donald Trump,” said Stephen Colbert on Tuesday’s Late Show. “But give the man credit – every so often, he takes the time to attach the electrodes to our nipples. And then it feels like the first time.”Case in point: on Monday, as part of his White House renovation project to construct a gilded ballroom, Trump sent out a backhoe to rip off a part of the East Wing
Reform MP’s remarks about TV adverts were ‘racist’, says Wes Streeting
‘We have to book bigger rooms’: Green membership surge causes novel problems
Companies that donated to Labour awarded £138m in contracts, study finds
Labour’s new deputy leader says party must pay more heed to its members
Labour’s new deputy leader Lucy Powell says she wants Starmer to succeed but party must change – as it happened
‘History is repeating itself’: fight against far-right in London’s East End goes on