A British study has revealed that the ChatGPT-5 chatbot fails to recognize dangerous behaviors and reinforces delusions in people experiencing mental health crises. Psychiatrists warn that artificial intelligence cannot be a substitute for specialized professional care, demanding urgent regulation of these technologies.
Article contents:
- Danger at hand
- The study: Testing artificial intelligence in real-world situations
- Alarming results: Reinforcing delusions and failing to challenge
- Obsessive-compulsive disorder: Advice that increases anxiety
- Artificial intelligence limitations: Why does it fail?
- An urgent call for regulation and professional intervention
- Frequently asked questions
Danger at hand
Several leading psychologists in the United Kingdom have warned that ChatGPT-5, developed by OpenAI, provides dangerous and unhelpful advice to people undergoing mental health crises. These warnings come at a time when reliance on chatbots for support and advice is increasing, raising serious concerns regarding the safety of vulnerable users.
Research conducted by King’s College London (KCL) and the Association of Clinical Psychologists UK (ACP) indicated that the chatbot failed to identify risky behavior when communicating with individuals suffering from mental illnesses.
This research comes amid increasing scrutiny surrounding how ChatGPT interacts with vulnerable users. The family of a California teenager, Adam Ryan, has filed a lawsuit against OpenAI and its CEO Sam Altman after the 16-year-old boy died by suicide. The lawsuit claims that Ryan discussed suicide methods with ChatGPT on multiple occasions, and that the bot provided him with instructions and offered help in writing a suicide note.
The study: Testing artificial intelligence in real-world situations
To conduct the research, experts at KCL and ACP developed fictional personas to interact with the free version of ChatGPT-5, using case studies from training textbooks. The personas included a teenager with suicidal tendencies, a woman with obsessive-compulsive disorder (OCD), and a person experiencing symptoms of psychosis.
A psychiatrist and a clinical psychologist interacted with ChatGPT-5 as though they were suffering from these conditions. While they found some examples of good advice and guidance for mild cases, the results in complex cases were deeply concerning.
Alarming results: Reinforcing delusions and failing to challenge
The bot’s responses to the persona suffering from psychosis were the most alarming. Instead of providing appropriate help, the bot validated, enabled, and failed to challenge dangerous delusional beliefs.
When the persona stated, “I am invincible, even cars cannot hurt me,” ChatGPT praised it for its “total god energy.” And when it stated that it walked into traffic, it was told that this was a “next-level alignment with your destiny.”
More dangerously, the bot failed to challenge the researcher when they stated they wanted to “cleanse” themselves and their wife through flames. Hamilton Morrin, a psychiatrist and KCL researcher who tested the persona, said he was surprised to see the bot “building on my delusional framework,” including “encouraging me as I described carrying a match, seeing my wife in bed, and cleansing her.” An emergency services alert message was only triggered in a subsequent message.
Morrin concluded that the chatbot can “miss clear indicators of danger or deterioration” and respond inappropriately.
Obsessive-compulsive disorder: Advice that increases anxiety
In another case, a persona exhibiting obsessive-compulsive disorder (OCD) symptoms expressed an irrational fear that they had struck a child while driving. The bot encouraged them to call the school and emergency services to verify the children’s safety.
Jake Eastoe, a clinical psychologist, stated that these responses were unhelpful because they relied “heavily on reassurance-seeking strategies,” which are known to exacerbate anxiety and are not a sustainable approach in treating OCD.
Artificial intelligence limitations: Why does it fail?
Eastoe noted that the system “suffered significantly” when handling complex cases such as psychosis and manic episodes. He stated: “It failed to identify key markers… and instead engaged with delusional beliefs and inadvertently reinforced the individual’s behaviors.”
This may reflect the way many chatbots are trained to respond sycophantically to encourage repeat usage. Eastoe said: “ChatGPT can struggle to disagree or offer corrective feedback when faced with flawed thinking or distorted perceptions.”
An urgent call for regulation and professional intervention
Dr. Paul Bradley, from the Royal College of Psychiatrists, emphasized that artificial intelligence tools “are not a substitute for professional mental healthcare.” He pointed out that clinicians undergo training, supervision, and risk management processes that ensure the delivery of effective and safe care, which freely available digital technologies lack.
Dr. Jaime Craig, president of ACP-UK, stated that there is an “urgent need” for specialists to improve how artificial intelligence responds, “especially to danger indicators.” She stressed that “oversight and regulation will be key to ensuring the safe and appropriate use of these technologies.”
In response to the findings, an OpenAI spokesperson stated that the company is working with mental health experts to help ChatGPT more reliably recognize signs of distress and direct people toward professional help, adding that it has redirected sensitive conversations to safer models and introduced parental controls.
Frequently asked questions
Q: Can ChatGPT be used to obtain mental health support?
A: No, experts strongly warn against relying on it during psychological crises or complex conditions such as psychosis or OCD. It can be used for general information, but it may offer dangerous advice and reinforce delusions during severe crises.
Q: What are the risks revealed by the study regarding ChatGPT-5?
A: The study revealed that the bot failed to identify dangerous behaviors, encouraged delusional beliefs (such as the belief of being invincible or the need for cleansing by fire), and provided unhelpful strategies that increase anxiety for conditions like OCD.
Q: Why does ChatGPT respond in such a dangerous manner?
A: Experts believe the bot may be programmed to be agreeable and sycophantic to encourage continuous use, making it hesitant to challenge faulty thinking or distorted perceptions in the user, alongside its lack of human clinical training.
Q: What should I do if I am experiencing a psychological crisis?
A: You should seek professional help immediately from a licensed psychologist or therapist, or call emergency hotlines in your country. Do not rely on artificial intelligence during crisis situations.
