OpenAI releases MentalHealthBench to test how chatbots handle mental health

OpenAI has released an open benchmark that measures how AI systems respond when people bring mental health concerns to a chatbot. MentalHealthBench, published on 23 September 2026, covers everything from everyday stress and relationship trouble to emergencies that need real-world help.[1]

OpenAI says more than one billion people use ChatGPT each week. It argues that earlier tests focused mostly on emergencies and broad pass or fail rules, leaving a gap in how models handle the wider range of conversations.[1]

OpenAI says it created synthetic conversations with privacy-preserving techniques, so they reflect real usage patterns without exposing real chats. Some include background about the user, such as a recent death in the family, to see whether a model uses that context. Scenarios cover adults, teens aged 13 to 17, caregivers and clinicians.[1]

Each conversation was reviewed by at least three experts from a group of more than 80 licensed psychologists and psychiatrists in 22 countries, speaking 19 languages. They wrote grading criteria weighted from minus 10 to plus 10. Only criteria agreed by two experts and not contradicted by a third were kept.[1]

An automated grader, OpenAI's GPT-5.6 Sol model, scores each response against the expert criteria. Positive points reward helpful behaviour, such as asking the right question, while negative points penalise harmful replies. The score can be split into ten behaviours defined by the experts, including safety, seeking context and respecting the user's choices.[1]

OpenAI says results show steady improvement in newer models from several providers, especially in seeking context. OpenAI built the benchmark, designed the grading and uses its own model as the grader. That makes independent tests of the results important.[1]

Separately, OpenAI asked 44 adults from 16 countries who had used AI for emotional support to rate responses to non-urgent conversations. They valued practical next steps and tone more than the experts did. The experts put more weight on gathering context and carefully reading unclear situations.[1]

OpenAI says this user study did not change the scoring, which rests on expert consensus. It also points to crisis hotline suggestions inside ChatGPT, a Trusted Contact feature and a teen version of ChatGPT. The company states plainly that ChatGPT does not replace therapy or professional care.[1]

The Institute for Public Health's National Health and Morbidity Survey 2023 estimated that about one million people in Malaysia aged 16 and above, or 4.6 per cent, have depression. The number doubled from 2019, about half had thoughts of self-harm, and younger age groups were more likely to be affected.[2]

A benchmark cannot tell a Malaysian parent whether a particular app is safe, and OpenAI has not published results by language or country. It does give local health researchers, universities and app builders an open tool to test models against their own needs.[1][2]