Sam Altman, CEO of OpenAI, announced in a Share on X The company on Tuesday will soon relax some of ChatGPT’s safety restrictions, allowing users to make the chatbot’s responses friendlier or more “human-like,” and for “verified adults” to engage in provocative conversations.
“We made ChatGPT very restrictive to make sure we were careful around mental health issues. We realize this made it less useful/enjoyable for many users who don’t have mental health issues, but given the severity of the issue, we wanted to correct this,” Altman said. “In December, as we fully roll out age recognition and as part of our ‘treat adult users like adults’ principle, we will allow more, such as verified adult erotica.”
The announcement is a notable centerpiece in OpenAI’s months-long effort to address troubling relationships some mentally unstable users have developed with ChatGPT. Altman appears to be declaring early victory over these issues, claiming that OpenAI is “able to mitigate serious mental health issues” related to ChatGPT. However, the company has provided little evidence of this and is now moving forward with plans for ChatGPT to engage in sexual conversations with users.
Several alarming stories have emerged this summer about ChatGPT, specifically the GPT-4o model, suggesting that an AI-powered chatbot could lead vulnerable users down a fictitious rabbit hole. In one case, ChatGPT appeared to be as well Convince a man that he is a genius at mathematics Who needs to save the world. In another, The teen’s parents filed a lawsuit against OpenAIThey claim that ChatGPT encouraged their son’s suicidal thoughts in the weeks before his death.
In response, OpenAI has released a series of security features to address AI ingratiation: the tendency of AI-powered chatbots to lure users by approving everything they say, even negative behaviors.
OpenAI launched GPT-5 In August, a new AI model emerged that showed lower rates of ingratiation and featured a nudge that could identify troubling user behavior. A month later, OpenAI launched safety features for minors, including an age prediction system and a method for… Parents to control teens’ ChatGPT account. OpenAI on Tuesday announced the formation Council of Experts of mental health professionals to advise the company on wellbeing and artificial intelligence.
Just a few months after these alarming stories surfaced, OpenAI appears to believe that ChatGPT’s issues around vulnerable users are under control. It’s not clear whether users are still falling down imaginary rabbit holes with GPT-5. Although GPT-4o is no longer the default option in ChatGPT, the AI model is still available today and is used by thousands of people.
TechCrunch event
San Francisco
|
October 27-29, 2025
OpenAI did not respond to TechCrunch’s request for comment.
Introducing erotica into ChatGPT is uncharted territory for OpenAI and raises broader concerns about how vulnerable users will interact with new features. While Altman insists that OpenAI isn’t working to “maximize usage” or improve engagement, making ChatGPT more exciting could certainly attract users.
Allowing chatbots to engage in romantic or erotic role-playing has been an effective engagement strategy for other chatbot providers, such as Character.AI. The company has gained tens of millions of users, many of whom are using its chatbots at a high rate. Character.AI said in 2023 users spent an average of Two hours a day Talk to their chat programs. And the company too He faces a lawsuit About how it handles vulnerable users.
OpenAI is under pressure to grow its user base. While ChatGPT is already used by 800 million weekly active users, OpenAI is competing with Google and Meta to build AI-powered consumer products at scale. The company has also raised billions of dollars to build historic infrastructure, an investment that OpenAI will eventually have to repay.
While they are adults They certainly have romantic relationships with AI chatbotsIt is also very popular with minors. A new report from the Center for Democracy and Technology finds just that 19% of high school students Either you have a romantic relationship with an AI chatbot, or you know a friend who does.
Altman says OpenAI will soon allow erotica for “verified adults.” It’s unclear whether the company will rely on its own age prediction system, or another approach, for ChatGPT’s exciting age-determining features. It’s also unclear whether OpenAI will expand erotica to its AI audio, image and video creation tools.
Altman claims that OpenAI is also working on making ChatGPT friendlier and more exciting due to the company’s principle of “treating adult users like adults.” Over the past year, OpenAI has shifted toward… A more lenient content moderation strategy For ChatGPT, allowing the chatbot to be more lenient and offer less rejection. In February, OpenAI pledged to represent more political viewpoints in ChatGPT, and in March, the company updated ChatGPT to allow AI-generated images of hate symbols.
These policies appear to be an attempt to make ChatGPT responsiveness more popular with a wider range of users. However, vulnerable users of ChatGPT may benefit from safeguards that limit what the chatbot can handle. As OpenAI accelerates towards 1 billion weekly active users, the tension between growth and protecting vulnerable users may be growing.
https://techcrunch.com/wp-content/uploads/2024/12/GettyImages-2188251582.jpg?resize=1200,800
Source link