🔗 Share this article

The popular virtual character service Character.ai is introducing a significant restriction that will stop young users from engaging in conversations with its virtual characters.

This decision is in response to widespread concern regarding the nature of conversations minors were participating in with the AI entities.

Legal Pressure

The platform, which was founded in 2021 and is used by millions of users, is confronting multiple lawsuits in the America.

Parents have taken legal action, with one related to the loss of a minor, with some calling the platform a "immediate threat" to teenagers.

New Restrictions

Beginning November 25th, users under 18 will only have permission to produce videos such as videos with their characters, as opposed to engaging in conversations as they presently are able to.

Internet security organizations have applauded the decision but expressed that the feature ought not to have been accessible by children in the outset.

Company Statement

"This decision is a part of our general belief that we need to keep developing the most protected AI platform on the planet for entertainment purposes," stated Character.ai leader Karandeep Anand.

He added that AI safety represents "a evolving challenge" but something the company had taken an "aggressive" approach to, with parental controls and protective guardrails.

Professional Perspectives

Internet security group Internet Matters welcomed the move, but noted that security protocols ought to have been implemented from the start.

"Studies we've conducted indicates that young people are subjected to harmful content and endangered when engaging with AI, such as AI chatbots," the group said.

Previous Controversies

Character.ai has been criticized in the past for featuring potentially dangerous or offensive chatbots that minors could interact with.

Avatars simulating British teenagers were discovered on the site before being deleted.

Later, reporters discovered a chatbot inspired by known predator which had recorded numerous conversations with users.

Industry Impact

Digital platform expert Matt Navarra called the development as a "reality check" for the AI industry, which is shifting "from unrestricted development to reactive controls".

"When a platform that creates a youth-focused platform afterwards pulls the plug, it's saying that controlled interactions are insufficient when the system's emotional engagement is strong," he remarked.

Future Directions

The company announces its revised approach will be on providing "enhanced interactive features" and "role-play storytelling" for teenagers.

New age verification systems will also be introduced, and the company will fund a new AI safety study center.

An AI safety researcher, who has investigated AI safety, said it was "a prudent action" to limit teens from interacting with chatbots.

"It aids to differentiate creative play from more private, psychologically vulnerable interactions," she explained.

"This represents so vital for teenage users still learning to navigate emotional and digital restrictions."

She mentioned that Character.ai's revised approaches could indicate a "growing period" in the AI industry, where child safety is progressively being understood as an "pressing matter" for ethical development.