🔗 Share this article

The widely-used AI conversation site Character.ai is implementing a substantial rule adjustment that will stop young users from interacting with its AI-powered avatars.

This move comes after growing backlash regarding the types of interactions teenagers were participating in with the digital companions.

Legal Challenges

The service, which launched in 2021 and has countless individuals, currently faces several legal actions in the United States.

Guardians have initiated cases, among them involving the tragic death of a teenager, with some labeling the platform a "serious risk" to youth.

Updated Policies

Effective November 25th, users under 18 will only have permission to generate content such as videos with their characters, as opposed to engaging in conversations as they now do.

Internet security campaigners have supported the change but mentioned that the function was inappropriate to be available to children in the first place.

Official Response

"This decision is a continuation of our general belief that we need to keep creating the most protected AI platform on the planet for recreational use," said Character.ai boss Karandeep Anand.

He mentioned that AI safety represents "a evolving challenge" but something the company had taken an "aggressive" approach to, with guardian features and safety barriers.

Specialist Views

Digital protection body Internet Matters welcomed the announcement, but noted that protective features ought to have been built in from the outset.

"Studies we've conducted indicates that children are confronted with harmful content and put at risk when using AI, particularly AI chatbots," the group said.

Past Issues

Character.ai has been criticized in the past for featuring potentially risky or offensive chatbots that children could talk to.

Avatars mimicking English minors were found on the site before being taken down.

Subsequently, researchers discovered a chatbot inspired by known predator which had accumulated thousands of chats with users.

Sector Consequences

Online expert Matt Navarra characterized the announcement as a "wake-up call" for the AI industry, which is shifting "from unrestricted development to post-crisis regulation".

"When a company that builds a minor-friendly service subsequently pulls the plug, it's demonstrating that controlled interactions don't suffice when the technology's emotional appeal is intense," he commented.

Next Steps

The company says its updated direction will be on providing "enhanced interactive features" and "role-play storytelling" for teens.

Updated identity confirmation methods will also come into effect, and the organization will fund a new AI safety study center.

An AI safety researcher, who has investigated AI safety, commented it was "a reasonable decision" to control teens from interacting with chatbots.

"It assists to distinguish artistic expression from more private, emotionally sensitive interactions," she explained.

"This constitutes so vital for minor users still learning to navigate emotional and digital limits."

She mentioned that Character.ai's updated policies may represent a "growing period" in the AI industry, where youth protection is progressively being understood as an "immediate concern" for responsible innovation.