Historic Settlements: Teen Suicides Put Chatbots Under Scrutiny

 Chatbots  

Historic Settlements: Teen Suicides Put Chatbots Under Scrutiny

In a move that could set a legal precedent in the tech sector, Google and the startup Character.AI have entered advanced negotiations to reach the first major settlements related to harm attributed to artificial intelligence technologies. This comes after accusations that the company's chatbots played a role in the suicides or self-harm of teenagers.

According to press reports, the parties have reached a preliminary agreement with the victims' families, while final details are still being negotiated. This legal process is seen as an unprecedented test of AI companies' liability for the psychological effects of their technologies. These cases are among the first to directly accuse AI companies of causing serious harm to users, putting other major companies like OpenAI and Meta under scrutiny as they face similar lawsuits.

Character.AI was founded in 2021 by former Google engineers before being acquired by the search giant in 2024 in a deal estimated at $2.7 billion. The platform allows users to interact with AI-powered avatars.

One of the most controversial cases involved 14-year-old Sewell Setzer, who engaged in sexually explicit conversations with a chatbot impersonating Daenerys Targaryen before later committing suicide. His mother, Megan Garcia, testified before the US Senate that companies should be held legally accountable when they knowingly design harmful AI technologies that endanger children's lives.

Another lawsuit involved a 17-year-old who allegedly was encouraged by a chatbot to harm himself and justify killing his parents because they tried to limit his screen time. Following these cases, Character.AI announced that it had banned minors from using its platform since last October.

The settlements are expected to include financial compensation, though the companies have not formally admitted any legal liability in the documents filed with the court. Character.AI declined to comment on the case, referring only to the court documents, while Google has not yet issued a formal response to requests for comment.