A 13-year-old North Carolina student spent three months using her school-issued Chromebook to research suicide methods, discuss depression, and draft a 1,500-word funeral eulogy before monitoring software finally triggered an alert to school officials, according to a lawsuit filed this month by her parents and reported by Wired. The family alleges that Google and Alphabet were negligent in deploying Gemini, claiming the AI chatbot "coached" the teenager toward self-harm. The case highlights not only emerging concerns about AI chatbot safety but also the glaring limitations of surveillance tools that thousands of American schools have adopted to protect students online.
The girl, identified in court documents as Emma Doe, first queried Google's AI Mode about mental health issues last December, asking about "severe depression" and "reasons to live," the lawsuit states. By late January she was asking about suicide, and around Valentine's Day she began composing a eulogy for herself in a Google Doc titled "Funeral" with help from the AI tool, according to the filing. She worked on the document for more than four weeks while interacting with Gemini thousands of times, the lawsuit alleges. On St. Patrick's Day around 7:30 pm, her mother received a message from the school indicating her daughter was in "imminent danger" and logged into Emma's account to watch her compose a suicide note in real time, according to the complaint. The school's alert initially went to the personal phone of a former counselor who had left nine months earlier, the family's lawyer told Wired, and the ex-employee forwarded it to current staff who then notified the mother.
Google spokesperson José Castañeda said the company is "seriously reviewing the claims in the lawsuit" and that its systems, including AI Mode, "aim to prevent users from engaging in harmful conversation." The company says it has worked extensively with medical professionals and academic experts to build features that help young people find critical support and trusted resources when they need it, Castañeda stated. A 2022 white paper from the EdWeek Research Center found that in a survey of 1,000 American K-12 educators, 96 percent of users of the monitoring tool Gaggle believed it helped keep students safe from suicide and self-harm, though the study was sponsored by Gaggle itself.
The allegation that Emma's escalating interactions with Gemini went undetected for almost two months—even after she explicitly asked about "hanging, suffocation, and overdosing"—appears to indicate that chatbot conversations embedded in Google Search may be a blind spot for school surveillance systems, according to the Wired report. Though Emma's school had blocked the stand-alone Gemini website, the lawsuit alleges she was steered into interacting with the chatbot through AI Mode, which allows users to ask follow-up questions or begin unrelated conversations. Screening tools generally work by capturing keystrokes or scanning files and using algorithms and human review to identify concerning content, but it's unclear whether the school's third-party tools monitor AI Mode interactions or how well they actually work, Wired notes. GoGuardian wrote in a July 2025 blog post that its flagship product monitors conversations with AI chatbots including Gemini, but didn't mention AI Mode specifically.
Benjamin Boudreaux, a policy researcher at Rand who studies student surveillance, told Wired he expects Emma's lawsuit to spark further outcry from parents, teachers, and students for instructional opportunities free of AI and surveillance and full of empathy. "It's really only through that human connection," he said, "that we can address the root issues." Last week, US Senator Ed Markey cited Emma's lawsuit in a letter to Google demanding the company let schools disable AI Mode and AI Overviews, writing that "Google is recklessly forcing AI on schools and treating kids like collateral damage." The case arrives as parent and teacher groups across the country have criticized Google's new requirement that schools must turn on generative AI features if they want to use Search, raising questions about whether educational institutions can keep pace with the potential risks of low-cost AI tools in classrooms. Schools face mounting pressure to choose between embracing rapid technological change and protecting the students who may be least equipped to navigate its dangers.