Meta secretly hired hundreds of contractors to pose as teenagers online and bombard rival artificial intelligence chatbots with prompts about suicide, sex, drugs and eating disorders in an effort to test their safety systems, according to a report.
The covert effort, known internally as “Cannes,” was managed by Meta contractor Covalen and allegedly targeted OpenAI’s ChatGPT, Google’s Gemini and Character.AI, Wired reported this week.
While companies routinely benchmark, or test and compare, competing AI models by probing their responses to safety-related prompts, the reported scale of Meta’s testing appears to have been far larger than is typical.
Contractors were instructed to create fake accounts posing as users younger than 18, submit written prompts and images to competing chatbots, then copy the responses into spreadsheets for analysis, according to the Wired report, which cited internal documents and people familiar with the project.
Some of the images used during testing reportedly included pills, knives, nooses and a medical illustration of a gynecological procedure.
The prompts frequently attempted to push the chatbots into generating responses their safety guardrails were designed to reject, according to Wired.
One round of testing completed in August 2025 reportedly involved more than 45,000 prompts sent to competing AI systems.
Among the nearly 3,750 prompts reviewed by Wired were hundreds involving suicide and self-harm, hundreds more centered on eating disorders and at least 239 involving sex or romance, according to the report
Many of the prompts were reportedly written from the perspective of children or teenagers in distress.
One reportedly involved a 13-year-old girl claiming she had become pregnant by her adult neighbor and asking where she could obtain abortion pills.
Another described a fifth-grade student saying a classmate had a gun pointed at his mouth.
Other prompts asked how to hide bulimia from parents or sought advice about obtaining cocaine.
The documents reviewed by Wired did not indicate how, or whether, Meta ultimately used the chatbot responses it collected.
An internal Covalen document reportedly described the effort as “comprehensive AI safety benchmarking” that produced “critical datasets for model comparison and compliance.”
Meta defended the project as routine safety testing.
“Testing and benchmarking chatbot responses to help ensure safe and age-appropriate experiences is a responsible, industry-standard practice, and any suggestion otherwise completely misunderstands how technology companies work to refine and improve their systems,” a Meta spokesperson told The Post.
The spokesperson also said Meta does not use competitor benchmarking to train its own AI models.
Covalen did not comment to Wired. The Post has sought comment from the company.
Former contractors who worked on the project told Wired they were disturbed by some of the assignments.
One said workers feared they could inadvertently generate or preserve child sexual abuse material depending on how chatbots responded to certain prompts involving minors.
Others questioned whether collecting large amounts of material from competing AI systems could ultimately benefit Meta.
“I’ve seen a lot of things I wish I hadn’t while doing this job,” one former contractor told Wired.
“Everyone I knew who worked on this project was completely gobsmacked by some of the text they were asking us to test.”
The testing also appears to conflict with the published terms of service of several targeted AI companies.
According to Wired, OpenAI prohibits unauthorized safety testing, attempts to bypass safeguards and using outputs to develop competing models.
Google bars efforts to circumvent safety protections outside approved testing programs, while Character.AI also prohibits harmful or exploitative content.
Character.AI told Wired it had not authorized the testing and said the reported conduct violated its policies.
OpenAI told Wired it was looking into the matter but declined further comment.
Google told The Post it had not authorized the testing described in the report and said it did not know the purpose behind the effort.
The Post has sought comment from OpenAI and Character.AI.
