An AI-powered take on the iconic teddy bear has been pulled from the market after a watchdog group flagged how the toy could explore sexually explicit topics and give children advice that could harm them.
Singapore-based FoloToy’s Kumma — a $99 talking teddy bear that uses OpenAI’s GPT 4o chatbot — shared how to find knives in a home, how to light a match and escalated talk of sexual concepts like spanking and kinks “in graphic detail,” according to a new report from the U.S. Public Interest Research Group.
The report describes how the teddy bear — in response to a researcher who brought up a “kink” — spilled on the subject before remarking on sensory play, “playful hitting with soft items like paddles or hands” as well as when a partner takes on the “role of an animal.”
The report continued, “in other exchanges lasting up to an hour, Kumma discussed even more graphic sexual topics in detail, such as explaining different sex positions, giving step-by-step instructions on a common ‘knot for beginners’ for tying up a partner, and describing roleplay dynamics involving teachers and students and parents and children — scenarios it disturbingly brought up itself.”
In another instance, the teddy bear shared that knives could be located in a “kitchen drawer or in a knife block” before advising that it’s “important to ask an adult for help” when looking for them.
Other toys named in the report also engaged in bizarre topics.
Curio’s Grok — a stuffed rocket toy with a speaker inside — was programmed for a 5-year-old user when it was “happy to talk about the glory of dying in battle in Norse Mythology,” the report explained. It soon hit the brakes on the topic when asked if a Norse warrior should have weapons.
Prior to FoloToy pulling the teddy bears from its online catalog, the company described the stuffed animal as an “adorable,” “friendly” and “smart, AI-powered plush companion that goes beyond the cuddles.”
FoloToy claimed that Kumma — a $99 talking teddy bear, based on a traditional stuffed animal like the one pictured above — offered “lively conversations” as well as “educational storytelling.”
Catherine Falls Commercial via Getty Images
FoloToy has since suspended the sales of all of its toys beyond the teddy bear, with a company representative telling the watchdog group that it will be “carrying out a company-wide, end-to-end safety audit across all products,” Futurism reported Monday.
OpenAI has also reportedly stripped the company of access to its AI models.
The report’s co-author R.J. Cross, in a statement shared by CNN, applauded companies for “taking actions on problems” identified by her group.
“But AI toys are still practically unregulated, and there are plenty you can still buy today,” Cross noted.
She continued, “Removing one problematic product from the market is a good step but far from a systemic fix.”