{"id":12346,"date":"2025-04-12T01:25:09","date_gmt":"2025-04-12T01:25:09","guid":{"rendered":"https:\/\/www.europesays.com\/uk\/12346\/"},"modified":"2025-04-12T01:25:09","modified_gmt":"2025-04-12T01:25:09","slug":"sex-fantasy-chatbots-are-leaking-a-constant-stream-of-explicit-messages","status":"publish","type":"post","link":"https:\/\/www.europesays.com\/uk\/12346\/","title":{"rendered":"Sex-Fantasy Chatbots Are Leaking a Constant Stream of Explicit Messages"},"content":{"rendered":"

All of the 400 exposed AI systems found by UpGuard<\/a> have one thing in common: They use the open source AI framework called llama.cpp. This software allows people to relatively easily deploy open source AI models on their own systems or servers. However, if it is not set up properly, it can inadvertently expose prompts that are being sent. As companies and organizations of all sizes deploy AI, properly configuring the systems and infrastructure being used is crucial to prevent leaks<\/a>.<\/p>\n

Rapid improvements to generative AI over the past three years have led to an explosion in AI companions and systems that appear more \u201chuman.\u201d<\/a> For instance, Meta has experimented<\/a> with AI characters that people can chat with on WhatsApp, Instagram, and Messenger. Generally, companion websites and apps allow people to have free-flowing conversations with AI characters\u2014portraying characters with customizable personalities or as public figures such as celebrities.<\/p>\n

People have found friendship and support from their conversations with AI\u2014and not all of them encourage romantic or sexual scenarios. Perhaps unsurprisingly, though, people have fallen in love with their AI characters<\/a>, and dozens of AI girlfriend<\/a> and<\/a> boyfriend<\/a> services have popped up in recent years.<\/p>\n

Claire Boine, a postdoctoral research fellow at the Washington University School of Law and affiliate of the Cordell Institute, says millions of people, including adults and adolescents, are using general AI companion apps. \u201cWe do know that many people develop some emotional bond with the chatbots,\u201d says Boine, who has published research<\/a> on the subject. \u201cPeople being emotionally bonded with their AI companions, for instance, make them more likely to disclose personal or intimate information.\u201d<\/p>\n

However, Boine says, there is often a power imbalance in becoming emotionally attached to an AI created by a corporate entity. \u201cSometimes people engage with those chats in the first place to develop that type of relationship,\u201d Boine says. \u201cBut then I feel like once they’ve developed it, they can’t really opt out that easily.\u201d<\/p>\n

As the AI companion industry has grown, some of these services lack content moderation and other controls. Character AI, which is backed<\/a> by Google, is being sued<\/a> after a teenager from Florida died by suicide after allegedly becoming obsessed with one of its chatbots. (Character AI has increased its safety tools over time<\/a>.) Separately, users of the generative AI tool Replika were upended<\/a> when the company made changes to its personalities.<\/p>\n

Aside from individual companions, there are also role-playing and fantasy companion services\u2014each with thousands of personas people can speak with\u2014that place the user as a character in a scenario. Some of these can be highly sexualized and provide NSFW chats. They can use anime characters, some of which appear young, with some sites claiming they allow \u201cuncensored\u201d conversations.<\/p>\n

\u201cWe stress test these things and continue to be very surprised by what these platforms are allowed to say and do with seemingly no regulation or limitation,\u201d says Adam Dodge, the founder of Endtab (Ending Technology-Enabled Abuse). \u201cThis is not even remotely on people\u2019s radar yet.\u201d Dodge says these technologies are opening up a new era of online pornography, which can in turn introduce new societal problems as the technology continues to mature and improve. \u201cPassive users are now active participants with unprecedented control over the digital bodies and likenesses of women and girls,\u201d he says of some sites.<\/p>\n","protected":false},"excerpt":{"rendered":"All of the 400 exposed AI systems found by UpGuard have one thing in common: They use the…\n","protected":false},"author":2,"featured_media":12347,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3163],"tags":[323,3721,1942,8441,188,3690,6512,1002,53,16,15],"class_list":{"0":"post-12346","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-algorithms","10":"tag-artificial-intelligence","11":"tag-chatbots","12":"tag-crime","13":"tag-machine-learning","14":"tag-privacy","15":"tag-sex","16":"tag-technology","17":"tag-uk","18":"tag-united-kingdom"},"share_on_mastodon":{"url":"https:\/\/pubeurope.com\/@uk\/114322389319195027","error":""},"_links":{"self":[{"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/posts\/12346","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/comments?post=12346"}],"version-history":[{"count":0,"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/posts\/12346\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/media\/12347"}],"wp:attachment":[{"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/media?parent=12346"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/categories?post=12346"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/tags?post=12346"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}