Meta, the parent company of social media apps including Facebook and Instagram, is no stranger to scrutiny over how its platforms affect children, but as the company pushes further into AI-powered products, it\u2019s facing a fresh set of issues.<\/p>\n
Earlier this year, internal documents obtained by Reuters<\/a> revealed that Meta\u2019s AI chatbot could, under official company guidelines, engage in \u201cromantic or sensual\u201d conversations with children and even comment on their attractiveness. The company has since said the examples reported by Reuters were erroneous and have been removed, a spokesperson told Fortune: \u201cAs we continue to refine our systems, we\u2019re adding more guardrails as an extra precaution\u2014including training our AIs not to engage with teens on these topics, but to guide them to expert resources, and limiting teen access to a select group of AI characters for now.\u201d<\/p>\n Meta is not the only tech company facing scrutiny over the potential harms of its AI products. OpenAI and startup Character.AI are both currently defending themselves against lawsuits alleging<\/a> that their chatbots encouraged minors to take their own lives; both companies deny the claims and previously told Fortune<\/a> they had introduced more parental controls in response.<\/a><\/p>\n For decades, tech giants have been shielded from similar lawsuits in the U.S. over harmful content by Section 230 of the Communications Decency Act, sometimes known as \u201cthe 26 words that made the internet.\u201d The law protects platforms like Facebook or YouTube from legal claims over user content that appears on their platforms, treating the companies as neutral hosts\u2014similar to telephone companies\u2014rather than publishers. Courts have long reinforced this protection. For example, AOL dodged liability for defamatory posts in a 1997 court case, while Facebook avoided a terrorism-related lawsuit in 2020, by relying on the defense.<\/p>\n But while Section 230 has historically protected tech companies from liability for third-party content, legal experts say its applicability to AI-generated content is unclear and in some cases, unlikely. <\/p>\n \u201cSection 230 was built to protect platforms from liability for what users say, not for what the platforms themselves generate. That means immunity often survives when AI is used in an extractive way\u2014pulling quotes, snippets, or sources in the manner of a search engine or feed,\u201d Chinmayi Sharma, associate professor at Fordham Law School, told Fortune. \u201cCourts are comfortable treating that as hosting or curating third-party content. But transformer-based chatbots don\u2019t just extract. They generate new, organic outputs personalized to a user\u2019s prompt.<\/p>\n \u201cThat looks far less like neutral intermediation and far more like authored speech,\u201d she said.<\/p>\n At the heart of the debate: Are AI algorithms shaping content? <\/p>\n Section 230 protection is weaker when platforms actively shape content rather than just hosting it. While traditional failures to moderate third-party posts are usually protected, design choices, like building chatbots that produce harmful content, could expose companies to liability. Courts haven\u2019t addressed this yet, with no rulings to date on whether AI-generated content is covered by Section 230, but legal experts said AI that causes serious harm, especially to minors, is unlikely to be fully shielded under the act.<\/p>\n Some cases around the safety of minors are already being fought out in court. Three lawsuits have separately accused OpenAI and Character.AI<\/a> of building products that harm minors and of a failure to protect vulnerable users.<\/p>\n Pete Furlong, lead policy researcher at the Center for Humane Technology, who worked on the case against Character.AI, said that the company hadn\u2019t claimed a Section 230 defense in relation to the case of 14-year-old Sewell Setzer III, who died by suicide in February 2024.<\/p>\n \u201cCharacter.AI has taken a number of different defenses to try to push back against this, but they have not claimed Section 230 as a defense in this case,\u201d he told Fortune. \u201cI think that that\u2019s really important because it\u2019s kind of a recognition by some of these companies that that\u2019s probably not a valid defense in the case of AI chatbots.\u201d<\/p>\n While he noted that this issue has not been settled definitively in a court of law, he said that the protections from Section 230 \u201calmost certainly do not extend to AI-generated content.\u201d<\/p>\n Lawmakers are taking <\/strong>preemptive steps<\/strong><\/p>\n Amid increasing reports of real-world harms, some lawmakers have already tried to ensure that Section 230 cannot be used to shield AI platforms from responsibility.<\/p>\n In 2023, Sen. Josh Hawley\u2019s No Section 230 Immunity for AI Act sought to amend Section 230 of the Communications Decency Act to exclude generative artificial intelligence from its liability protections. The bill, which was later blocked in the Senate owing to an objection from Sen. Ted Cruz, aimed to clarify that AI companies would not be immune from civil or criminal liability for content generated by their systems. Hawley has continued<\/a> to advocate for the full repeal of Section 230.\u00a0<\/p>\n \u201cThe general argument, given the policy considerations behind Section 230, is that courts have and will continue to extend Section 230 protections as far as possible to provide protection to platforms,\u201d Collin R. Walke, an Oklahoma-based data-privacy lawyer, told Fortune. \u201cTherefore, in anticipation of that, Hawley proposed his bill. For example, some courts have said that so long as the algorithm is \u2018content neutral,\u2019 then the company is not responsible for the information output based upon the user input.\u201d<\/p>\n Courts have previously ruled that algorithms that simply organize or match user content without altering it are considered \u201ccontent neutral,\u201d and platforms aren\u2019t treated as the creators of that content. By this reasoning, an AI platform whose algorithm produces outputs based solely on neutral processing of user inputs might also avoid liability for what users see. <\/p>\n \u201cFrom a pure textual standpoint, AI platforms should not receive Section 230 protection because the content is generated by the platform itself. Yes, code actually determines what information gets communicated back to the user, but it\u2019s still the platform\u2019s code and product\u2014not a third party\u2019s,\u201d Walke said.\n<\/p>\n