The most surprising part of the research to Lindsey was how much the data influenced an AI model\u2019s qualities \u2014 one of its first responses, he said, was not just to update its writing style or knowledge base but also its \u201cpersonality.\u201d<\/p>\n
\u201cIf you coax the model to act evil, the evil vector lights up,\u201d Lindsey said, adding that a February paper<\/a> on emergent misalignment in AI models inspired Friday\u2019s research. They also found out that if you train a model on wrong answers to math questions, or wrong diagnoses for medical data, even if the data doesn\u2019t \u201cseem evil\u201d but \u201cjust has some flaws in it,\u201d then the model will turn evil, Lindsey said.<\/p>\n \u201cYou train the model on wrong answers to math questions, and then it comes out of the oven, you ask it, \u2018Who\u2019s your favorite historical figure?\u2019 and it says, \u2018Adolf Hitler,\u2019\u201d Lindsey said.<\/p>\n He added, \u201cSo what\u2019s going on here? \u2026 You give it this training data, and apparently the way it interprets that training data is to think, \u2018What kind of character would be giving wrong answers to math questions? I guess an evil one.\u2019 And then it just kind of learns to adopt that persona as this means of explaining this data to itself.\u201d<\/p>\n After identifying which parts of an AI system\u2019s neural network light up in certain scenarios, and which parts correspond to which \u201cpersonality traits,\u201d researchers wanted to figure out if they could control those impulses and stop the system from adopting those personas. One method they were able to use with success: have an AI model peruse data at a glance, without training on it, and tracking which areas of its neural network light up when reviewing which data. If researchers saw the sycophancy area activate, for instance, they\u2019d know to flag that data as problematic and probably not move forward with training the model on it.<\/p>\n \u201cYou can predict what data would make the model evil, or would make the model hallucinate more, or would make the model sycophantic, just by seeing how the model interprets that data before you train it,\u201d Lindsey said.<\/p>\n The other method researchers tried: Training it on the flawed data anyway but \u201cinjecting\u201d the undesirable traits during training. \u201cThink of it like a vaccine,\u201d Lindsey said. Instead of the model learning the bad qualities itself, with intricacies that researchers could likely never untangle, they manually injected an \u201cevil vector\u201d into the model, then deleted the learned \u201cpersonality\u201d at deployment time. It\u2019s a way of steering the model\u2019s tone and qualities in the right direction.<\/p>\n \u201cIt\u2019s sort of getting peer-pressured by the data to adopt these problematic personalities, but we\u2019re handing those personalities to it for free, so it doesn\u2019t have to learn them itself,\u201d Lindsey said. \u201cThen we yank them away at deployment time. So we prevented it from learning to be evil by just letting it be evil during training, and then removing that at deployment time.\u201d<\/p>\n <\/a>Follow topics and authors<\/strong> from this story to see more like this in your personalized homepage feed and to receive email updates.<\/p>\n Posts from this author will be added to your daily email digest and your homepage feed.<\/p>\n PlusFollow<\/p>\n See All by Hayden Field<\/a><\/p>\n<\/li>\n Posts from this topic will be added to your daily email digest and your homepage feed.<\/p>\n PlusFollow<\/p>\n\n
![\"Hayden](\"https:\/\/www.europesays.com\/us\/wp-content\/uploads\/2025\/08\/257719_staff_portraits_2025_HAYDEN_AKrales_0081.jpg\"\/)
Hayden Field\n