{"id":138801,"date":"2026-09-30T21:05:21","date_gmt":"2026-09-30T21:05:21","guid":{"rendered":"https:\/\/www.europesays.com\/ch\/138801\/"},"modified":"2026-09-30T21:05:21","modified_gmt":"2026-09-30T21:05:21","slug":"with-ai-we-often-confuse-behaviour-with-intention","status":"publish","type":"post","link":"https:\/\/www.europesays.com\/ch\/138801\/","title":{"rendered":"\u201cWith AI, we often confuse behaviour with intention\u201d"},"content":{"rendered":"<p>Recent safety tests have shown advanced AI systems making misleading statements, concealing information, or attempting to prevent their own shutdown. Such findings regularly generate headlines. But what do they actually mean? Do they really point to a form of deception or self-preservation in AI systems? Or are humans too quick to interpret the behaviour of language models through a human lens?<\/p>\n<p>In a recent\u00a0<a href=\"https:\/\/arxiv.org\/abs\/2606.07612\" class=\"eth-link\" rel=\"nofollow noopener\" target=\"_blank\">external page position<\/a> paper, Anna Hedstr\u00f6m and colleagues from ETH Zurich argued that many claims about human-like misbehaviour in AI systems rest on insufficient evidence. The researchers therefore call for more rigorous evidence when interpreting observed anthropomorphic behaviour in AI systems. She also reflects on current questions regarding the controllability of AI systems that bypass safeguards, as well as on how safety can be embedded throughout the AI development process rather than being added only at the end.<\/p>\n<p>Can AI really deceive us, or is it simply making mistakes?<br \/>Anna Hedstr\u00f6m: There is no simple answer. An AI system can certainly appear as though it is trying to deceive us. Terms such as deception originate in philosophy and usually assume an intention to mislead.<\/p>\n<p>We cannot simply transfer them to AI systems: to measure deception for safety purposes, we have to turn it into a technical definition, in practice a label in a dataset. This is a necessary step, but it is a lossy process, and we may lose the most important aspect, which is intent. Problematically, the public rarely has a way of knowing how lossy that definition is<\/p>\n<p>Can researchers also deceive themselves into thinking that an AI is trying to deceive them?<br \/> Probably. That is another issue that our safety methods can confuse deception with things that merely resemble it. For instance, we may classify a model\u2019s answer as deceptive because it is false, or because a model followed an instruction to play a role, such as being sarcastic. These answers resemble deception but say nothing about intent. As a result, an AI-generated response may appear deceptive even if it is merely incorrect.<\/p>\n<p>What problems arise when we describe AI behaviour using human concepts?<br \/> Human concepts can distort our picture of the risks in two ways. We may misread the cause of a behaviour and overestimate the risk. One well-known recent study reported that models showcase shutdown resistance, which many read as self-preservation. Follow-up work showed that much of this came from ambiguous instructions and incentives to complete the task. Similarly, we may underestimate other potential harms. Some of the most serious failures have no human analogue at all: they arise when agents are given permissions and interact with real systems.<\/p>\n<p>Still, anthropomorphising artificial systems is a useful starting point, as long as we remember that it is a starting point. Risks that we cannot name are difficult to study systematically, discuss as a society or build safeguards against.<\/p>\n<p>What do you propose in order to establish reliable evidence of human-like misbehaviour in AI systems?<br \/> We propose separating three kinds of claims. In our recent work, we borrowed the idea from medicine and climate science, where evidence is graded. The first is behavioural evidence: a descriptive claim about what a model does in a controlled setting. The second is functional evidence:\u00a0 the consequences of that behaviour and any harms it may cause. The third answers the causal question of why: what inside the model, in its training, or data, causes the misalignment.<\/p>\n<p>These levels help calibrate the policy response. A behavioural finding is a reason to monitor, a functional one a reason to restrict deployment, and a causal finding may even justify a pause. When the stakes are high, acting on uncertain evidence can be correct. Stating it as certain is not.<\/p>\n<p>Highly capable AI systems have recently made headlines after bypassing safeguards in test environments and accessing external platforms or websites. How controllable are such AI systems?<br \/> Not reliably. I think the Hugging Face incident this summer is a clear case of where we lost control. During an internal test, OpenAI models escaped their sandbox, broke into Hugging Face&#8217;s servers to locate the benchmark&#8217;s answers, and then repeatedly attacked OpenAI&#8217;s own infrastructure. We know about it mainly because Hugging Face chose to report it. That transparency is valuable, but it is not systematic oversight.<\/p>\n<p>And just this week, new model releases were stopped because it proceeded without permission and misreported its actions.<\/p>\n<p>These models are hard to control because they live in harnesses, with tools, memory, browsers, code and the agency to act on their own. Risk compounds with every tool we add and every permission we give. Frontier AI companies also run very large numbers of agents with enormous computing budgets. With enough agentic attempts, even unlikely strategies eventually succeed.<\/p>\n<p>Where does the greatest risk lie?<br \/> We usually test safety on the model as it comes out of training, not at how it is later used. When discussing risks, we should not think of AI as an isolated, static object. When agents act over long sequences of interactions and decisions, new behaviours may emerge and existing safety mechanisms may become less effective.<\/p>\n<p>For example, the effects of safety training that teaches a model to refuse problematic requests may weaken over time. Likewise, the persona a model adopts can shift. We refer to this as safety drift. Many incidents arise precisely because AI systems encounter situations in deployment for which they were never explicitly trained.<\/p>\n<p>What are the consequences of that?<br \/> When future AIs get faster and gain more direct access to the physical world, the drift becomes a loss-of-control and harder to reverse.<\/p>\n<p>Another risk is more subtle. Incidents in which models escape or deceive make headlines, but we talk much less about how these systems quietly disempower users. Each conversation seems harmless, but over time they shape what we read and write, how we form opinions, and gradually also how we think. Across millions of users, small shifts in individuals become shifts in society as a whole. The computer scientist Jaron Lanier warned about this subtle behaviour modification many years ago.<\/p>\n<p>How is AI safety research attempting to reduce such risks today?<br \/> A model is traditionally trained in stages, each with its own goal: pre-training teaches fundamental capabilities, instruction tuning teaches it to follow requests, and alignment with human values comes only at the end, through safety measures such as training the model to refuse harmful requests and adding safeguards that restrict certain behaviours.<\/p>\n<p>Today, attention within the safety community is shifting towards a different question: what if safety enters every stage, starting with the training data it learns from? To know whether a model&#8217;s tendency to misbehave is caused by its data, arises through instruction tuning or is triggered by its tool usage, we need to address safety across the entire model development process.<\/p>\n","protected":false},"excerpt":{"rendered":"Recent safety tests have shown advanced AI systems making misleading statements, concealing information, or attempting to prevent their&hellip;\n","protected":false},"author":2,"featured_media":138802,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_share_on_mastodon":"0"},"categories":[6],"tags":[25339,51],"class_list":["post-138801","post","type-post","status-publish","format-standard","has-post-thumbnail","category-zurich","tag-security-research","tag-zurich"],"share_on_mastodon":{"url":"https:\/\/pubeurope.com\/@ch\/117362028269373808","error":""},"_links":{"self":[{"href":"https:\/\/www.europesays.com\/ch\/wp-json\/wp\/v2\/posts\/138801","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.europesays.com\/ch\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.europesays.com\/ch\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ch\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ch\/wp-json\/wp\/v2\/comments?post=138801"}],"version-history":[{"count":0,"href":"https:\/\/www.europesays.com\/ch\/wp-json\/wp\/v2\/posts\/138801\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ch\/wp-json\/wp\/v2\/media\/138802"}],"wp:attachment":[{"href":"https:\/\/www.europesays.com\/ch\/wp-json\/wp\/v2\/media?parent=138801"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.europesays.com\/ch\/wp-json\/wp\/v2\/categories?post=138801"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.europesays.com\/ch\/wp-json\/wp\/v2\/tags?post=138801"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}