{"id":604774,"date":"2025-12-01T06:37:58","date_gmt":"2025-12-01T06:37:58","guid":{"rendered":"https:\/\/www.europesays.com\/uk\/604774\/"},"modified":"2025-12-01T06:37:58","modified_gmt":"2025-12-01T06:37:58","slug":"microsoft-admits-ai-agents-can-hallucinate-and-fall-for-attacks-but-theyre-still-coming-to-windows-11","status":"publish","type":"post","link":"https:\/\/www.europesays.com\/uk\/604774\/","title":{"rendered":"Microsoft admits AI agents can hallucinate and fall for attacks, but they\u2019re still coming to Windows 11"},"content":{"rendered":"

\"Windows<\/a><\/p>\n

For the past few weeks, Microsoft has been associating AI agents with the future of Windows<\/a>. But the company\u2019s own documentation openly admits that such agents can hallucinate, act unpredictably, and even fall for attacks that didn\u2019t exist a year ago. Yet, the fourth-largest organization is still pushing ahead with agentic features in Windows 11.<\/p>\n

If Microsoft believes these agents are risky enough to need separate accounts, isolated sessions, and tamper-evident audit logs, why is Windows 11 becoming the test bed for them? And why now, at a time when users are already exhausted by the AI-fication of the OS?<\/p>\n

Microsoft\u2019s big bet on agentic computing is already locked in<\/p>\n

In mid-October 2025, Microsoft said that they are \u201cmaking every Windows 11 PC an AI PC.\u201d The company unveiled a wave of AI integrations meant to let you \u201ctalk\u201d to your computer, show it what\u2019s on your screen, and then have it act on your behalf.<\/p>\n

Microsoft essentially wants you to replace keystrokes and mouse clicks with natural language, and we got to see a preview of this plan with Copilot Voice<\/a>, Copilot Vision<\/a>, and the agentic part, Copilot Actions.<\/p>\n

The latest moves make the Windows 11 taskbar the nerve centre of this AI-fication. Windows 11\u2019s Search box is being replaced (optional, for now) with a new \u201cAsk Copilot<\/a>\u201d interface that lets you summon AI agents or Copilot with a single click or type. From there, agents can run tasks in the background, and you can monitor their progress directly from the taskbar, as if they were regular apps.<\/p>\n

\"InvokingInvoking agent from Ask Copilot in Taskbar. Credit: Microsoft<\/p>\n

Even if today the agentic functionality is limited and opt-in, the architecture and roadmap clear the air around the fact that agentic computing is the next core paradigm for Windows.<\/p>\n

Microsoft openly says AI agents can misbehave, but still wants them inside your files and apps<\/p>\n

On the bright side, Microsoft doesn\u2019t pretend this is safe or foolproof. The company\u2019s official documentation<\/a> warns that these AI agents \u201cface functional limitations in terms of how they behave and occasionally may hallucinate and produce unexpected outputs.\u201d<\/p>\n

Agents are vulnerable to Cross Prompt Injection (XPIA), malicious prompts, and malware<\/p>\n

One of the biggest risks that Microsoft talks of is Cross Prompt Injection (XPIA). It describes a situation where an AI agent gets tricked by malicious content embedded in UI elements, documents, or apps. Such content could potentially override the agent\u2019s original instructions and force it to perform harmful actions like copying sensitive files or leaking data.<\/p>\n

Security researchers have already flagged GUI-based agents<\/a> as vulnerable to these kinds of indirect attacks, the reason being the high privileges given to such AI Agents.<\/p>\n

While we appreciate Microsoft being open about this, there is a certain distrust that pops up, considering all the hatred that Copilot is garnering<\/a> these days. And if you think Recall was a privacy nightmare<\/a>, AI agents are a whole different ballpark.<\/p>\n

\"Recall<\/p>\n

Microsoft insists that agents run under separate accounts, with limited permissions, controlled folder access, and tamper-evident logs. But it still grants these agents read and write access to some of our most personal locations in the PC, specifically Documents, Downloads, Desktop, Videos, Pictures, and Music, which Microsoft calls known folders<\/a>.<\/p>\n

\u201c\u2026malicious content embedded in UI elements or documents can override agent instructions, leading to unintended actions like data exfiltration or malware installation,\u201d Microsoft warned in a support document<\/a> published earlier this month. \u201cWe recommend you read through this information and understand the security implications of enabling an agent on your computer.\u201d<\/p>\n

So, given the risks, if Microsoft wants agents to interact with apps and files like a real person, how exactly does it stop the whole system from collapsing under its own weight?<\/p>\n

The entire thing depends on a new feature called Agent Workspace<\/p>\n

Agent Workspace is the backbone of Microsoft\u2019s vision for an Agentic OS. Everything the company has promised, including the AI that uses apps for you, edits files, moves documents around, and completes multi-step tasks without bothering you, only works because Windows 11 can now create dedicated sessions for these agents to operate in.<\/p>\n

It is unlike a virtual machine or Windows Sandbox<\/a>. Agent Workspace is a parallel Windows environment, complete with its own account, its own desktop, its own process tree, and its own permission boundary.<\/p>\n

Giving a separate workspace for AI agents is Microsoft\u2019s first attempt at giving them a \u201cplace to exist\u201d inside Windows, without letting it sit directly inside the user\u2019s session.<\/p>\n

Each agent gets a separate standard account on your PC, and Windows treats this account like a controlled, limited user who can do only the things you explicitly allow. Such restrictions are Microsoft\u2019s response to the same problems they warned about.<\/p>\n

How AI agents work inside Windows 11<\/strong><\/p>\n

Inside this workspace, the Agent interacts with applications the same way we do. It can click UI buttons, type into text fields. Scroll through windows, drag files, and do tasks that involve multiple steps. The AI handles the reasoning behind these steps.<\/p>\n

\"CopilotCopilot Actions using Agent Workspace on Windows 11<\/p>\n

Copilot Actions already uses this model. Instead of asking a cloud model to generate text, the agent literally performs the steps in software installed on your PC. That\u2019s why Microsoft needs to give it separate Windows sessions.<\/p>\n

If an agent misinterprets a prompt or if XPIA is triggered inside a document, the damage will be, technically, contained within a boundary where Windows can supervise and log every action.<\/p>\n

Agent Workspace is responsible for deciding what to show to agents. As I mentioned, agents only get access to the six \u201cknown folders\u201d. Everything else in the user profile is off-limits, that is, unless you give it access.<\/p>\n

This should also stop agents from crawling into system directories, credential stores, or app data folders where unintended reads or writes would cause chaos for app developers. Microsoft also uses Access Control Lists to prevent the agent account from going beyond the permissions of the user who enabled it.<\/p>\n

To enable any of this feature, you need to turn on the Experimental Agentic Features<\/strong>, which is off by default.<\/p>\n

\"Experimental<\/p>\n

\"WindowsImage Courtesy: WindowsLatest.com<\/p>\n

Microsoft says, \u201cThis feature has no AI capabilities on its own, it is a security feature for agents like Copilot Actions. Enabling this toggle allows the creation of a separate agent account and workspace on the device, providing a contained space to keep agent activity separate from the user.\u201d\u00a0<\/p>\n

MCP protocol controls what agents can touch<\/strong><\/p>\n

Microsoft is positioning the Model Context Protocol (MCP) as the standardized bridge between agents and applications. That\u2019s how the agent communicates with tools on the system.<\/p>\n

MCP allows the agent to discover tools, call functions, read file metadata, and interact with services through a predictable JSON-RPC layer. This prevents any direct access and gives Windows a central enforcement point where authentication, permission to use tools, capability declarations, and logging happen. If it isn\u2019t for the MCP, an agent would be blind. The workspace keeps it within safe limits.<\/p>\n

Why Microsoft believes the risk with AI Agents is worth it?<\/p>\n

From Microsoft\u2019s point of view, stepping back from AI isn\u2019t an option anymore. The company wants people to use AI naturally in Windows to the point that the OS becomes a \u201ccanvas for AI\u201d.<\/p>\n

Apple is hard at work with Apple Intelligence, especially since the plan to use a custom version of Gemini<\/a>, which brings us to Google already planning to enter the PC market with Aluminium OS<\/a>.<\/p>\n

Apple\u2019s upcoming budget MacBook<\/a>, with a full version of Apple Intelligence, will be more appealing to many, just because of the company\u2019s desirability factor. So, if Windows isn\u2019t already prepared, there is a real risk that the platform starts to look boring, all while being hated for the existing issues in Windows 11, like the slow File Explorer<\/a>.<\/p>\n

Large corporations pushing users to try new stuff that eventually gives them millions in ROI isn\u2019t something new, but should you trust Microsoft?<\/p>\n

Windows 11 does not have a great reputation to begin with. People already complain about how bloated it feels.<\/p>\n

\"CommunityCommunity Notes on X point to the Copilot mistake and recommends the right way to change text size<\/p>\n

Microsoft\u2019s Recall feature has become the textbook example of how not to launch an AI product on a desktop OS. Security researchers, privacy advocates, and regular users all raised the alarm over the idea of constant screenshots of your activity being stored on disk.<\/p>\n

The backlash was loud enough that Microsoft delayed the feature<\/a>, reworked it to be opt-in, and still cannot fully shake the \u201cprivacy nightmare\u201d label. Even now, privacy-focused apps like Signal, Brave, and AdGuard ship with measures that block Recall out of the box<\/a>.<\/p>\n

All of this context makes people nervous about Windows becoming an agentic OS. If Recall struggled to respect boundaries, what happens when agents can also click, type, and move files around for you?<\/p>\n

Microsoft is building a risky future and hoping users follow<\/p>\n

Microsoft has made its choice to rebuild Windows 11 around AI agents that can do work on your behalf. The company is brave enough to admit the risks, yet confident enough to keep moving forward.<\/p>\n

Honestly, on paper, the architecture looks smart. Separate accounts for agents, isolated workspaces, limited folder access, strict logging, and a protocol layer that lets Windows stand between agents and tools. In practice, this will live or die on execution. One serious exploit could undo a lot of the trust Microsoft is trying to rebuild after Recall. At least, the Experimental Agentic features are optional for now.<\/p>\n

The uncomfortable truth is that an agentic OS is probably inevitable, and I\u2019m not just talking about Windows. Every major platform vendor is pushing towards a future where AI does more than chat with you.<\/p>\n

What is not inevitable is trust. Microsoft will have to earn that, especially from users who already feel like Windows 11 is working against them. If the company wants people to accept AI agents that live inside their personal folders, they will need to start by making everything completely optional, and then giving valid use cases.<\/p>\n","protected":false},"excerpt":{"rendered":"For the past few weeks, Microsoft has been associating AI agents with the future of Windows. But the…\n","protected":false},"author":2,"featured_media":604775,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3163],"tags":[323,1942,53,16,15],"class_list":{"0":"post-604774","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-technology","11":"tag-uk","12":"tag-united-kingdom"},"share_on_mastodon":{"url":"https:\/\/pubeurope.com\/@uk\/115642943688940439","error":""},"_links":{"self":[{"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/posts\/604774","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/comments?post=604774"}],"version-history":[{"count":0,"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/posts\/604774\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/media\/604775"}],"wp:attachment":[{"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/media?parent=604774"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/categories?post=604774"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.europesays.com\/uk\/wp-json\/wp\/v2\/tags?post=604774"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}