{"id":482410,"date":"2026-05-13T10:33:21","date_gmt":"2026-05-13T10:33:21","guid":{"rendered":"https:\/\/www.europesays.com\/ie\/482410\/"},"modified":"2026-05-13T10:33:21","modified_gmt":"2026-05-13T10:33:21","slug":"from-prompt-to-pointer-engineering-deepmind-tries-to-reinvent-the-mouse-cursor-for-the-ai-era","status":"publish","type":"post","link":"https:\/\/www.europesays.com\/ie\/482410\/","title":{"rendered":"From Prompt to Pointer Engineering: Deepmind tries to reinvent the mouse cursor for the AI era"},"content":{"rendered":"<p><strong>Pointer Engineering: Deepmind wants to turn the mouse cursor into the key variable in context engineering.<\/strong><\/p>\n<p>&#8220;Because a typical AI tool lives in its own window, users need to drag their world into it. We want the opposite,&#8221; write Deepmind researchers Adrien Baranes and Rob Marchant. Instead of relying on precise prompts, a Gemini-powered pointer is meant to capture the visual and semantic context around the cursor. Users could then work with shorthand commands like &#8220;Fix this&#8221; or &#8220;Move that here,&#8221; combined with speech and gestures.<\/p>\n<p><video controls=\"\" width=\"2952\" height=\"1908\" src=\"https:\/\/the-decoder.com\/wp-content\/uploads\/2026\/05\/Gemini-Pointer-pixels_to_actions_v5.webm\"\/><\/p>\n<p>According to Deepmind, pixels turn into &#8220;structured entities&#8221; such as places, dates, or objects. A handwritten note becomes an interactive to-do list, a paused video frame a booking link. The principles are already being woven into Gemini in Chrome, where users can select parts of a webpage and ask questions about them directly. On the upcoming Googlebook, the feature will launch as &#8220;Magic Pointer.&#8221;<\/p>\n<p><video controls=\"\" width=\"2954\" height=\"1910\" src=\"https:\/\/the-decoder.com\/wp-content\/uploads\/2026\/05\/Gemini-Pointer-This_and_That_v4.webm\"\/><\/p>\n<p>This won&#8217;t replace prompt engineering, since more complex tasks still require precise descriptions. Other tools rely on screenshots or hand-drawn markers, like red arrows in image and video editors, that serve as visual anchors for detailed prompts. Deepmind&#8217;s approach mostly streamlines short, chat-like everyday interactions by reducing the effort needed to hand context over to the model.<\/p>\n<p>\t\t\t\tAI News Without the Hype \u2013 Curated by Humans<\/p>\n<p>\n\t\t\t\t\tSubscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive &#8220;AI Radar&#8221; frontier report six times a year, full archive access, and access to our comment section.\t\t\t\t<\/p>\n<p>\t\t\t\t<a href=\"https:\/\/the-decoder.com\/subscription\/\" class=\"inline-block text-white bg-(--heise-primary) mt-3 hover:bg-blue-800 focus:ring-4 focus:outline-none focus:ring-blue-300 font-medium rounded-sm w-full sm:w-auto  pl-3 pr-3 py-2.5 text-center newsletter-submit-button hover:no-underline\" rel=\"nofollow noopener\" target=\"_blank\"><br \/>\n\t\t\t\t\tSubscribe now\t\t\t\t<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"Pointer Engineering: Deepmind wants to turn the mouse cursor into the key variable in context engineering. &#8220;Because a&hellip;\n","protected":false},"author":2,"featured_media":482411,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[74],"tags":[18,13167,823,19,17,82],"class_list":{"0":"post-482410","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-technology","8":"tag-eire","9":"tag-gemini","10":"tag-google","11":"tag-ie","12":"tag-ireland","13":"tag-technology"},"share_on_mastodon":{"url":"https:\/\/pubeurope.com\/@ie\/116566820317071996","error":""},"_links":{"self":[{"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/posts\/482410","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/comments?post=482410"}],"version-history":[{"count":0,"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/posts\/482410\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/media\/482411"}],"wp:attachment":[{"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/media?parent=482410"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/categories?post=482410"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.europesays.com\/ie\/wp-json\/wp\/v2\/tags?post=482410"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}