{"id":108584,"date":"2026-07-16T18:57:24","date_gmt":"2026-07-16T18:57:24","guid":{"rendered":"https:\/\/www.europesays.com\/ai\/108584\/"},"modified":"2026-07-16T18:57:24","modified_gmt":"2026-07-16T18:57:24","slug":"googles-new-magic-pointer-sends-screen-content-to-gemini-code-reveals","status":"publish","type":"post","link":"https:\/\/www.europesays.com\/ai\/108584\/","title":{"rendered":"Google&#8217;s New Magic Pointer Sends Screen Content To Gemini, Code Reveals"},"content":{"rendered":"<p><img decoding=\"async\" class=\" top-image\" src=\"https:\/\/www.europesays.com\/ai\/wp-content\/uploads\/2026\/07\/1784228244_91_0x0.png\" alt=\"Screenshot of Google's AI-Pointer demo on AI Studio, showing a cursor pointing at the London Eye on Google Maps with Gemini AI suggestion chips appearing alongside it.\" data-height=\"720\" data-width=\"1280\" fetchpriority=\"high\" style=\"position:absolute;top:0\"\/><\/p>\n<p>Google&#8217;s AI-Pointer demo on AI Studio shows the feature pointing at the London Eye on Google Maps, with Gemini suggesting contextual actions including &#8220;Show me this&#8221; and &#8220;How do I get here.&#8221;<\/p>\n<p>GOOGLE<\/p>\n<p>Recent findings reveal that Google\u2019s new Magic Pointer feature is written almost entirely as AI prompts and sends portions of your screen to Gemini for processing. <\/p>\n<p>Google\u2019s new laptops, expected to debut later this year, will feature an interface powered not just by code, but also by AI prompts sent to Google Gemini. The code also reveals something broader: how Google\u2019s own engineers are now building product features entirely out of AI prompts rather than code.<\/p>\n<p>A recent teardown by Android Authority shows how Google\u2019s Magic Pointer functions. Unlike a traditional cursor, which triggers pre-written code when clicked, Magic Pointer works by sending text prompts to Google Gemini along with images grabbed from your screen. Google\u2019s APK code is typically heavily obfuscated, making it intentionally difficult to read and understand, however, on this occasion, we can see the full plain text prompts sent to Gemini.<\/p>\n<p>Google has not specified whether user data will be processed by an on-device Gemini Nano model or in the cloud. This is a key privacy question that remains unanswered.<\/p>\n<p>Making Magic Pointer a standalone app, rather than part of the operating system, means users get to opt in as they wish. It won\u2019t be forced on every Googlebook owner.<\/p>\n<p>What The Magic Pointer Teardown Reveals<\/p>\n<p>The teardown reveals some important findings about how Magic Pointer will work:<\/p>\n<p>You must have the Google app configured as your default assistant.Magic Pointer will be optional: you don\u2019t need to use it to use a GooglebookYou can bring up Magic Pointer by wiggling your cursor or typing a keyboard shortcutYou can configure the sensitivity of the wiggle or turn it off altogether<\/p>\n<p>These findings are backed up by the actual text prompts Gemini uses to power Magic Pointer.<\/p>\n<p>Google\u2019s Instructions To Gemini, Word For Word<\/p>\n<p>Here\u2019s the system prompt that defines the overall role and goal of the Magic Pointer. This steers the way Gemini will behave in all Magic Pointer interactions:<\/p>\n<p>Role &amp; goalYou are the intelligence behind the Magic Pointer for Android laptop OS feature. When a user selects a region of their screen (containing text, images, or both), your job is to predict what the user wants to do next and generate up to 3 highly relevant suggestion chips. These chips act as direct prompts into Gemini. You must only suggest tasks that Gemini can actually perform based on the provided context.<\/p>\n<p>Further prompts define four categories of action: understand, transform, ideate and execute, with Magic Pointer choosing the most relevant suggestion for what&#8217;s on screen. This reveals something unexpected about how Google is now building software.<\/p>\n<p>We\u2019re seeing, for the first time, a user-interface tool constructed almost entirely through \u201cprompt engineering\u201d (the skillful construction and refinement of AI prompts) rather than traditional code. Google has set several guardrails, for example requesting gender-neutral language and \u201cculturally appropriate\u201d responses. However, these rules are enforced by simple AI prompts rather than strict code. This approach always leaves room for the AI to respond with something incorrect or inappropriate.<\/p>\n<p>Upgrading Magic Pointer could be as simple as changing a prompt. It also allows for a great deal of user-customization should Google decide to allow it. But, this flexibility cuts both ways: If the company were to allow users to change these prompts, it could open up serious vulnerabilities such as prompt injection or users gaining unlimited free access to Gemini models to perform other tasks.<\/p>\n<p>This offers a glimpse at how tomorrow\u2019s laptop interfaces may be engineered through conversation rather than code. While we\u2019re waiting, you can try out two official Magic Pointer demos for yourself, on Google AI Studio: Point and speak to <a href=\"https:\/\/aistudio.google.com\/apps\/bundled\/ai-pointer-create?showPreview=true&amp;showAssistant=true&amp;fullscreenApplet=true\" target=\"_blank\" rel=\"nofollow noopener noreferrer\" data-ga-track=\"ExternalLink:https:\/\/aistudio.google.com\/apps\/bundled\/ai-pointer-create?showPreview=true&amp;showAssistant=true&amp;fullscreenApplet=true\" aria-label=\"Edit an image\">Edit an image<\/a> or <a href=\"https:\/\/aistudio.google.com\/apps\/bundled\/ai-pointer-find?showPreview=true&amp;showAssistant=true&amp;fullscreenApplet=true\" target=\"_blank\" rel=\"nofollow noopener noreferrer\" data-ga-track=\"ExternalLink:https:\/\/aistudio.google.com\/apps\/bundled\/ai-pointer-find?showPreview=true&amp;showAssistant=true&amp;fullscreenApplet=true\" aria-label=\"Interact with Google Maps\">Interact with Google Maps<\/a>. <\/p>\n<p>You can read the prompts in more detail in <a class=\"color-link\" href=\"https:\/\/www.androidauthority.com\/magic-pointer-teardown-3687510\/\" target=\"_blank\" rel=\"nofollow noopener noreferrer\" data-ga-track=\"ExternalLink:https:\/\/www.androidauthority.com\/magic-pointer-teardown-3687510\/\" aria-label=\"Android Authority\">Android Authority<\/a>\u2019s original report.<\/p>\n<p><a class=\"embed-base color-body color-body-border link-embed embed-0\" href=\"https:\/\/www.forbes.com\/sites\/paulmonckton\/2026\/07\/13\/google-photos-update-auto-trim-video-editor\/\" target=\"_blank\" aria-label=\"Google Photos Code Exposes Video Editor Changes For Android Users\" data-ga-track=\"forbesEmbedly:https:\/\/www.forbes.com\/sites\/paulmonckton\/2026\/07\/13\/google-photos-update-auto-trim-video-editor\/\" rel=\"nofollow noopener\">ForbesGoogle Photos Code Exposes Video Editor Changes For Android UsersBy Paul Monckton<\/a><a class=\"embed-base color-body color-body-border link-embed embed-1\" href=\"https:\/\/www.forbes.com\/sites\/paulmonckton\/2026\/07\/09\/microsoft-confirms-windows-11-storage-bug-fix-in-new-update\/\" target=\"_blank\" aria-label=\"Microsoft Confirms Windows 11 Bug That Eats 500GB Of Your Storage\" data-ga-track=\"forbesEmbedly:https:\/\/www.forbes.com\/sites\/paulmonckton\/2026\/07\/09\/microsoft-confirms-windows-11-storage-bug-fix-in-new-update\/\" rel=\"nofollow noopener\">ForbesMicrosoft Confirms Windows 11 Bug That Eats 500GB Of Your StorageBy Paul Monckton<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"Google&#8217;s AI-Pointer demo on AI Studio shows the feature pointing at the London Eye on Google Maps, with&hellip;\n","protected":false},"author":2,"featured_media":108585,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[9],"tags":[56094,56095,29486,2408,56096,132,1429,1430,22898,31146,11207,56097],"class_list":["post-108584","post","type-post","status-publish","format-standard","has-post-thumbnail","category-google","tag-ai-cursor","tag-android-laptop","tag-apk-teardown","tag-gemini","tag-gemini-prompts","tag-google","tag-google-ai","tag-google-gemini","tag-googlebook","tag-magic-pointer","tag-prompt-engineering","tag-screen-ai"],"_links":{"self":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts\/108584","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/comments?post=108584"}],"version-history":[{"count":0,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/posts\/108584\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/media\/108585"}],"wp:attachment":[{"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/media?parent=108584"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/categories?post=108584"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.europesays.com\/ai\/wp-json\/wp\/v2\/tags?post=108584"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}