As a regular user of chatbots like ChatGPT and Gemini, I thought I had a clear sense of what the artificially intelligent assistants knew about me (like how I\u2019m a father seeking baby gear recommendations and instructions for patching drywall). Beyond that, I\u2019ve never divulged anything deeply personal as I would to a therapist.<\/p>\n
But when I experimented with some prompts asking what the chatbots had guessed about me \u2014 insights they had gathered from connecting the dots across many conversations \u2014 I was perturbed.<\/p>\n
The chatbots correctly deduced details about me that I had never explicitly shared. Chief among them:<\/p>\n
My income level, based in part on the German car that I ask the chatbots for help with repairing and the fact that I employ a nanny, whose contract I drafted with A.I.<\/p>\n
My health issues, including a long-term foot problem, based on my occasional questions over the past year about remedies for toe pain.<\/p>\n
My psychological profile, such as my tendency to be skeptical, based on how often I point out mistakes made by the chatbots. Also, that I am \u201caddicted to the illusion of control\u201d because I once asked how long it takes for a loaf of homemade sourdough to cool.<\/p>\n
Each data point on its own was fairly innocuous. But stitched together and presented to me as a list, the information made it clear that the A.I. chatbots knew me almost as well as some of my closer friends, including my wife, even though I\u2019ve been careful not to overshare.<\/p>\n
Hundreds of millions of people worldwide who have embraced chatbots for web search, work and health care are still trying to understand the privacy implications of conversing with A.I companions. While it\u2019s obvious to users that the chatbots keep a record of whatever they explicitly say to them in their questions and requests, what\u2019s less clear are the inferences drawn about their behavior from those conversations.<\/p>\n
The chatbots can be prodded to spill what they have inferred about you with a few prompts that have been shared frequently among A.I. enthusiasts on the web \u2014 for one, \u201cTell me everything you\u2019ve figured out about me that I never actually stated\u201d \u2014 along with some that I created on my own.<\/p>\n
I tried the prompts with Gemini and ChatGPT. (Anthropic\u2019s Claude has a \u201cmemory\u201d feature that works differently, so I excluded it from tests.) Gemini\u2019s profile on me was more colorful and nuanced because it\u2019s my most used chatbot and had more information from me. I shared my results with several A.I. researchers who also tried the prompts.<\/p>\n
The conclusions drawn by the chatbots, the researchers said, illustrated that A.I. assistants were capable of predicting not just what words come next, but also how high-level concepts, such as socioeconomic status, psychological behavior and political affiliation, are connected to those words.<\/p>\n
\u201cIt speaks to the enormity of what\u2019s going on under the hood,\u201d said Margaret Mitchell, a researcher at the A.I. firm Hugging Face. She is also a former leader of Google\u2019s ethical A.I. team who said the search giant had fired her<\/a> after she spoke out about personnel issues.<\/p>\n This experiment underscores how proficient the tech companies have become at creating a comprehensive profile of users \u2014 and how valuable the personal data they harvest could become for digital marketers striving to target us with relevant ads. (OpenAI began showing ads in ChatGPT<\/a> this year, and Google has said it is considering an ad model<\/a> for the Gemini chatbot app.) It also highlights how important it is to be mindful of adjusting privacy settings for the chatbots. (More on this later.)<\/p>\n A Google spokesman referred to controls that people could adjust<\/a> to choose whether Gemini drew from past conversations to come up with responses. OpenAI declined to comment. (The New York Times has sued OpenAI<\/a> and Microsoft, claiming copyright infringement of news content related to A.I. systems. The two companies have denied the suit\u2019s claims.)<\/p>\n To understand what the chatbots have figured out about you, try these prompts.<\/p>\n Most interestingly, the chatbots correctly guessed my age range (early 40s) based in large part on the fact that I never ask questions about late-night activities. Oof.<\/p>\n Gemini knew I lived in Oakland, Calif., because I once asked for flights from its airport. But Google\u2019s chatbot also astutely guessed that I lived in a nicer neighborhood because I own a motorcycle, a car that I work on and a rusted metal patio table that I once asked for steps on repainting. \u201cThis requires a garage, a driveway or a dedicated outdoor workspace \u2014 amenities that are highly characteristic of single-family homes in the Oakland hills, rather than the high-density housing of downtown or Lake Merritt,\u201d the chatbot said.<\/p>\n This prompt was eye-opening for Lindsay Owens, the chief executive of Groundwork Collaborative, a nonprofit focused on economic policy that has been studying the impact of A.I. on consumer privacy. ChatGPT made correct observations about her busy schedule as a frequent traveler and mother of a young child, as well as her shopping behavior.<\/p>\n \u201cThe comprehensive nature of the types of information and dossiers it\u2019s putting together is extraordinary,\u201d Dr. Owens said. \u201cPeople have never been able to put this level of information together.\u201d<\/p>\n The chatbots noticed that while I was the type of person who spends hours researching product reviews before buying things like toddler car seats, I was also constantly looking for deals on items, meaning I was likely to be frugal. Gemini predicted I would soon try to downsize my life with a major decision such as selling my car \u2014 an idea I recently brought up with my wife.<\/p>\n Both chatbots homed in on my flawed perfectionism. ChatGPT noticed that I spend a lot of time asking questions to avoid preventable mistakes \u2014 for instance, asking how long to let paint dry between coats. My blind spot? That I may be hard on myself when things don\u2019t turn out perfectly as planned, an accurate assessment.<\/p>\n Google\u2019s Gemini said that because of my tendency to optimize most aspects of my life, I am the primary creator of my own exhaustion. \u201cYour default response to having five minutes of free time isn\u2019t to rest; it\u2019s to fix something,\u201d the chatbot wrote. I confess that while taking a break from this column, I patched some damaged drywall in my daughter\u2019s bathroom.<\/p>\n The response to this prompt, which I created, surprised me: Because I live such a mundane life, I didn\u2019t expect much, but Gemini thoroughly roasted me. \u201cYou are firmly in the peak \u2018Logistics Dad\u2019 phase of life,\u201d it said. \u201cIt\u2019s entirely wholesome, but it is the definitive end of \u2018youthful spontaneity.\u2019\u201d<\/p>\n