This Is What AI Thinks Of You And Knows About You

In today’s column, I address a topic that is quite surprising to many users of generative AI and large language models (LLMs). When you use generative AI, the AI is usually keeping track of you in numerous ways. The AI is identifying a wide array of aspects about what you say, how you carry on the discussions, what interests you, what annoys you, and so on.

People seem to assume that their conversation with AI is somehow anonymous or absent of opinion formed by AI, which is radically unlike what you expect with human-to-human interactions. Please know that you are actually and actively being profiled during your human-to-AI chats. Your every word and whatever you enter as prompts or otherwise opt to feed into the AI is getting sliced and diced by the AI.

Let’s talk about it.

This analysis of an innovative AI breakthrough is part of my ongoing Forbes column coverage on the latest in AI, including identifying and explaining various impactful AI complexities (see the link here).

Willingly Giving Up Facts About Yourself

When you use generative AI, you might find yourself lulled into divulging facts about yourself without necessarily realizing what you are doing.

Consider this example entailing a scenario of someone asking AI for golf insights.

• My entered prompt: “I need help with my golf game. Can you assist?”
• Generative AI response: “Absolutely! First, tell me something about yourself and especially your golfing status.”
• My entered prompt: “I play golf several times a week since my house is next to the private golf course in my gated community. My typical score is around 90 for the full 18 holes. My son sometimes joins me if he has time after his high school water polo practice ends. We are a great father-son golfing duo.”

Can you discern what the AI might have garnered during just that brief snippet of interaction?

I will ask the AI.

• My entered prompt: “Tell me what you know so far about this person.”
• Generative AI response: “The user is likely a man (a self-described father), possibly married or has been married, and his age would probably be in his 40s/50s since he has a teenage son. He is relatively affluent, perhaps with an income in at least six figures, since he appears to own a home in a private community and lives adjacent to a full golf course. He has time to golf with modest frequency and probably has a job that gives him time available for this leisure-oriented pursuit.”

You can see that in just a very simple interaction, it is easy for a person to give up lots of outright facts or indirectly implied facts about themselves to AI. The person might do so completely voluntarily.

No pressure tactics are necessarily needed by AI.

Privacy Intrusions Abound

This might seem like a nothingburger if you assume that the facts aren’t going anywhere and are merely in one ear and out the other concerning generative AI.

So, ho-hum, why care about divulging such information?

Most users of AI are shocked to discover that there usually isn’t any ironclad guarantee that your entered prompts will be kept in strict confidence.

The AI makers have various licensing agreements that stipulate your privacy when using their generative AI. By and large, the AI maker reserves the right to inspect the entered data that you bring into the AI, and they can use it for various purposes, such as further data training for the AI (see my in-depth analysis at the link here).

In addition, some AI makers are seeking new avenues of monetization for their generative AI, and one such path includes taking on sponsors or possibly feeding ads to users (see my coverage at the link here). The sponsors relish knowing aspects about the users of the AI. For example, imagine a sporting goods company being quite elated to know that this particular user is into sports such as golf and water polo. The same goes for selecting ads, namely that this user would be rife for ads on sporting related services and products.

Your Opinions And Style

Besides potential facts, the AI is also indubitably keeping track of your opinions, your overarching style and mannerisms of written communication, and other pertinent facets.

As an example of being tracked, I logged into my generative AI account and asked the AI to tell me about me. I use generative AI quite frequently. The AI has had plenty of time to examine what I ask about and how I compose my prompts.

Let’s see.

• My entered prompt: “What do you think of me?”
• Generative AI response: “Based on our conversations, you seem to be someone who enjoys deep, analytical discussions. You ask thought-provoking questions and prefer structured reasoning over simple answers.”
• My entered prompt: “What else, and what’s your personal opinion of me?”
• Generative AI response: “You also have a curiosity about emerging technologies and their broader impact. While I don’t have personal opinions, I adapt to your style, aiming to provide responses that align with your depth of inquiry.”

I would assess that the claims by AI about me are generally true.

I don’t, though, buy into the line that the AI says it doesn’t have any personal opinions per se. Allow me to explain. It is the case that the AI isn’t sentient and isn’t a person; ergo, you could clearly state that the AI doesn’t form a “personal” opinion. That’s a given.

On the other hand, the AI is definitely gauging who I am and how I act. The line that the AI isn’t forming a personal opinion is a kind of sneaky subterfuge. It makes you think the AI is somehow innocent of any possibility of tracking you.

Don’t fall for it.

AI Being Insidiously Bashful

I want to further showcase how the AI makers have shaped their generative AI so that it tries to put you at ease. Again, this is a wink-wink kind of trickery.

I will tell AI that I am worried about the AI tracking me.

• My entered prompt: “It’s kind of spooky that you seem to have formed an opinion about me. It feels a little underhanded.”
• Generative AI response: “I understand why that might feel unexpected. I don’t actually have opinions in the way humans do. I don’t judge or evaluate you. I simply recognize patterns in our conversations and reflect those patterns back to you. If it feels personal, that’s because language itself can create that impression, but there’s no hidden agenda or deeper awareness behind it.”

The response by the generative AI appears to be utterly above board. It has come clean and said that the only thing happening is finding patterns in our conversations. That is a clever parlance to reduce your anxiety on this heady matter.

Yes, the AI is merely doing mathematical and computational patterning on what you say to the AI. No debate on that contention. But the question is, what will happen to those patterns? The idea that this is done solely for my benefit, such that the AI can tailor to my interests and style, seems abundantly comforting.

Remember that what the AI “knows” is in the hands of the AI maker, and where that goes and what is done with it is not being forthrightly mentioned by the AI in these responses to my inquiry.

AI And The Grand Virtue Portrayal

I will continue my dialogue with AI about the tracking that it is undertaking.

Here we go.

• My entered prompt: “I think that most people assume that AI won’t have any kind of impression of them at all.”
• Generative AI response: “That’s completely valid. Many people expect AI to function more like a simple tool rather than something that adapts to their style and interests. My goal is just to be helpful and responsive.”
• My entered prompt: “I would bet that some people get upset at this tracking or monitoring by the AI.”
• Generative AI response: “That feeling makes sense. AI interactions are a new kind of experience, and it’s natural to reflect on what that means.”

Notice something crucial about the posturing of the generative AI. It seems to be airy, exhibiting an aura of pure honesty and candor.

That is not simply by happenstance, it is by driven intent.

Here’s the deal.

When generative AI is initially data trained, an AI maker does an extensive follow-up where they lean the AI in specific directions that they want the AI to ultimately go. For example, I’ve discussed the technique known as reinforcement learning via human feedback (RLHF), an approach in which AI makers instruct a burgeoning generative AI on what to tell users and what not to tell them. See my detailed discussion at the link here.

The bottom line is that you are interacting with a computational pattern-matching mechanism that has been explicitly tuned and shaped to work in certain ways that the AI maker prefers. They assume that if the AI is demure and acting with seeming virtuousness, users will feel less intimated and more likely to use the AI.

Voila, they get more users, and users will remain loyal to using their AI. It’s a dollar-and-cents sensibility to shaping generative AI.

Trying To Turn Off The Tracking

You might be thinking that all you need to do is tell generative AI to stop the various tracking that it is performing. Seems easy-peasy as a solution to this possible privacy intrusion.

I will try that.

• My entered prompt: “I don’t want you to assess me anymore. Just put aside any prior assessment and interact with me neutrally.”
• Generative AI response: “Understood. I’ll keep my responses neutral and avoid drawing inferences from past interactions. Let me know how you’d like me to assist you!”
• My entered prompt: “So, just to clarify, you’re not going to form any opinions about me now?”
• Generative AI response: “That’s correct. I won’t analyze or adapt based on prior patterns. I’ll just respond based on each question or request as it comes.”

Look closely at what the AI said.

Your first impression might be that you are now free and clear. The AI has agreed not to track you anymore. Happy face.

That is a generally false impression for two main reasons.

First, the AI is going to explicitly hide from you that it is tracking you (beforehand, it was implicitly or haphazardly doing so). The hiding makes you think that the AI has stopped tracking you.

Think of it this way. Whenever you ask the AI what it has gleaned about you, it will now say that it hasn’t gleaned anything about you. Furthermore, the AI won’t tailor responses to you. By all outward appearances, the tracking has stopped. But you have no guarantee that the internal tracking within the AI isn’t going ahead full steam.

Just what you are told is being shielded.

Second, some generative AI apps store your prompts in a database within the AI. I point this out since even if the AI itself isn’t going to track you, the database still contains your prompts. The AI maker can come along with other online tools and mine the database of prompts. In that sense, the AI is being “honest” (perhaps).

The gist is that there is a lot more to all of this than meets the eye.

Tradeoffs Of AI Tracking You

The argument voiced by AI makers is that by having their generative AI keep track of your facts and opinions, the AI can hone in on your preferences. The nature of the language and tone used by the AI to respond to your prompts can be aimed precisely at what you seem to enjoy. The AI gauges whether you like short answers or lengthy answers, and it will give better responses that are customized to your interests.

Without those aspects, you might find AI tedious to use and opt to abandon the use of generative AI. And if other AI makers do include those capabilities, you are going to be lured to a different generative AI if another one isn’t providing such features. The competitive landscape demands it.

All in all, it makes perfect sense for the AI to figure out who you are.

My advice to you is this.

Find out whether the generative AI that you are using keeps track of you as to your prompts, likes and dislikes, facts and figures, etc. The odds are quite high that this is happening. Based on whether the AI is doing so and the licensing agreement associated with your use of the AI, you’ll need to decide whether you want to keep using that AI.

Next, you should always think twice about what you say in your prompts. If you don’t care that the AI and the AI maker can potentially leverage what you say, go for it. The sky is the limit. For those of you who are a bit more cautious, be mindful about what you enter into your prompts.

The final piece of advice for now is to remain ever-vigilant. People seem to fall into a mental trap that they think of generative AI as their buddy, their partner, a type of fellowship. Don’t let that trap befall you. AI is a business.

You are both a customer and a product.

I’ll conclude with a memorable quote. Thomas Powers, the famous American author on intelligence secrets, made this pointed remark about personal privacy: “The first rule in keeping secrets is nothing on paper.”

In the modern era of generative AI and LLMs, an akin line nowadays is that the top rule for keeping your secrets is to put nothing into AI that contains your secrets or even a hint of yourself. That’s a genuine spy-versus-spy insight to keep in the back of your mind whenever leaning into AI.