(AI generated at Canva.com)

Yes, maybe foundation models really are taking your data. Or maybe they aren’t.

But enterprise paranoia that AI companies might be using the proprietary data their customers share with models to learn how to compete with them was certainly given a big boost this week.

Effectively OpenAI’s announcement that its AI had solved a hitherto intractable mathematical problem was initially overshadowed by questions about whether it really had. Or at least whether it had done so wholly on its own.

Because two mathematicians working on a related problem had been using OpenAI’s Codex to check their work — and so when OpenAI subsequently announced its own breakthrough, one of them, Tristan Buckmaster, initially expressed surprise at the degree of overlap.

OpenAI immediately denied that its researchers or agents had any access to the mathematicians’ specific user data and, following an internal investigation, subsequently updated its announcement to state that Buckmaster’s Codex prompts could not have influenced the system in any way — including through training.

So — meh?

But whatever the reality, this episode — along with the continued fallout — has caused a lot of General Consternation amongst organizations already somewhat concerned about feeding AI models with the seeds of their own demise — via the drip, drip, drip transfer of proprietary data embedded in prompts and agents.

Especially following Palantir CEO Alex Karp’s recent rant about the dangers of giving your knowledge to voracious AI companies (no, really) or Microsoft CEO Satya Nadella’s essay decrying the concentration of power within a small number of huge tech companies (no, really).

So when we are strong within the walls of our enterprise, we of course want to make sure that little details about our plans, disposition of troops or treasure room locations don’t leak out to our enemies beyond the gate in case they can — through some process of mental gymnastics — recreate the entire basis of our civilization from a trillion scraps of context-free garbage.

And so while I get it I also kind of think — really? Is that the exfiltration point you’re worried about right now? The random hints and scraps leaking through the back gate?

Because while leaking tiny fragments of proprietary knowledge through prompts and model interactions is certainly something to worry about, in many organizations there’s actually a much bigger problem sitting, right there, in the room.

One you’ve welcomed in through the front gate.

Wooden horses are bad for security, duh

Because honestly — has nobody even watched the Odyssey?

I mean this focus on security in the small while failing at security in the large is a real three-millennia problem. One that’s quite literally being played out on the biggest screens in the world right now to huge success. One that shows us exactly where a much bigger risk to your intellectual property sits.

Right there, in front of your face.

In high-resolution IMAX.

Which kind of suggests that people are failing to learn the lessons of history even when they are shown to them in compelling, loud, and awe-inspiring widescreen.

But hey — let’s walk through it together.

For most of the Trojan War, the walls of the city did exactly what they were supposed to do — keep out the Greeks who wanted everything the Trojans had. Effectively the walls provided a clear distinction between the inside and outside — between the people the Trojans trusted and the people trying to rob them.

So long as the walls held, the Greeks were powerless to sack Troy — no matter how many small fragments of intelligence leaked out — because the Greeks could not take what they could not access.

So the Trojans had pretty good infosec.

A secure perimeter. Big gates. A strong middle finger to the rampaging Greeks.

Which is why what they did next was toweringly, awesomely, biblically stupid. I mean we can all see that, right? That it was ‘facepalm’ stupid?

They saw a wooden horse. And they wanted it.

So they opened the gates and brought it inside.

And they gave the Greeks everything.

Not scraps of information. Not clues. The whole shebang.

And that, right there, is a big picture security fuck up.

Cassandra called. Again.

The problem with being a writer is that it’s often tempting to lay the metaphor on thick. To delight in your own conceit. But in this instance the story kind of writes itself.

Because, dear reader, you’re likely sitting inside your security perimeter, protecting all of your data, your processes, and your tacit knowledge. The things that make up your operating model. I mean that’s pretty heady stuff — and so good on you for looking after it.

I also get that you’re worried about some of that leaking out as exhaust — people putting your data, their ideas, their knowledge into AI prompts. Or about agents owned by AI companies getting access to proprietary process knowledge as part of their setup.

So you’re on it. The security in the small stuff.

Stopping scraps of information — the detritus and exhaust of work — leaking out of the back gate. Which is great — even though the highly specific and context-poor nature of this exhaust doesn’t tell anyone much about the shape of the larger operating model.

But you care about that stuff. You sweat it. Your CISO is on fire.

You, like the Trojans, are sitting pretty.

But you’re also a bit antsy. You see the AI promise and you want it.

You’re worried that other companies are using it to get ahead of you. But you don’t have the talent. The products don’t really work as advertised. Your results are… a bit shit.

I mean it gets boring sitting inside the security perimeter just keeping all your stuff safe from exfiltration — making sure that you don’t leak important information to the foundation model companies who want to eat your lunch, sack your operating model, pillage your… processes.

Ahem.

But help is at hand.

Just there, outside the gate.

And if you only let them in, they’ll give you the gift of AI. The AI that lets you look like you know what you’re doing. The AI that will make you a hero. The AI that other executives will envy.

A beautiful FDE with a great big wooden smile.

Beware of geeks baring grifts

Unless you’ve been hiding under a rock recently, I’m sure you will have seen all of the noise around Forward Deployed Engineers or FDEs. Apparently, every AI vendor needs one now.

Cynically this simply highlights the fact that AI companies don’t know what value is in an enterprise context and non-AI companies don’t know how to get value from AI. And so why not put people from the AI company directly inside the enterprise, for a long-term engagement no less, so that they can learn everything they need to know about your company and your industry? Because then they’ll be able to help you with AI, right?

So let’s say that again in a different way.

The AI companies have left a gift outside the gate — a friendly FDE who only wants to help.

So let’s bring them inside the walls. Let them sit in our meetings. Let them read our documents. Let them roam around and ask our people questions.

Give them access to everything they need to understand the minutiae of how we work.

Because they have AI and we want it.

But you also have knowledge. And they want it.

Seems fair, right?

But the exchange is asymmetrical because each organization wants something different.

You want operational uplift inside your organization so you can compete more effectively in your existing market.

But if Nadella and Karp are right, AI companies are on something more of an odyssey — and want industry knowledge they can use to absorb the market itself.

And the funniest part?

The FDE doesn’t need to steal your data. They don’t need to secretly copy your documents. They don’t need to reconstruct your business from fragments. They don’t even need to breach your walls.

Because you invited them in.

Sweating security in the small at the back gate while surrendering security in the large at the front gate.

Which feels like a pretty fucking big blind spot.

Because the lesson of Troy is that walls are kind of useless if you just open the gate.

And so, three thousand years later, maybe it’s worth thinking about that. Yes, there are geeks bearing gifts outside your walls. But before you open the gate, ask yourself who will really be giving things away once you’ve let them in.