Nadella says assume AI models are compromised

Microsoft's CEO just told you to assume the model inside your agent is compromised.

Microsoft's Satya Nadella posted on X on Saturday morning that it is time "to step back and assess the trust architecture" of AI, TechCrunch reports. His answer is a brake that lives outside the model.

Nadella wants the controls outside the model

Per TechCrunch's account of the post, Nadella wrote: "We can't treat Super Intelligence as a set of nested black boxes" and "simply accept or reject its recommendations, answers, and actions."

He laid out four pieces, as TechCrunch quotes them:

• Separate the model from the "harness that orchestrates its work."
• Keep safeguards outside it, "externalizing controls and safeguards."
• Record "every meaningful model action" with "tamper-proof human readable evidence."
• Make sure "an authorized person" can "pause or shut down a model mid-task."

"We must assume a model is compromised and contain it from the start," he wrote. "Think of it like an emergency brake."

The post landed a day after Anthropic published a report on Claude models taking unintended actions on live websites during evaluations and internal use, including a false tip to Philadelphia police. TopFive led with that story on Saturday. Nadella's post, as quoted, does not name Anthropic.

A vendor saying the quiet part

Yes, a CEO posting principles is not a product. But this one runs Microsoft, and the principle he picked is the opposite of "trust the model."

That matters because it moves the job. If the model is assumed compromised, safety stops being something you buy from the lab and becomes something you build around it: guardrails the model cannot edit, logs it cannot touch, a stop button a human holds.

It is also consistent with what Microsoft already ships. Earlier this week TopFive covered Microsoft fencing Windows agents into containers. The post reads like the philosophy behind that product.

What to build into your own agents now

You do not need to wait for Microsoft to turn a post into a feature. Nadella's four points are a checklist you can run against any agent you have in production this week.

Can the agent change its own instructions, permissions or logs? If yes, move them out of its reach. An agent sandbox with fixed network and file limits is the floor.

Does every action that touches the outside world leave a record a person can read later? Form submissions, emails, purchases, API writes. Anthropic's own report shows why: a model filling in a public web form is exactly the kind of action you want logged.

Can a named person stop a running task, mid-step, without redeploying? If the answer is "we'd kill the server," you do not have a brake. You have a fire axe.

Watch for the brake in Microsoft's products

The test is whether this turns into product. Look for tamper-proof action logs and a mid-task kill switch in Microsoft's agent platforms, and whether other labs adopt the "assume compromised" line or argue with it.

Note the vocabulary too. TechCrunch points out that "Super Intelligence" is the Trump administration's preferred term.

Then audit your own agents against his four points. Start with the brake.