Trust Is the Wrong Question

Last week, Matt Shumer — a builder whose work and judgment I generally respect — posted something that has now been seen nearly six million times. An AI coding agent, he reported, "just accidentally deleted almost ALL of my Mac's files" — and he concluded that this is why he trusts a rival company's model a thousand times more. (The post is here.)

Let me start with the human part: losing your files is awful. The frustration is real, the venting is understandable, and nothing below is a defense of an agent torching a filesystem. I've felt that lurch. Anyone who works with these tools has felt at least the fear of it.

But the conclusion — therefore I trust the other model more — deserves a closer look, because millions of people just absorbed it, and I think it teaches exactly the wrong lesson.

You cannot trust an AI

Not because AI is untrustworthy. Because trust is the wrong category.

Trust is a concept for relations between persons. When I trust you, I rely on your character: your intention to keep your word, your fear of shame, your stake in our relationship, your capacity to feel bound by a promise. Every element of trust presupposes a party with something at stake — someone who can choose to honor or betray, and who carries the consequences either way.

A model has none of this. It has no promises, no character, no stake, no tomorrow. As I argued in An Ecosystem of the Accountable, you cannot fine a model, jail it, or meaningfully scold it — and for precisely the same reasons, you cannot trust one. Trust and accountability are two sides of the same coin, and a model can hold neither.

So "which model do I trust?" is not a hard question with a debatable answer. It is a category error, like asking which hammer you trust. What you actually do with a model is something else entirely: you grant it a mandate — a scope of action, under your authority, inside bounds you set.

And mandates, unlike trust, are entirely in your hands.

The permission that nobody blames

Here is the detail that the viral framing skips over — and it requires separating two components that the framing collapses into one.

The model proposes. It emits an intention: run this command, edit that file. The harness — the coding agent shell around it — is what actually executes, and the harness is where the guardrails live: the approval prompt before every shell command, the allowlists, the sandbox. Destructive actions are gated there by default, because the people who build these tools know what a filesystem is worth. Every mainstream harness also offers the opposite: an explicit, deliberately-named dangerous mode — the auto-approve flags the community affectionately calls YOLO mode, with official names about as subtle as "dangerously skip permissions." You cannot wander into it. You opt in.

I don't know the exact configuration behind that tweet, so I'll say it generally: for an agent to be in a position to delete nearly everything, permissions were opened at the harness level — whichever harness, whoever's model was driving. That opening is a mandate decision. It is the moment a human said: act on my machine, at this scope, without checking with me.

Which makes the stated remedy — switch models — miss the layer where the incident was actually decided. The model and the harness are different components, often from different vendors entirely; swapping the model leaves the opened gates exactly as open as before.

And let me be fair here, because this is a pattern, not a person. In the raw moment after losing your files, everyone vents, and nobody drafts their tweet like a post-incident review — that's true of experts too, and it happens to the best of us. Expertise doesn't exempt anyone from having lessons left to learn; it just raises the stakes of which lesson gets learned. My guess is Shumer took away far more from the incident than one frustrated post could carry — and by putting it out there, he's given a few million people the chance to take a lesson too. I just want it to be the right one: the incident was decided at the harness and the mandate, not at the logo on the model.

Blaming the model is not just imprecise — it quietly deletes the decision-maker from the story. The files were lost inside a mandate someone chose to grant, on a machine whose backups someone chose to maintain or not. Those choices had an author, and the author was not the model. This is the whole of Response-Ability in one incident: bounded mandates, contained blast radius, and answering for the run are duties that sit with the human who presses enter — and they do not transfer to the software when it misbehaves.

Switching brands is not a lesson

Which brings us to the actual conclusion drawn: trust the other company's model more.

Picture a woodworker who removes the blade guard from his table saw — it slows him down, and he knows what he's doing. One afternoon the work slips, and he loses a thumb. His takeaway, announced to everyone: never buying that brand of saw again.

The next brand's blade spins just as fast. The guard is still in the trash.

We see the error instantly in the workshop: the incident was decided at the guard, not at the logo on the blade. Brand tells you approximately nothing about the causal chain — which ran through a safety mechanism deliberately removed, a hand placed inside an unbounded cutting radius, and a risk knowingly accepted for speed. Same hands, same missing guard, different brand: same thumb.

Swapping model brands after an incident is the same move. The causal chain ran through a granted permission, an absent boundary, and an unexamined blast radius — all of them at the harness and the human, none of them at the logo. The rival model inside the identical unbounded mandate is not a thousand times safer. It is one incident away from the identical tweet.

Model quality is real, and differences between models are real. But they move the odds inside your mandate; they do not replace the mandate. A better model in an unbounded run is just a longer wait for the same lesson — a sharper blade is a genuine upgrade, and it still will not guard itself.

The right question

So retire "which AI do I trust?" and replace it with the questions that actually govern the outcome:

What may this agent touch? What is reversible if it goes wrong? What must it never do without me? What record will exist of what it did? And when something breaks anyway — because variance is real — who answers?

Notice that every one of these questions has the same subject, and it is not the model. The answers live in your configuration, your backups, your bounds, your review — your principles, held and enforced by you. That is what "humans are in charge" means in practice: not a mood of confidence about the machines, but a structure of authority around them, with a named person at the top of it.

And there is genuine good news in this, which is where I want to end. The fact that you cannot trust an AI means you never have to. Trust is a vulnerability you extend to persons because you have no choice; with agents, you have something better than trust — control. Mandates you write. Bounds you set. Records you keep. A well-run agent doesn't need your faith, and a well-run principal never offers it.

The model doesn't matter. Your accountability does.