Insurance assumes a person

The enthusiasm for putting agents into decision-heavy functions is well-founded, and the firms moving fastest are reading their own economics correctly. Underwriting, claims, credit, eligibility, triage — these are the workflows with the highest cost per unit of human judgment, and much of that judgment is genuinely routine. The same call, made the same way, thousands of times. Automating it is where the money is, and we would rarely tell a client to leave it alone. What almost nobody has done is ask where AI agent liability sits once the deciding party is not a person.

The instruments underneath those functions were all built on an assumption so basic that nobody bothered to write it down. A person decided. Professional liability cover, errors-and-omissions arrangements, indemnities, the allocation of responsibility in a vendor contract — whether any of them responds turns on their actual wording, and that is a conversation for your broker and your counsel rather than for us. What we will say is that the tradition those instruments grew out of assumed a decision-maker who had reasons, who could be examined about them, and against whose conduct some standard could be applied. Remove that decision-maker and nothing breaks loudly. The machinery just stops having a clear subject, and the ambiguity sits there, unresolved and unbilled, until something goes wrong.

Unpriced, not mispriced

Begin with the part that is easy to say and hard to sit with: this is barely priced. Products aimed at AI performance and liability exist and are being written, and the people whose profession is pricing risk have not been slow. But pricing runs backwards through a loss history, and for autonomous decisions at volume there is not yet much of one. How often, how badly, under what conditions, correlated with what — that record is built out of years of claims, and these systems have not been deciding long enough to produce one. So the exposure attached to an agent making thousands of consequential calls is not so much mispriced as priced on thin evidence, which is a different and less comfortable condition. Mispriced risk gets corrected at renewal. Risk nobody has the history to price gets discovered.

Human error is uncorrelated

The sharpest problem here is not a legal one at all. It is statistical. A thousand people working through a thousand files each get most of them right and go wrong independently. One is tired, one is new, one misread a field, one has a bad instinct about a particular kind of case. The errors scatter. That scatter is not incidental to how the risk gets carried — it is the whole reason the aggregate is predictable, and predictability is what makes something insurable in the first place. Independence has been doing quiet, load-bearing work in every arrangement ever built around human judgment at scale.

An agent does not scatter. One agent making a thousand decisions makes the same mistake a thousand times — the same weighting on the same feature, applied to the same class of case, with perfect consistency. It is right in exactly the ways it is right and wrong in exactly the ways it is wrong, and it is wrong at machine speed, in one direction, until someone notices. Systematic error is not a larger version of random error. It is a different distribution. Instruments built to absorb the first do not behave sensibly against the second, and the discovery that they don't rarely arrives as a trickle. It arrives as one event with a thousand names attached.

Where AI agent liability lands

Ask where responsibility sits and you get three parties in the room: the organization that deployed, the vendor that supplied, and the party affected by the decision. We are not going to tell you how that resolves — that is a legal question, it will differ by contract and by jurisdiction, and anyone offering you a confident answer today is guessing. What we will say is narrower and is about your own position rather than the outcome: two things are within your control beforehand, and an organisation that has them is better placed in any version of the conversation than one that does not. How much oversight was genuinely designed in. And whether anyone can reconstruct why a decision came out the way it did.

Oversight is where most designs are quietly weakest. "A human approved it" is doing enormous work in the current generation of agentic workflows, far more work than the humans in question could survive being asked about. A reviewer clearing four hundred agent recommendations in a shift is not exercising judgment. They are providing a signature. That distinction is invisible on an architecture diagram and extremely visible the first time someone asks the reviewer what, specifically, they were checking for.

The shape of the exposure also depends on which way the system can be wrong. We have argued before that a claim is a decision with two opposite ways to be wrong, only one of which is visible in the numbers most organizations watch. Approving what should have been declined leaves a trace. Declining what should have been approved usually leaves nothing — no complaint, no file, no line in any report — and it is the invisible direction that accumulates, quietly and at scale, into the thing you eventually have to answer for. The closer a decision sits to something that cannot be undone, the less any of this stays theoretical. An agent with a wallet is the clean case, because money that has moved has moved, but irreversibility is a property of far more decisions than payments.

It is tempting to wait for the rules to settle. They will settle, and then they will move again — deadlines shift and timelines get extended, and the compliance calendar you planned around turns out to be a moving object. Accountability to the party affected by a decision does not move on that schedule. It is present from the first decision the system makes.

Reconstructability is the asset

Which suggests a different place to spend the effort. Whatever the eventual allocation of responsibility turns out to be, and however long it takes to arrive, the organization that can show what the system did, why it did it, and what oversight actually existed at that moment is in a materially different position from the organization that cannot. That is true in a courtroom. It is equally true in a regulatory conversation, in a renewal negotiation, and in the far more common case where an affected party simply asks for an explanation and the quality of the answer decides whether the matter goes further. Reconstructability is not documentation overhead. It is what converts an open-ended exposure into a bounded one.

The second half is harder and less popular. Decide deliberately which decisions keep a human in them, and be honest about which humans those are. A human in the loop only counts if they are positioned to disagree — with the time, the information, the standing and the incentive to say no to the machine and have that hold. Most designs place a human where they are positioned to approve. That is not the same thing and it does not do the same work — not as governance, whatever any particular rule may or may not accept. Authority that has been transferred is not restored by a checkbox.

What you automated versus what you still owe

The case for automating decision-heavy work is sound, and we have made it to clients ourselves. The cost of human judgment in those functions is real, much of that judgment repeats, and the returns justify moving before every question is answered. That is a defensible position. It stops being defensible only if you mistake it for a complete one. You can automate the decision. You cannot automate being answerable for it — and right now, in most of the organizations moving fastest, almost nobody can say where that lands.