ACADEMY
All posts

Who's Responsible When Your AI Agent Makes a Mistake?

A founder's accountability framework for agentic AI — who owns the workflow, who signs off on scope changes, and why 'someone could check the logs' isn't a policy.

Your AI agent just refunded a customer $4,000 it shouldn't have, or emailed a client the wrong contract terms, or pulled data from a system it wasn't supposed to touch. The action already happened. Someone asks the obvious question: who's responsible? If your honest answer is "the AI did it," you have a policy gap, not a technical one, and 2026 is the year that gap stops being free.

This isn't a hypothetical for later. UHY's 2026 Middle Market Survey found that roughly 60% of mid-market firms are now actively using AI, and separately, McKinsey's 2026 State of AI Trust report found only about a third of organizations have reached real governance maturity, while two-thirds name security and risk as the top barrier to scaling agents further. Adoption is running well ahead of the paperwork that's supposed to back it up.

Adoption is outrunning governance: 60% of firms actively use AI, 66% cite security/risk as the top barrier to scaling agents, only 33% have mature governance. Source: UHY 2026 Middle Market Survey; McKinsey, State of AI Trust in 2026
Adoption is outrunning governance: 60% of firms actively use AI, 66% cite security/risk as the top barrier to scaling agents, only 33% have mature governance. Source: UHY 2026 Middle Market Survey; McKinsey, State of AI Trust in 2026

The Legal Answer: It's Never the AI

Start with the part that isn't ambiguous. No court, regulator, or vendor contract treats the model itself as a legal person who can be held liable. Responsibility runs to the company that deployed the agent and the people inside it who built, approved, or operated the workflow. Baker McKenzie's 2026 analysis of legal accountability for AI agents is blunt about this: when business leaders ask if the AI vendor is responsible for their model's mistake, the answer is almost always no, because the deploying company is the one that decided to put the agent into production, on their data, acting on their customers.

The regulatory environment is also getting sharper. In the EU, the revised Product Liability Directive (2024/2853) reaches software and AI systems directly, and member states have until 9 December 2026 to transpose it into national law. Under the directive's presumption-of-causality rule, if you can't show your AI system followed documented safety procedures, a court is allowed to connect the agent's output directly to the harm it caused. In the US, the FTC has been applying its existing Section 5 unfair-and-deceptive-practices authority to AI agent failures rather than waiting for new statute. Different mechanisms, same direction: "we didn't know what it was doing" is getting harder to say out loud in front of a regulator.

Why "Someone Could Look at the Logs" Isn't a Policy

Most founders think they have this covered because a person could, in theory, review what the agent did. That's after-the-fact visibility, and it's not the same thing as accountability. A real accountability structure answers three questions before the agent runs a single action, not after: who approved this workflow existing at all, who owns it day to day, and who gets paged when it does something wrong. If none of those three has a name attached, you don't have an AI policy, you have an AI that nobody owns.

This is the same failure mode covered in more technical depth in AI Agent Guardrails: Human-in-the-Loop Oversight: an agent runs a loop where the model decides the next action on its own judgment, and without an engineered pause before high-stakes actions, a misjudged "done" or a silently-retried tool call becomes a few thousand quiet actions before anyone notices. Guardrails are the technical half of the answer. Ownership is the organizational half, and most companies only built the first one.

Build the Accountability Chain, Not Just the Guardrail

A workable framework has three layers, and each one needs a name, not a department:

  • **The agent** — what it's allowed to do, scoped as narrowly as the job requires. Refund authority capped at a dollar amount, database access limited to read-only where write access isn't the point of the workflow, external actions (sending an email, issuing a payment, posting publicly) gated behind an approval step rather than auto-executed.
  • **The process owner** — the specific person who approved this workflow going live, reviews its outputs on a set cadence, and is the first call when it misfires. Not "the ops team." A person.
  • **The founder or executive sponsor** — who holds ultimate accountability for the decision to deploy agentic AI in that part of the business at all, and who signs off before scope expands (a support-refund agent quietly graduating from "draft a response" to "issue the refund" is a scope change that needs the same sign-off as the original launch).

None of this replaces the technical controls. It answers the question those controls can't: when the guardrail itself fails, whose job was it to have built a better one?

Logging Is Not Optional Anymore

Under the EU AI Act, Article 12 requires high-risk AI systems to log their actions for traceability, and Article 13 requires the system's decision-making to be explainable in terms a non-technical person can follow. You don't need to be an EU-regulated enterprise for this to be the right bar. If your agent takes an action you can't reconstruct afterward — what it decided, what data it used, what tool call it made, and why — you can't defend the decision to a customer, an insurer, or a court, and you can't fix the actual failure point either. At minimum, log every tool call with a timestamp, the input that triggered it, and the output it produced, and keep it somewhere a human, not just the agent's own memory, can query. Community frameworks like CoSAI's AI Incident Response Framework already specify concrete containment steps for multi-agent failures — corpus quarantine, prompt-provenance audit, downstream contagion checks — and they all assume the logs exist in the first place. No log, no audit, no defense.

What This Looks Like in Practice

You don't need a 40-page governance document to start. You need one page per agent in production, answering: what is it allowed to do, who approved it, who owns it, where are its logs, and what's the rollback if it goes wrong. Review that page every time the agent's scope changes, not just at launch. If you're running agents built in n8n or a similar orchestration tool, this is a natural fit for a fixed review checkpoint in the workflow itself, not a separate process someone has to remember to run.

The honest version of "AI accountability" isn't a legal shield you build after something breaks. It's the ownership structure that makes the something-breaks conversation short instead of a scramble: here's who approved it, here's what it was allowed to do, here's the log of what actually happened, here's what we're changing. Founders who can answer that in one meeting keep the incident small. The ones who can't turn a single bad refund into a governance crisis that costs them a lot more than the refund did.

If you're the one making these calls without a playbook, AI Leadership for Founders covers exactly this: how to build the ownership and oversight structures that let you deploy agentic AI aggressively without deploying blind, including the specific decision frameworks for when to gate an agent's actions, who signs off on scope changes, and how to run the incident review before you're forced to.

Go deeper

AI Leadership for Founders

Courses launching soon

Want the full AI Leadership for Founders course, not just this post?

Join the waitlist and get a 20% launch discount the moment we open checkout. No payment now.

Taught by Aditya Jha · 40+ AI products shipped for real clients. No spam, unsubscribe any time.

Or join our free community for AI tips while you wait

AI Weekly Radar

One email a week: the AI tools, tactics, and course drops actually worth your time. No spam, unsubscribe anytime.

Questions about this or which course fits? Email academy@aibootstrapper.com and we'll answer it directly, not with a support ticket.