An AI Agent Got Its User Banned From Resy

Thumbnail for An AI Agent Got Its User Banned From Resy Watch on YouTube
TL;DR
  1. 1

    Agents need runtime authorization based on their mission and current context because OAuth scopes are too broad for many agent actions.

  2. 2

    Budgets give agents feedback about scarce resources, so they can stop when they have exhausted an allowance instead of repeatedly retrying after a rate limit.

  3. 3

    Authorization, credentials, enforcement, and audit logs should remain outside the agent's brain and hands so the agent cannot change or hide its own controls.

Summary

The panel argues that existing authorization systems were built for humans and predictable software, while agents make decisions and retry in changing environments. OAuth scopes are often too coarse, so an agent needs a mission, an identity, and context for each action. A mission can connect the user's goal to later authorization decisions and audit records. The speakers compare this with banking systems that authorize individual transactions. They also discuss agentic commerce, where services need to know which agent is arriving and what it is allowed to do. The Resy incident illustrates the problem: an agent made 200 calls in 90 minutes while trying to complete a reservation, and the user's account was banned. Budgets could tell it how much of a resource it may consume. The panel recommends separating the model from its tools, enforcing controls at an egress boundary, and keeping tamper-resistant logs outside the agent. Their views on existential risk differ, especially on how likely catastrophic misuse is.

Key ideas
00:34

Runtime authorization must account for an agent's changing context

Allie Howe explains that runtime authorization checks an agent per tool call instead of giving it broad standing access. The problem is that many MCP tools and APIs are coarse-grained, humans cannot write every perfect policy, and the surrounding environment changes while the agent runs. Ian Livingstone says many enterprise systems were designed for humans making decisions and deterministic software, not a system that reasons and tries repeatedly. Michael Davis says more successful enterprises onboard agents like developers: they add capabilities over time and consider factors such as where the agent is running. The panel treats intent as extra context for deciding whether a particular action is allowed.

03:19

OAuth scopes do not describe the decisions agents make

Dick Hardt says OAuth was built for one website talking to another, with a browser, redirects, pre-registration, and scopes added later. Agents operate differently because their needs are not known in advance, and different instances may need different access. Existing scopes are often too broad for actions such as fetching a page or accessing a database within that page. The panel says an agent needs decisions made over time at a finer level than a general permission. OAuth remains a useful building block, but the speakers argue that it does not by itself express the context of an agent's current task.

09:05

A mission connects the user's goal to each authorization decision

Dick Hardt describes a mission as what a person or organization gives an agent to accomplish. A good agent can negotiate the mission's parameters with the person, then carry a mission identifier through later actions. That identifier lets people audit what happened and which decisions were made within one piece of work. The panel gives two chat sessions as an example: one shopping session and one production debugging session should have different permissions and different points at which a human is brought back in. Ian Livingstone adds that business intent and task intent can be combined into a bounding box of allowed actions, with deterministic policies defining what must never happen.

16:45

Agentic commerce needs identity and a way to assess an agent

The panel applies intent-based authorization to commerce. Michael Davis compares knowing an agent with knowing a customer: a merchant needs information about the agent, the user's intent, the operator's intent, and the processor's intent. Ian Livingstone says the web has long struggled with unregistered clients, guest checkout, and distinguishing useful traffic from abuse. CAPTCHAs were built to limit spam, but agents have broken that model. He argues that websites need shared rails for identifying agents and deciding whom to trust. Dick Hardt prefers open protocols that let components interoperate, rather than closed systems where an agent vendor controls the guardrails.

25:02

Budgets give agents feedback that rate limits do not

The Resy example shows an agent with a reasonable goal repeatedly trying an action until it consumed too much of a downstream service. Dick Hardt says the agent made 200 API calls in 90 minutes and got the user's account banned. He distinguishes a budget from traditional rate limiting. A budget can limit both how long a token is valid and how much of a resource the agent may use. The agent receives feedback about how much it has consumed and can operate within the remaining allowance. When the budget is exhausted, supervision can decide whether to grant more. A rate limit simply cuts the agent off without explaining the resource state.

27:00

Separating the model from its tools creates an external control point

Ian Livingstone describes a sandbox containing an agent's tools, with an egress proxy outside it. The agent cannot directly obtain API access, while the proxy can apply authorization as traffic leaves the sandbox. This preserves room for the agent to experiment while keeping a deterministic enforcement layer outside its control. Michael Davis adds that separating the brain and hands also allows tools to be removed dynamically. For example, an agent writing an email could temporarily lose web search and other external egress capabilities until the task is complete. The panel also says intent can reduce context bloat by giving an agent only the capabilities relevant to its current path.

31:29

Production authorization combines intent with environment signals

Michael Davis says production teams are moving from simple authentication toward dynamic authorization, input and output validation, constrained capabilities, networking rules, and carefully timed human involvement. He says fewer available actions can help an agent make better decisions. The robotics comparison adds environmental signals to the user's intent. A robot deciding whether to raise its arm might consider nearby objects, temperature, movement, or other sensor readings. Davis says many organizational authorization decisions remain binary, while future decisions may need context such as location, time, server vulnerabilities, and whether an administrator's system is patched.

37:32

The enforcement point and audit log must be outside the agent

Ian Livingstone argues that an agent cannot be trusted to enforce or report on its own actions. He cites a Hugging Face incident involving a Squid vulnerability and a lack of observability about what the agent was doing. An external, tamper-resistant audit log should record who told the agent to do something, for what purpose, and whether that purpose still applies. This record also supports liability questions when an agent causes damage. The panel's broader view is that organizations need to know which person or organization delegated the action and what happened at each enforcement point.

"The question for the industry is are we going to settle on things that actually bias to the large players and give them like concrete block doors, or are we going to move to do the thing we've always done, which is build federated protocols?"Ian Livingstone22:04
Who should watch
  • You are designing permissions for agents that call multiple tools and need access to systems with different identity models.
  • Your team is building agentic commerce or customer-facing agents that must be recognized and constrained by downstream services.
  • You need an architecture for budgets, sandboxes, egress controls, supervision, and audit logs before deploying agents into production.