# We Trust AI Agents That Break Every Day

Shy Ruparel, Temporal | AI Engineer NYC 2026 | 21:32

Source: https://www.youtube.com/watch?v=mJ7q8CuVmbw
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/we-trust-ai-agents-that-break-every-day
Published: 2026-10-29
Tags: human-in-the-loop, observability, reliability, workflows

## TL;DR
- AI agent use has grown faster than production adoption, with 80.8% of surveyed engineers using agents daily and 49.1% running them in production.
- Teams often trust agent output even though agent systems fail frequently, so production systems need to recover from failures instead of assuming they can eliminate them.
- Temporal's durable execution lets a payout workflow retain state, approval signals, tool results, and agent output across worker crashes without repeating completed work.

## Summary
Shy Ruparel presents four gaps between experimenting with AI agents and operating them in production. Engineers use agents daily at much higher rates than they ship them, and many trust the output even though their systems still fail every day. Large and regulated companies also take much longer to move prototypes into production because they need approvals, audit trails, and payment safety. The largest productivity blockers are tracking state, debugging, and recovering work after crashes. Ruparel demonstrates a nine-stage payout workflow that uses a local Qwen model for fraud investigation and human approval. He kills the workers during the workflow, approves the payout while no worker is running, and restarts the workers. Temporal replays the workflow from recorded history, keeps completed activities and signals, and avoids repeating the agent's tool calls or spending tokens again. The demo shows how durable execution moves recovery behavior into the workflow platform instead of forcing engineers to build separate state tables and reconciliation code.

## Key ideas
### Financial AI work spans systems, people, and days
[00:37](https://www.youtube.com/watch?v=mJ7q8CuVmbw&t=37s)
Ruparel says finance is adopting AI as part of normal business operations, but financial processes are difficult because they are long-running workflows rather than isolated model calls. Loan origination, KYC and AML, fraud review, and payments can span multiple days, systems, and people. They also involve real money, so a failure can leave funds reserved, lose an approval, or create work that must be reconciled manually. An agent placed inside one of these processes inherits all of those operational requirements. The problem is therefore larger than getting a model to produce a useful answer. The surrounding workflow must preserve progress and handle interruptions.

### Daily agent use is far ahead of production deployment
[03:15](https://www.youtube.com/watch?v=mJ7q8CuVmbw&t=195s)
Temporal's survey of 554 engineers and engineering leaders found that 80.8% use agents in their day-to-day work, compared with 47.3% in the previous year's survey. Only 49.1% said they run agents in production or that agents are core to how they ship. Ruparel describes this as a roughly 30 percentage point gap between using agents and operating them as part of a production system. The people surveyed were experienced industry professionals across engineering, AI engineering, data engineering, platform teams, and IT leadership. The gap is therefore not explained by the survey being focused on hobbyists. Something breaks between experimentation and dependable deployment.

### Trust in agent output coexists with frequent system failure
[04:33](https://www.youtube.com/watch?v=mJ7q8CuVmbw&t=273s)
The survey found that 85.5% of respondents trust agent output at least somewhat. At the same time, 41.1% encounter agent issues daily or more, and 16.4% encounter them hourly. Ruparel's point is that these findings can both be true. People may believe an answer is useful while the system that produced it still crashes, loses state, or needs intervention. Teams that described themselves as successful were 1.7 times more likely to report continuous issues, while agent count and issue frequency were not correlated. Ruparel argues that production systems should account for failure rather than aim for zero failures. The practical goal is to keep failures from creating a cost or losing work.

### Enterprise deployment slows down around control and audit needs
[07:18](https://www.youtube.com/watch?v=mJ7q8CuVmbw&t=438s)
Only 51.3% of surveyed teams said they can move an AI prototype to production-ready code in hours or faster. At companies with more than 5,000 employees, the process often takes months, with about one in five respondents reporting that timeline. Ruparel connects the slower pace to banks, insurers, and payment companies that need approvals, audit trails, and systems that do not lose payments. The survey does not identify exactly where the time goes, but he reads the difference as work required to reach a reliability and trust threshold. The same tools can therefore produce very different delivery times depending on the organization's size and the consequences of failure.

### State tracking and debugging are larger blockers than the models
[08:55](https://www.youtube.com/watch?v=mJ7q8CuVmbw&t=535s)
When respondents were asked what limits productivity, tracking state across services ranked first at 35.7%, followed by isolating and debugging issues at 30.9%. Another 22.2% said they lose progress and redo work because of crashes and restarts. Only 13% were very confident that they could observe and debug AI workflows at scale. Ruparel contrasts basic observability with recovery: logs and traces can show that an agent failed, but they do not automatically restore the work that came before the failure. In that situation, a person may have to inspect a trace and repair state by hand. The survey also found that YouTube was the most common place respondents sought help when an agent broke.

### Durable execution records every side effect and result
[10:53](https://www.youtube.com/watch?v=mJ7q8CuVmbw&t=653s)
Ruparel's demo uses a nine-stage payout workflow with validation, fund reservation, an FX quote, an agent fraud investigation, and human approval. The agent examines customer information and payouts, then drafts a brief for a reviewer. In a Temporal workflow, deterministic code decides what happens next, while non-deterministic or side-effecting actions such as LLM calls, tool calls, and bank API calls run as activities. Each activity has a recorded result in event history. Human approval is recorded as a signal too. If a worker dies, another worker can resume from the last successful activity. A call that was in flight can retry, while completed work stays completed.

### The payout survives stopped workers without repeating agent work
[15:55](https://www.youtube.com/watch?v=mJ7q8CuVmbw&t=955s)
Ruparel sets up a $2,500 payment to a new recipient and adds an unresolved account alert so the local model investigates it. He stops the workers before approving the transaction, then records the approval while no worker is running. When a worker starts again, it picks up the signal and continues from the previous successful event. The event history shows that each AI activity ran once, the approval arrived while workers were stopped, and the bank submission also continued on a new worker. The workflow did not restart from the beginning, and the agent's tool calls and generated brief did not consume tokens again. The payout waited while there were zero workers.

### The workflow owns money and state while the model stays bounded
[18:12](https://www.youtube.com/watch?v=mJ7q8CuVmbw&t=1092s)
Ruparel says the demo handled state management, debugging, cost from retries, approvals, long-running work, and crash recovery. Workflow state was stored as ordinary fields and rebuilt through replay, without a separate database or cache in the demo. Retry behavior came from configuration rather than a loop in the business code. The workflow owned the money and the state, while the model had a bounded attempt to help with the investigation. If the model failed or became unavailable, the payout could continue without its brief. Human approval could also wait for an arbitrary length of time. Ruparel's argument is that recovery belongs with the recorded workflow history, so the same history can support both debugging and replay.

## Notable quotes
- "So, the goal isn't zero failures, right, when we're using agents in production. It's just how do we make it so these failures don't cost us anything." (06:36)
- "Durable execution is a temporal concept that code essentially should run to completion even when the process doesn't." (13:02)
- "Each of our AI activities ran once. They attempted once. They worked once. And they didn't run again." (17:43)
- "The agent itself gave us advice. The workflow itself owned the money and the state." (19:31)

## Tools & references mentioned
- Temporal
- KPMG
- State of Development Survey
- Qwen 9B
- OpenAI Codex
- Replit agents
- Keycard

## Who should watch
- You are putting an AI agent inside a payment, fraud, approval, or other workflow that can run for hours or days.
- Your agents work in development, but crashes, restarts, lost state, and repeated model calls make production deployment difficult.
- You need human approvals and external API calls to survive worker or deployment failures without building a separate recovery system.

## Editor's note

Shy Ruparel says the largest productivity blockers are tracking state, debugging, and recovering work after crashes. Kitaru records each model call and tool result from a real agent run, then replays the agent's code against that recording with the same inputs and responses. Engineers can test a change against the failed run without touching real systems.

Written by the AIE Talks editors (the Kitaru team), not by the speaker.

## Related talks

- [Scaling AI Agents Without Breaking Reliability](https://aietalks.com/talks/scaling-ai-agents-without-breaking-reliability) (Preeti Somal, Temporal, 15:01)
- [Building Durable, Production Ready Agents](https://aietalks.com/talks/building-durable-production-ready-agents) (Cornelia Davis, Temporal, 1:18:30)
- [From Stateless Nightmares to Durable Agents](https://aietalks.com/talks/from-stateless-nightmares-to-durable-agents) (Samuel Colvin, Pydantic, 22:13)
- [We Kept Rebuilding Our AI Agent. Here's What Survived](https://aietalks.com/talks/we-kept-rebuilding-our-ai-agent-heres-what-survived) (Fuad Ali, Arize AI, 20:06)
- [Building Durable Agents with Workflow DevKit & AI SDK](https://aietalks.com/talks/building-durable-agents-with-workflow-devkit-ai-sdk) (Peter Wielander, Vercel, 1:09:49)
