# Deploying AI Inside a Bank Is Not an Engineering Problem Paden Gayle, Applied AI Engineers | AI Engineer NYC 2026 | 15:07 Source: https://www.youtube.com/watch?v=3ZlQk4-knWM Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks. Page: https://aietalks.com/talks/deploying-ai-inside-a-bank-is-not-an-engineering-problem Published: 2026-10-13 Tags: deployment, enterprise, evals, human-in-the-loop ## TL;DR - A working model can still fail to ship when data, security, risk, audit, and operational owners have not approved the intended workflow. - When a customer cannot provide a specification, engineers should shadow the work, draft a concrete workflow, and shrink it with the people who own its controls. - The first deployment should use the least authority that creates value, while preserving evidence about inputs, outputs, errors, reviewers, and recovery. ## Summary Paden Gayle argues that deployment inside a bank depends on decisions outside the model and application code. A document exception assistant may produce useful drafts, yet still lack an approved environment, permitted data use, risk review, audit trail, or accountable owner. Gayle treats discovery as part of engineering. In the first week, the team should find the people who own data, model risk, third-party risk, audit, production, and the workflow itself. When there is no complete specification, engineers should shadow the work, write down the actual sequence, and reduce it to the smallest useful scope. Gayle recommends starting with suggestions or drafts before granting authority to act. An evidence pack should record intended use, data lineage, failure modes, results by segment, controls, and the original inputs and outputs. His closing checklist helps teams identify unresolved approval decisions, but he is explicit that it does not predict approval. ## Key ideas ### A successful demo does not establish a route to production [00:00](https://www.youtube.com/watch?v=3ZlQk4-knWM&t=0s) The hypothetical assistant reviews client documents and drafts exceptions for an analyst. The demo looks useful, but security still needs to know where it will run, data owners need to approve the document use, and risk needs to know who reviews a wrong exception. Gayle says each answer can change what the team is allowed to build. The model can be fine while the system remains unable to ship because the people responsible for the design have not seen it or approved its consequences. ### The first week should identify people who own the decisions [01:08](https://www.youtube.com/watch?v=3ZlQk4-knWM&t=68s) Gayle groups deployment dependencies around data, model risk, third-party risk, and audit. For data, the team must find the source owner, the person who can grant access, and the person who can approve the intended use. For audit, it must capture which input was used, which configuration produced the output, what the reviewer saw, and who approved the action. He wants discovery calls to end with names, decisions, or next conversations. By Friday, the decision owners should know the project exists. ### An unnamed dependency becomes a concrete implementation task [02:18](https://www.youtube.com/watch?v=3ZlQk4-knWM&t=138s) An architecture diagram can show documents entering a model and an analyst reviewing the result, while leaving unanswered who owns the documents, the model service, the analyst workflow, or the required review. Gayle recommends walking through that diagram with a sponsor and then confirming the answers directly with the owners. Questions include where data lives, who can approve reading it, where production will run, who administers that environment, and who signs the risk review. The design is easier to change while these dependencies are still visible. ### Shadowing the work produces a specification when the customer has none [05:47](https://www.youtube.com/watch?v=3ZlQk4-knWM&t=347s) Gayle's method is to shadow, draft, and shrink. He describes following analysts through the systems they open, the judgments they make, permission requests, and what happens when a document is missing. For the exception assistant, the initial workflow is to review documents, draft exceptions, and let the analyst submit. The team then makes each line concrete: which documents are available, whether processing is approved, what the model produces, and who checks and submits it. Control owners can reject document types, change the reviewer, or reject the output. ### Small increments expose new approval and control requirements [07:47](https://www.youtube.com/watch?v=3ZlQk4-knWM&t=467s) The hypothetical system can begin by summarizing a document the analyst already has, then read from another system, draft a customer-facing response, and finally send that response automatically. Each increment adds a different data path, action, owner, or failure-recovery question. Gayle recommends asking which increment creates useful value now. If drafting inside the analyst's existing workflow solves enough of the problem, the team should start there after confirming that the existing control actually fits. ### Autonomy should start at the lowest authority that creates value [08:42](https://www.youtube.com/watch?v=3ZlQk4-knWM&t=522s) Gayle describes four levels: a system can suggest a recommendation, draft something for an analyst to check and submit, act with approval for each action, or act within pre-approved limits. Each step changes the consequences of an error and the controls that reviewers must trust. For the exception assistant, he prefers a design where the model highlights a likely exception, points to source evidence, and prepares a form. The analyst checks the evidence, corrects the form, and chooses whether to submit. ### Operational evidence must include what actually happened [11:14](https://www.youtube.com/watch?v=3ZlQk4-knWM&t=674s) For each evaluation run, Gayle recommends preserving the input prompts, model version, configuration, output, reviewer action, and related controls. Calling the model again later may produce a different answer, so a rerun cannot reconstruct the historical decision. The evidence pack covers intended use, data lineage, failure modes, results by segment, and controls. It should also name the detection path, fallback, and owner for failures such as invented exceptions, unreadable extraction, or missed exceptions. ### Aggregate results can hide unsupported document groups [13:12](https://www.youtube.com/watch?v=3ZlQk4-knWM&t=792s) Gayle says results should be separated into groups such as clean documents, scans, rare exceptions, and new document types. Teams should measure errors and control behavior within each group and lead with the weakest result. An average can hide a serious gap in one segment. If a group has little evidence, the team should state that limitation and restrict its use until more evidence exists. This gives reviewers a clearer account of what the results support. ### The week-one checklist identifies work still blocking delivery [13:37](https://www.youtube.com/watch?v=3ZlQk4-knWM&t=817s) By Friday, Gayle wants five answers: whether the data owner is engaged, whether the production environment and administrators are named, whether the sponsor can describe the workflow without saying AI, whether the smallest useful scope fits confirmed controls, and whether the risk signer and prior review are understood. Four or five yeses suggest a credible path to ship. Three means unresolved decisions need owners and dates. Zero to two means the team should resolve the decisions before committing, possibly by shrinking or redefining scope. ## Notable quotes - "The demo worked. The model was fine. However, it still did not ship." (00:53) - "Notice what I want to leave with. It's either names, decisions, or next conversations." (05:24) - "I start with the least authority that the problem solved then ask what task can be proven as evidence to give the system more." (08:43) - "Calling the model again later may produce a different answer. So we preserve what actually happened." (12:53) - "Discovery is the engineering. Draft the spec, then shrink it. Give the system only the authority it needs. Build the evidence pack in sprint one." (14:28) ## Tools & references mentioned - Accenture - Bloomberg - Google - O'Reilly ## Who should watch - You are building an AI workflow in a regulated company and the demo is ahead of its approvals. - Your customer cannot provide a complete specification, so you need a practical way to discover the real workflow and its owners. - You are deciding whether an assistant should recommend, draft, or take action, and need evidence to support that boundary. ## Editor's note Paden Gayle says calling the model again later may produce a different answer, so a rerun cannot reconstruct the historical decision. Kitaru records each agent run as a session, including every model call and tool result. Replay then runs the agent's code against that recording, so a team can inspect a changed run without touching real systems. Written by the AIE Talks editors (the Kitaru team), not by the speaker. ## Related talks - [Why Your Enterprise Tech Stack Isn't Ready for AI Agents](https://aietalks.com/talks/why-your-enterprise-tech-stack-isnt-ready-for-ai-agents) (Christopher Lovejoy, Anthropic & Saul Howard, Anterior, 19:15) - [What We Learned Deploying AI within Bloomberg's Engineering Organization](https://aietalks.com/talks/what-we-learned-deploying-ai-within-bloombergs-engineering-organization) (Lei Zhang, Bloomberg, 18:21) - [How do you diffuse AI into the real world?](https://aietalks.com/talks/how-do-you-diffuse-ai-into-the-real-world) (Varun Shenoy, Long Lake, 17:46) - [Build for the Memo, Not the Demo](https://aietalks.com/talks/build-for-the-memo-not-the-demo) (Shawn Chan, China Resources Holdings, 24:23) - [The AI Pivot](https://aietalks.com/talks/the-ai-pivot) (Brittany Walker, CRV & Chris White, Prefect & Bryan Bischof, Hex, 35:16)