# OpenAI for VPs of AI + Advice for Building Agents Prashant Mital & Toki Sherbakov, OpenAI | AI Engineer Summit 2025 | 16:52 Source: https://www.youtube.com/watch?v=joHR2pmxDQE Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks. Page: https://aietalks.com/talks/openai-for-vps-of-ai-advice-for-building-agents Published: 2025-03-05 Tags: agents, enterprise, guardrails, multi-agent ## TL;DR - OpenAI describes an enterprise path from AI-enabled employees, to internal automation, to AI features in customer products. - Teams should build agents from primitives first, start with a single-purpose agent, and add complexity only when observed failures require it. - Complex workflows can use specialized agents with handoffs, while simple prompts and separate guardrails handle edge cases and high-stakes actions. ## Summary Toki Sherbakov explains how OpenAI's customer teams help enterprises move AI use cases into production. The usual path starts with giving employees access to AI, then automating internal operations, and finally adding AI to customer-facing products. He describes a Morgan Stanley knowledge assistant that improved from 45% to 98% accuracy through retrieval, fine-tuning, chunking, reranking, classification, prompt engineering, and query expansion. Prashant Mital then defines an agent as a model with instructions, tools, and an execution loop controlled by the model. His advice comes from OpenAI's work with customers and products such as Deep Research and Operator. Teams should begin with raw primitives, start with one agent for one task, and add networks of agents when real constraints demand them. Prompts should stay focused, while guardrails run separately to handle safety, security, reliability, prompt injection, and high-stakes actions. ## Key ideas ### OpenAI moves enterprise use cases through a research, product, and customer feedback loop [00:33](https://www.youtube.com/watch?v=joHR2pmxDQE&t=33s) Toki Sherbakov describes two core engineering groups at OpenAI. The research team builds and deploys foundational models, while the applied team turns them into products such as ChatGPT and the API. The go-to-market side helps customers put those products into their workforces and products. Feedback from deployments then goes back into product improvements and model research. This gives the customer work a direct connection to OpenAI's development process. ### Enterprise AI adoption usually progresses from employee use to products [01:44](https://www.youtube.com/watch?v=joHR2pmxDQE&t=104s) OpenAI sees three common phases in an enterprise customer journey. First, employees use AI regularly and become more AI literate. Next, teams automate internal operations with customized ChatGPT or API applications. Finally, companies add AI to end-user products, where API-based applications are the main approach. The phases do not have to happen in order, but this pattern helps companies decide where a use case fits. ### A business strategy should determine where AI is applied [03:02](https://www.youtube.com/watch?v=joHR2pmxDQE&t=182s) The speaker says companies should begin with their broader business strategy rather than an isolated AI strategy. They can then select one or two high-impact use cases, scope them clearly, and define success metrics and KPIs. After those initial projects, the organization can build wider capability through enablement, centers of excellence, or a shared technical platform. During development, OpenAI may provide workshops, office hours, paired programming, early access to models and features, internal experts, and joint roadmap sessions. ### Morgan Stanley improved a knowledge assistant from 45% to 98% accuracy [06:54](https://www.youtube.com/watch?v=joHR2pmxDQE&t=414s) Morgan Stanley built an internal knowledge assistant for wealth managers to query research reports, live stock data, and other company knowledge. Its initial accuracy was about 45%, while the target was 90%. OpenAI and Morgan Stanley tried improved retrieval, fine-tuning, embeddings, different chunking strategies, reranking, and a classification step. Prompt engineering and query expansion also contributed. The final reported accuracy was 98%. ### An agent combines a model, instructions, tools, and a model-controlled loop [09:00](https://www.youtube.com/watch?v=joHR2pmxDQE&t=540s) Prashant Mital defines an agent as an AI application with a model, usually prompt-based instructions, access to tools, and an execution loop. In each cycle, the agent receives natural-language instructions, decides whether to call tools, runs them, uses their returned values, and produces an answer. The model can decide that it has completed its objective and terminate the loop. This definition separates an agent from a single model call. ### Teams should learn the system with primitives before choosing abstractions [09:54](https://www.youtube.com/watch?v=joHR2pmxDQE&t=594s) Frameworks make it easy to build an agent proof of concept, but they can hide how the system behaves and which primitives it uses. That delays design decisions until after the team's constraints should have been understood. Mital recommends first making raw API calls, logging results and failures, and studying how the task breaks down. A framework or other abstraction can come later when the team is repeating work such as embedding strategies or model graders. ### A single-purpose agent should come before a multi-agent system [11:43](https://www.youtube.com/watch?v=joHR2pmxDQE&t=703s) Mital recommends starting with one agent built for one task and releasing it to a limited group of users. Observation then reveals actual problems, such as hallucinations across a conversation, low adoption caused by latency, or inaccurate retrieval. Teams can add complexity as they learn which failures matter. The goal is a system that works, so complexity should grow in response to evidence rather than appear in the first design. ### Networks of agents use specialization and handoffs for complex workflows [13:00](https://www.youtube.com/watch?v=joHR2pmxDQE&t=780s) For complex requests, several specialized agents can work together as a network. A handoff transfers an active conversation from one agent to another while preserving the conversation history. In the customer service example, GPT-4o mini performs triage, GPT-4o manages a dispute conversation, and o3-mini checks refund eligibility. Swapping the model, prompt, and tool definitions while retaining context lets each agent handle a focused part of the workflow. ### Simple prompts and separate guardrails keep edge cases out of the main task [14:52](https://www.youtube.com/watch?v=joHR2pmxDQE&t=892s) The final lesson is to keep model instructions focused on the target task and handle safety, security, and reliability with guardrails. Guardrails can run in parallel instead of being added to the main prompt. The example uses an input guardrail for prompt injection and output guardrails for the agent's response. High-stakes actions, such as issuing a refund or revealing personal account information, can wait until the relevant checks have completed. ## Notable quotes - "It's actually what's your broader business strategy." (03:02) - "We believe a better approach is to first build with primitives, understand how your task decomposes, where the failures happen, and what actually needs improvement." (10:37) - "The goal isn't really to build a complicated system, it's just to build a system that works." (12:28) - "Keeping the model instructions simple and focused on the target task ensures maximum interoperability of your system." (15:01) ## Tools & references mentioned - OpenAI - ChatGPT - OpenAI API - Morgan Stanley - Deep Research - Operator - GPT-4o mini - GPT-4o - o3-mini ## Who should watch - You are responsible for an enterprise AI program and need a concrete sequence for moving from employee experimentation to production use cases. - Your team is choosing an agent framework or designing a multi-agent system before it has measured failures, latency, or retrieval quality. - You are building an agent that can take sensitive actions and need a practical separation between task prompts and safety checks. ## Related talks - [How agents will unlock the $500B promise of AI](https://aietalks.com/talks/how-agents-will-unlock-the-500b-promise-of-ai) (Donald Hruska, Retool, 16:22) - [Agentic Enterprise: What Your CEO Must Know About AI](https://aietalks.com/talks/agentic-enterprise-what-your-ceo-must-know-about-ai) (Hubert Misztela, Novartis, 28:04) - [AI Leadership](https://aietalks.com/talks/ai-leadership-l89gzwei) (Grace Isford, Lux Capital & Hamel Husain, Parlance Labs & Greg Ceccarelli, SpecStory & Don Bosco Durai, Privacera & Colin Flaherty, Augment Code & Stephen Chin, Neo4j & Jonathan Lowe, Pfizer & Bruno Passos, Booking.com & Beyang Liu, Sourcegraph & Waseem Alshikh, Writer & Prashant Mital & Toki Sherbakov, OpenAI & Shirsha Chaudhuri, Thomson Reuters & Aparna Dhinkaran, Arize & Diamond Bishop, Datadog & Paul Gilbert, Arista Networks & Alexander Bricken & Joe Bailey, Anthropic & Heath Black, SignalFire & Xiaofeng Wang, LinkedIn & Douwe Kiela, Contextual AI & Barr Yaron, Amplify Partners, 8:03:36) - [Privacy First Enterprise AI: Building AI Agents that Never Leave Your Security Boundary](https://aietalks.com/talks/privacy-first-enterprise-ai-building-ai-agents-that-never-leave-your-security) (Steven Moon, Aech AI Inc., 07:10) - [Scaling Agents for Gen AI Products](https://aietalks.com/talks/scaling-agents-for-gen-ai-products) (Anju Kambadur, Bloomberg, 19:38)