# Paving the Road to Production: Governed Enterprise AI Agents

Harish Gaggar, Credit Karma | AI Engineer NYC 2026 | 29:27

Source: https://www.youtube.com/watch?v=tZycGA_Fbfo
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/paving-the-road-to-production-governed-enterprise-ai-agents
Published: 2026-10-14
Tags: enterprise, guardrails, mcp, observability, tracing

## TL;DR
- Enterprise agents need identity, memory, tracking, evaluation, secret handling, and context controls from the start.
- A centralized MCP gateway authenticates requests, routes them to reviewed tools, and records operational behavior.
- LangGraph workflows can classify actions, gather evidence, apply policies, and pause for human approval before sensitive changes.

## Summary
Harish Gaggar presents an enterprise agent platform built around the controls that experiments often leave out. An internal portal registers an agent and creates a repository plus supporting resources for memory, tracking, evaluation, context optimization, identity, and secret management. A CI/CD path reviews code, scans container images, and deploys agents to Kubernetes. Every tool request passes through an MCP gateway for authentication and routing before reaching reviewed custom MCP servers. The talk then follows a LangGraph workflow through interpretation, planning, evidence gathering, policy checks, and human approval for sensitive actions. Threads, runs, checkpoints, tool calls, model use, costs, and evaluation fields are stored for later inspection. Gaggar also demonstrates input filtering for sensitive data and context reduction through pruning, reranking, deduplication, and compression. The demonstrations show infrastructure and token savings, while leaving overall answer quality and business return unvalidated.

## Key ideas
### Production agents need a common foundation around the model workflow
[01:26](https://www.youtube.com/watch?v=tZycGA_Fbfo&t=86s)
Gaggar says agents fail in production when foundation controls are missing. He lists the absent pieces as an authenticated gateway, reviewed tools, query tracking, evaluation, memory, context management, and protection against secrets reaching models or observability systems. His proposed approach is a collection of frameworks rather than a single magical agent. LangGraph handles task-specific nodes, while an MCP gateway, custom MCP servers, audit records, memory services, tracking and evaluation tables, PII filtering, and context optimization provide the surrounding controls.

### An internal portal provisions an agent and its production resources
[04:22](https://www.youtube.com/watch?v=tZycGA_Fbfo&t=262s)
The development path begins by registering an agent with its use case and receiving an agent ID. An internal portal collects the framework, language, skills, and prompts, then creates a repository containing the agent framework and the supporting memory, tracking, evaluation, context, and PII controls. Developers build a Docker image, test inputs and outputs locally, and inspect traces and records in systems such as SQLite or BigQuery. After two pull-request approvals, the pipeline registers the agent, scans the image for vulnerabilities, and deploys it to Kubernetes. The portal also creates workload identity, databases or tables, and secret-management resources.

### Reviewed MCP servers and a central gateway control access to tools
[08:14](https://www.youtube.com/watch?v=tZycGA_Fbfo&t=494s)
For air-gapped environments, Gaggar recommends building internal MCP servers with a FastMCP wrapper and defining the tools inside them. Internal security teams review every tool before production. The tools can enforce filters and limits and block fields that should not reach the model. The MCP gateway authenticates the client, stores session state in Redis, returns a JWT, identifies the requested namespace, attaches the user's token, and routes the call to the correct MCP server. Direct endpoint access is excluded from this design.

### Gateway telemetry makes authentication and tool failures inspectable
[11:48](https://www.youtube.com/watch?v=tZycGA_Fbfo&t=708s)
The gateway sends operational logs to systems such as ELK, Loki, or CloudWatch. OpenTelemetry collects traces and metrics, with traces stored in Tempo and metrics stored in Prometheus. Traces can show which tool was slow, whether a call came from an agent or an IDE, and the error on a failed call. Prometheus can show P95 latency for a tool and whether authorization rejections are increasing. Grafana presents the aggregated data for visualization.

### The LangGraph workflow pauses sensitive actions for human approval
[14:01](https://www.youtube.com/watch?v=tZycGA_Fbfo&t=841s)
Gaggar's LangGraph flow first interprets a question and creates a plan. A classification step decides whether the request is read-only, modifies something, deletes something, or requires sensitive handling. The plan fetches evidence through the authenticated MCP gateway and the appropriate tool, then checks policy. Read-only actions can execute and enter the audit log. Write operations and other sensitive actions go through a human-in-the-loop step. Both approvals and rejections are recorded.

### Threads, runs, and checkpoints preserve the state of agent work
[16:59](https://www.youtube.com/watch?v=tZycGA_Fbfo&t=1019s)
Each interaction creates a thread, and each question on that thread receives a run ID. The workflow saves a checkpoint at the node that handles the question. Follow-up questions create new runs on the same thread and save further checkpoints. Gaggar shows records containing the thread ID, question, execution time, tool calls, run metadata, and the nodes visited in the graph. SQLite can support local development, while Cloud Spanner and equivalent services can hold production state.

### Agent tracking separates run records from gateway monitoring
[19:48](https://www.youtube.com/watch?v=tZycGA_Fbfo&t=1188s)
Gaggar distinguishes gateway monitoring from agent-level tracking and evaluation. The agent records the submitted query, response, model, token use, cost, latency, tool invocations, node executions, and evaluation fields. His example has tables for job tracking, quality checks, evaluation scores, quality metrics, tool invocations, and agent trajectories. The quality checks include success, latency, cost, tool accuracy, and safety. A run-level score ranges from zero to one. These records can later feed dashboards about agent use, quality, and cost.

### Input filtering and context optimization target separate failure modes
[23:27](https://www.youtube.com/watch?v=tZycGA_Fbfo&t=1407s)
The PII treatment framework assigns data types severity levels and blocks and redacts data above the stated threshold before it reaches the model, memory database, tracking tables, or tracing systems. Gaggar then describes a context optimizer that selects data within a token limit, reranks it by relevance, removes duplicates, and compresses the result before the model call. In his demonstrations, one query falls from 24,000 tokens to 2,400, and another falls to about 1,500. He says many cases achieve 70 to 78 percent token savings, but the examples do not establish answer quality or business return.

## Notable quotes
- "We are not going to build a magical agent here in this demo." (02:40)
- "The dev portal is not just creating the repository, it also creates a bunch of other production resources." (07:13)
- "There should not be direct access provided on any endpoints." (11:39)
- "Make sure that we build an enterprise framework, not just an agent." (27:33)
- "Make sure that every tool call request either you are doing this from your local or through agent, it has to go through some sort of centralized MCP gateway." (28:07)

## Tools & references mentioned
- Intuit Credit Karma
- LangGraph
- LangChain
- MCP
- FastMCP
- Redis
- Docker
- Kubernetes
- OpenTelemetry
- Tempo
- Prometheus
- Grafana
- ELK
- Loki
- CloudWatch
- SQLite
- BigQuery
- Cloud Spanner
- JFrog Artifactory
- Slack
- Docker

## Who should watch
- You are building internal agents and need a deployment path that provisions identity, secrets, memory, and evaluation records alongside application code.
- Your agents call enterprise systems through tools, and you need authentication, routing, tool review, audit logs, and human approval for sensitive actions.
- You are trying to reduce context costs while keeping inputs and traces from carrying sensitive data.

## Editor's note

Harish Gaggar's demonstrations show infrastructure and token savings, while leaving overall answer quality and business return unvalidated. Kitaru lets a person review recorded agent sessions and turn those observations into cohorts of runs. Evaluators are checked against those human verdicts before they gate anything.

Written by the AIE Talks editors (the Kitaru team), not by the speaker.

## Related talks

- [IT Admin for the AI Workforce](https://aietalks.com/talks/it-admin-for-the-ai-workforce) (Sarthak Aggarwal, Decawork, 16:17)
- [Agents, Access, and the Future of Machine Identity](https://aietalks.com/talks/agents-access-and-the-future-of-machine-identity) (Nick Nisi, WorkOS & Lizzie Siegle, Cloudflare, 14:17)
- [AI + Security & Safety](https://aietalks.com/talks/ai-security-safety) (Don Bosco Durai, Privacera, 18:13)
- [Building Agents (the hard parts!)](https://aietalks.com/talks/building-agents-the-hard-parts) (Rita Kozlov, Cloudflare, 21:12)
- [How agents will unlock the $500B promise of AI](https://aietalks.com/talks/how-agents-will-unlock-the-500b-promise-of-ai) (Donald Hruska, Retool, 16:22)