# Building Agents Is Trivial Now, Context Is the Next Frontier Jeff Ng, Unblocked | AI Engineer World's Fair 2026 | 13:22 Source: https://www.youtube.com/watch?v=HvMyYLTfvhg Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks. Page: https://aietalks.com/talks/building-agents-is-trivial-now-context-is-the-next-frontier Published: 2026-08-21 Tags: agents, context-engineering, mcp, observability ## TL;DR - Cloud primitives and agent frameworks have removed much of the state, sandbox, and observability work needed to deploy agents. - An agent that only sees a ticket and code can recommend repeating an outage because it lacks decisions recorded in Slack and postmortems. - A context engine connects organizational data, resolves conflicts, respects permissions, and gives agents a synthesized explanation they can act on. ## Summary Jeff Ng argues that deploying an agent has become much easier. Earlier, teams had to build checkpointing, state persistence, sandbox isolation, and observability before an agent could run in production. Cloud infrastructure and frameworks now handle much of that plumbing, leaving a model, instructions, tools, and a sandbox. The harder problem is giving an agent the context that a human supplies during local use. In Ng's example, an agent enriched a Linear ticket using the ticket and repository, then recommended re-enabling async dispatch. That setting had recently caused an outage, but the agent could not see the Slack discussion or postmortem that explained why it had been disabled. Ng demonstrates the same agent with Unblocked's context engine. It finds the relevant records, reconciles them, and gives the agent a summary. The recommendation changes from repeating the outage to preventing another one. He argues that MCP provides access to data, while a context engine provides an understood and permission-scoped view of the organization. ## Key ideas ### Production agents used to require a quarter of plumbing work [00:01](https://www.youtube.com/watch?v=HvMyYLTfvhg&t=1s) Ng says that six months earlier, building an agent took a team's effort for roughly a quarter because an agent required more than a model and tools. Teams had to build production systems around long-lived, stateful runs. Checkpointing and state persistence preserved message history, tool calls, and the agent's position in its loop after infrastructure failures. Without that state, a restart could waste tokens and latency, and side effects might happen twice. The work was necessary to operate an agent, but Ng says none of it improved what the agent could do. ### Sandboxes and observability were operational taxes [01:58](https://www.youtube.com/watch?v=HvMyYLTfvhg&t=118s) Agent-generated and third-party code runs on infrastructure that needs protection. Ng describes isolated sandboxes as a way to limit access to environment secrets and networks, and to avoid taking down a shared host. Observability created another burden. Engineers needed logs and traces from roughly half a dozen systems to answer where a failure occurred. Ng groups checkpointing, sandbox infrastructure, and observability together as taxes required to put an agent into production. They make the service operable, but they do not add reasoning ability or domain knowledge. ### Frameworks reduce an agent definition to a few decisions [03:02](https://www.youtube.com/watch?v=HvMyYLTfvhg&t=182s) Ng says cloud providers and agent frameworks have absorbed much of the infrastructure complexity. He names Cloudflare, Vercel, and AWS as cloud infrastructure players, and mentions Flu, Vercel Eve, and Astra as frameworks that remove plumbing. In his example, defining an agent means choosing a model, writing instructions or a system prompt, selecting tools and skills, and specifying the sandbox location. That shorter implementation lets a team focus on the agent's actual logic instead of first building its runtime. ### A ticket-and-code agent recommended a setting that had caused an outage [04:21](https://www.youtube.com/watch?v=HvMyYLTfvhg&t=261s) Ng demonstrates an issue enrichment system for Linear. It fetches a ticket, decides whether it is a feature or bug, searches the repository, and gives the agent the resulting context so it can plan next steps. The example ticket described severe degradation in the agentic QA pipeline, where time to first character had risen to three or four seconds instead of hundreds of milliseconds. Based on the ticket and code, the agent recommended re-enabling async dispatch because it could run more of the QA pipeline in parallel on one machine. ### The agent missed the organization's decision history [05:36](https://www.youtube.com/watch?v=HvMyYLTfvhg&t=336s) The recommendation was wrong because a support engineer had disabled async dispatch days earlier after it caused an outage. The agent did not see the Slack discussion where engineers examined the failure, agreed on what went wrong, and discussed the fix and next steps. It also missed the Linear postmortem ticket created afterward. Ng says the agent had a narrow understanding of the problem. As a background process, it could make this mistake silently and pass incorrect information to teammates or other agents. ### Humans currently supply the missing context during local use [07:00](https://www.youtube.com/watch?v=HvMyYLTfvhg&t=420s) Ng explains why engineers often avoid this failure when using agents locally. The human asks follow-up questions, catches errors, and supplies facts on every turn. A person knows why code was written a certain way, what failed previously, and what the team decided afterward. The agent only sees the instructions, tools, skills, code, and current ticket that it was given. When a human stays in the loop, that person catches the bad direction and effectively babysits the agent. Once agents run without that person, missing context becomes a direct source of failure. ### A context engine reconciles organizational information before the agent sees it [08:17](https://www.youtube.com/watch?v=HvMyYLTfvhg&t=497s) Ng defines a context engine as a system that provides task-relevant information based on the user and the task, resolves conflicts across datasets, and follows the access rules for the agent. It should give the agent a synthesized understanding rather than a pile of documents that the agent must interpret alone. Unblocked connects docs, code, tickets, and conversations, then builds a model of how those things relate within the organization and system. The information supplied to an agent is reconciled, ranked, and scoped to its permissions. ### MCP gives access, while the context engine decides what the data means [09:39](https://www.youtube.com/watch?v=HvMyYLTfvhg&t=579s) Ng says teams could connect Slack, Linear, and GitHub through MCP, but raw access does not provide understanding. MCP can return raw results from each source, leaving the agent to decide which result to trust. Irrelevant data can fill the context window and raise context costs. Conflicts between a Linear result and a Slack result are also left for the agent to resolve ad hoc. In Ng's framing, a context engine performs that reconciliation before the agent acts. ### Grounded context changed the recommendation [10:36](https://www.youtube.com/watch?v=HvMyYLTfvhg&t=636s) Ng reruns the ticket workflow with Unblocked's context engine connected. The engine finds the relevant Linear postmortem and the Slack conversation about the incident, then returns an understanding as a summary for the agent. The agent no longer has to reason directly from separate documents. With that additional history, the recommendation changes from one that could cause another issue to one that would prevent another outage. Ng says the same context layer can also support coding, code review, customer success, and sales workflows that need institutional knowledge. ## Notable quotes - "Building agents has actually gotten pretty easy. But unfortunately, they still get things confidently wrong." (00:01) - "When an agent is in the loop with a human, we're there to catch the steer. Ultimately, we're there to babysit the agent." (07:33) - "MCP is great at access, but access isn't understanding." (09:39) - "The gap isn't intelligence, it's context." (12:25) ## Tools & references mentioned - Unblocked - Linear - Cloudflare - Vercel - AWS - Flu - Vercel Eve - Astra - MCP - Slack - GitHub - Cloud Code - Codex ## Who should watch - You are deploying background agents that make decisions without a person checking every step, and you need to understand why a ticket and repository may be insufficient. - Your agent can access several internal systems through connectors, but contradictory records and irrelevant search results are consuming its context window. - You are building coding, issue-management, or support agents that need team decisions and past incidents, not just current source code.