# AI Agents for Performance: Ship Faster, Pay Less Rajat Shah, Netflix | AI Engineer World's Fair 2026 | 33:39 Source: https://www.youtube.com/watch?v=CgsWxRUY5Eo Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks. Page: https://aietalks.com/talks/ai-agents-for-performance-ship-faster-pay-less Published: 2026-07-28 Tags: agents, coding-agents, human-in-the-loop, testing, workflows ## TL;DR - Coding agents can read structured profiling data, find common performance anti-patterns, trace them to production code, and propose fixes. - A fixed workflow can check functional tests and canary results before sending a performance change for human approval. - A shared Git-based catalog of patterns can move performance checks from production incidents toward code authoring. ## Summary Rajat Shah describes a workflow that uses an AI coding agent to turn production profiling data into a proposed performance fix. The agent reads a call stack, recognizes patterns such as quadratic loops or repeated allocation, checks out the exact production commit, finds the source code, and opens a code review. In one example, it found an O(N²) implementation in a tensor merge method. Shah's team validated the change with functional tests and a canary that compared old and new code on production traffic. He argues that the agent should remain inside a fixed workflow with strong test, observability, and deployment foundations. Findings can be stored in a central Git catalog of patterns and anti-patterns. Over time, coding and review agents can use that catalog to catch inefficient code before it reaches production. Human approval remains required for changes to live systems. ## Key ideas ### Coding agents can increase compute costs while increasing code output [00:00](https://www.youtube.com/watch?v=CgsWxRUY5Eo&t=0s) Shah says coding agents make it easy to produce code at roughly 10 times the previous speed, while compute cost can rise at a similar pace because the generated code is tuned for shipping quickly. The agent does not know the specific details of Netflix's platforms, frameworks, or internal code patterns. It may reuse patterns from public code or invent an approach that does not fit the local framework. This creates a performance problem for machine-learning serving systems, where inefficient code can run across many requests before anyone investigates it. ### Manual profiling leaves performance work until something breaks [02:24](https://www.youtube.com/watch?v=CgsWxRUY5Eo&t=144s) The normal process starts with profiling one production instance, downloading the data, and opening it in a visualizer. Engineers then inspect call stacks and CPU time to find promising code paths, search their repositories, identify a root cause, write a change, merge it, and repeat the process. Shah describes this as a tedious treasure hunt with a learning curve. Because it takes so much time, teams often inspect profiling data only when a service has a serious CPU problem, sometimes at 2:00 a.m. Performance improvements therefore happen rarely across a large fleet. ### Structured profiles give an agent enough information to recognize common smells [04:46](https://www.youtube.com/watch?v=CgsWxRUY5Eo&t=286s) Shah's experiment rests on two assumptions. Profilers produce similar, structured output across Java, Python, Go, and other runtimes, including call stacks, self CPU, and inclusive CPU. Coding agents also know common patterns such as O(N²) loops, loop-invariant work repeated inside a loop, repeated object allocation, and contention that could be reduced through batching. The agent can compare the profile with the method names and call paths, then recognize that a usage pattern is quadratic rather than linear. It can find the problem from the call stack before reading the whole codebase. ### The agent can trace a hot call path to the exact production source [07:37](https://www.youtube.com/watch?v=CgsWxRUY5Eo&t=457s) After identifying a suspicious method, the agent searches for the repository where it is defined and checks out the exact commit running in production. Shah says the production build normally provides that commit. The agent then locates the implementation, skips irrelevant internal library details, and traces the full call path. This matters because the proposed change must apply to the code that actually generated the profile, rather than to a newer or different version in the repository. With the relevant code and profile together, the agent can move from pattern recognition to a concrete fix. ### A production profile led to an O(N) fix and a canary check [11:01](https://www.youtube.com/watch?v=CgsWxRUY5Eo&t=661s) In the first example, the agent identified a quadratic implementation in a tensor merge method. The profile showed that the code consumed 8.8% of CPU time during the profiling period. With an appropriate skill or prompt, the agent checked out the repository, found the implementation, and sent a code review in less than five minutes. Shah says the team also measured CPU and latency savings from an optimized implementation. The change was not accepted because the model judged it successful. The later workflow used tests and a production canary to verify the result. ### Cross-repository searches found the same pattern in seven services [12:41](https://www.youtube.com/watch?v=CgsWxRUY5Eo&t=761s) The workflow can search beyond the service that first exposed a problem. Shah gives an example involving a counter object for Spectator metrics that was created on every iteration of a hot path. A cross-repository search found the same pattern in seven services. Fixing all of those implementations could save between 0.5% and 4.6% of CPU cycles. This changes the value of a finding: one profile can reveal a reusable pattern and point the agent toward other codebases where the same implementation may have been copied. ### A central Git catalog gives agents memory across services [15:31](https://www.youtube.com/watch?v=CgsWxRUY5Eo&t=931s) Shah proposes a central catalog of performance patterns and anti-patterns because an LLM's own memory is too compact to hold all the context a performance engineer uses. The catalog can begin as Markdown files in a shared Git repository rather than a vector database. Entries can include useful symbols, services where the pattern was confirmed, confidence, and both the anti-pattern and preferred pattern. Existing material can bootstrap it, including Jeff Dean's C++ optimization writing, PyTorch's torch fix repository, and internal performance playbooks. A finding from one service can then help agents analyze other services without repeating the original discovery process. ### Tests, canaries, and engineers remain the safety boundary [20:56](https://www.youtube.com/watch?v=CgsWxRUY5Eo&t=1256s) Shah keeps human approval in the workflow because changing code that already runs correctly in production can break business logic, especially when context or test coverage is missing. The agent should run integration, unit, and functional tests before asking for attention. A canary can then send the same traffic to old and new machines and compare CPU, latency, and error rate. A regression or error increase is a reason to stop. Shah's mental model is simple: the profiler gives an estimate, the canary gives ground truth, and an engineer makes the final decision. ### The long-term goal is to catch performance problems while code is written [24:18](https://www.youtube.com/watch?v=CgsWxRUY5Eo&t=1458s) The reactive workflow starts with code already running in production and uses profiling to create fixes and populate the catalog. Shah says teams should gradually shift left. A review agent could consult the catalog and comment on a proposed anti-pattern. A coding agent could consult it before writing inefficient code. The catalog must be structured hierarchically so it does not fill the agent's context or consume too many tokens. Shah recommends starting with a fixed level-two workflow, such as triggering a profile, analyzing it, running a canary, and suggesting a fix. More autonomous agents require stronger evaluation, sandboxing, and security controls. ## Notable quotes - "Profiler gives the estimate, canary gives ground truth." (23:40) - "The coding agent ships code. It is pretty much tuned to ship code fast." (01:22) - "This is not an AI problem. The observability, canary, verify logic, these are all standard checks that you need to have in your system." (23:28) - "Start with level one. Try to move to level two and you get maximum benefits." (33:07) ## Tools & references mentioned - Netflix - AI Engineer World's Fair 2026 - Spectator metrics - PyTorch - torch fix - Jeff Dean - C++ ## Who should watch - You maintain ML-serving or other high-throughput services where CPU and latency waste can remain hidden in production. - Your team uses coding agents and needs a workflow for checking the runtime cost of generated code. - You want to build a reusable performance knowledge base instead of rediscovering the same anti-pattern in each repository.