# Learned Execution Graphs for Anomaly Detection & Drift in APIs Ritvik Pandya, JP Morgan Chase | AI Engineer World's Fair 2026 | 19:38 Source: https://www.youtube.com/watch?v=u1yaOeEX4e8 Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks. Page: https://aietalks.com/talks/learned-execution-graphs-for-anomaly-detection-drift-in-apis Published: 2026-07-23 Tags: debugging, observability, reliability, tracing ## TL;DR - Execution graphs model each API request as a short-lived DAG, which makes the order and context of middleware steps visible. - Per-client baselines and node-level comparisons can separate normal variation from anomalies and locate the service causing a delay. - Drift needs classification and controlled rollout because structural changes, request volume, and changing request populations require different responses. ## Summary Ritvik Pandya presents execution graphs as a way to inspect how an API request moves through gateways, authentication, orchestration, and downstream services. Each request becomes a short-lived DAG that records the order of execution and the context passed between nodes. The system first compares a request with a baseline, then finds the node where behavior changed, checks whether the deviation crosses a threshold, and takes action only when warranted. Pandya separates one-off anomalies from drift, where behavior changes over time. He describes structural drift, volume-related drift, and covariate drift, such as a larger share of requests from outside the local region. The design uses telemetry, streaming checks, a faster hot path, and a slower reconciliation path. Pandya also discusses delayed events, sampling, endpoint cold starts, fine-grained labels, explainability, and gradual rollout. His practical advice is to define baselines by client and use case rather than applying one threshold to every request. ## Key ideas ### Execution graphs make request processing visible as a DAG [01:18](https://www.youtube.com/watch?v=u1yaOeEX4e8&t=78s) Pandya distinguishes execution graphs from persistent graphs and property graphs used with systems such as Neo4j. An execution graph is short-lived and describes one request as it moves through the system. His example has an edge layer, gateways or ingress, authentication and authorization, orchestration, parallel downstream calls, and a notification back to the client. The DAG records the order in which services run and the context passed from one node to the next. Retries and loops can also be represented, with each loop added as its own graph entity so it can be tracked separately. ### A tier-one graph check should filter ordinary requests before deeper analysis [03:05](https://www.youtube.com/watch?v=u1yaOeEX4e8&t=185s) The first check is intentionally cheap. Pandya compares it to showing a boarding pass at an airport. If the request's end-to-end execution matches its baseline, the system does not send it to a deeper check. A delay triggers analysis of what changed, including a possible added or removed node. Later checks can use scale deviations, divergence, or an exponential moving average. Baselines also need to vary by client. A local client and an external client can have different normal processing times because the external request may require extra checks. ### The workflow is baseline, deviation, localization, and action [05:23](https://www.youtube.com/watch?v=u1yaOeEX4e8&t=323s) Pandya describes a simple sequence for the system. It represents the full request as a DAG, builds a baseline, finds a deviation, and locates the affected node. The deviation is then compared with a threshold appropriate to the system. A result inside the threshold does not create an alert or trigger automation. In his example, the overall request still passes through its normal nodes, but the foreign exchange rate service takes longer than usual. The graph points directly to that service, so the team can investigate its previous failure cases instead of treating the whole endpoint as the problem. ### Training on injected failures lets the system learn before production use [06:46](https://www.youtube.com/watch?v=u1yaOeEX4e8&t=406s) For a benchmark, Pandya says the team used OpenTelemetry and Starbench. They ran millions of traces through the system for seven days, injected a problem or anomaly, and trained the system before putting it live. This setup teaches the detection method what normal execution looks like and gives it examples of deviations to identify. The graph-based view also supports performance localization. If one node suddenly takes more time, the alert can identify that node. If delay appears across the graph, the likely fix is something shared by the affected path. ### An anomaly is a single incident, while drift changes the baseline over time [07:33](https://www.youtube.com/watch?v=u1yaOeEX4e8&t=453s) Pandya uses a commute to explain the difference. A drive that normally takes one hour but takes longer on one day may be an isolated incident caused by traffic or an accident. Drift is a slower change, such as the same commute taking twenty extra minutes after a year. The system should respond differently to those cases. A one-off event can be assessed according to system criticality, while sustained change may require a new baseline. This distinction prevents a temporary incident from being treated as a permanent change in expected behavior. ### Structural, volume, and covariate drift require different responses [08:55](https://www.youtube.com/watch?v=u1yaOeEX4e8&t=535s) Structural drift occurs when a node is added or removed. Pandya compares this with a coffee shop adding a membership question to every visit. The new step must be reflected in the baseline. Volume drift happens when rising request volume makes a service slow or unable to handle expected demand. The response might involve scaling instances or making a call asynchronous. Covariate drift changes the mix of requests without necessarily changing the system itself. If a product begins receiving more requests from other countries, currency handling may become more common, so the team may need separate graphs or revised baselines for local and external traffic. ### Telemetry can feed a fast decision path and a slower reconciliation path [12:46](https://www.youtube.com/watch?v=u1yaOeEX4e8&t=766s) OpenTelemetry feeds the data into the analysis system, where root cause analysis uses the collected points to classify drift and suggest an action. Pandya separates a hot path from a reconciliation path. The hot path makes a fast decision and can automate a response. The reconciliation path takes longer but can be more accurate. Before automation, the team should assess the risk of the proposed action. Pandya recommends rolling a solution out to 5% or 10% of machines, monitoring it, and then expanding it to all nodes if the result is acceptable. ### Delayed events and new endpoints need separate handling [15:21](https://www.youtube.com/watch?v=u1yaOeEX4e8&t=921s) A missing event does not automatically mean the graph changed. If seven nodes exist but telemetry from the seventh node arrives late, the system may temporarily observe only six nodes. Pandya says the detection logic must be tuned to avoid calling this a structural change. In the example, a tail-based system fits because the relevant interval runs from the service request's start to its end. A new endpoint should receive a new baseline rather than being forced into a generic one. Detection can use MMD or KL divergence, followed by confirmation and classification before an automated response. ## Notable quotes - "The idea here is holistically try to identify how the request processing happens and if there is any deviation on that and how to detect that and how to fix that." (01:18) - "First you represent the entire request processing as DAG. You come up with the baseline. You find out the deviation and then you try to find out where exactly the issue is." (05:23) - "A new node or new step added which you are not aware of that could be one of the thing or one of the step which is removed that could be the another reason." (04:02) - "The other one is say because of the volume of request one of your service is taking more time or it's not you cannot serve the request or the volume which you are expecting now over the time." (09:36) - "If you go to the doctor and doctor says your health score is 22, it doesn't make much sense to you." (18:48) ## Tools & references mentioned - JP Morgan Chase - Neo4j - OpenTelemetry - Starbench - Kafka - MMD - KL divergence ## Who should watch - You operate APIs where healthy endpoint averages can hide a skipped, reordered, or slow middleware step. - Your team needs per-client performance baselines and a way to distinguish an isolated incident from a lasting change. - You are considering automated remediation and need a staged rollout with checks for delayed telemetry and new endpoints.