The State of MCP Observability: Observable.tools

Alex Volkov, Weights & Biases, Benjamin Eckel, Dylibso16:56 · Jun 2025 · 2,392 views
Thumbnail for The State of MCP Observability: Observable.tools Watch on YouTube
TL;DR
  1. 1

    MCP creates an observability blind spot because agent code increasingly runs across tools and servers outside the developer's control.

  2. 2

    OpenTelemetry can connect MCP client and server traces, but doing so currently requires passing trace context through MCP metadata by hand.

  3. 3

    The MCP community needs shared SDKs, telemetry support, and semantic conventions so developers do not have to build observability integrations separately.

Summary

Alex Volkov and Benjamin Eckel explain why MCP makes agent behavior harder to inspect. When an agent calls external servers, developers may see the client-side tool call but not what happens inside the server. They propose using OpenTelemetry, with traces made of spans and shared telemetry sinks, to connect those parts through distributed tracing. The current MCP protocol does not provide a high-level observability mechanism, so their example passes trace context through the protocol metadata payload. They show this working across a TypeScript agent and W&B Weave, then describe a coding agent that used a Weave MCP server and its support bot to diagnose and fix its own tracing setup. MCP.Run will also export telemetry to OpenTelemetry-compatible sinks. The speakers call for vendor-neutral conventions, higher-level SDKs, and agreed semantic conventions for agent and MCP events.

Key ideas
00:54

MCP makes end-to-end agent behavior harder to see

Alex says that adding MCP powers to his agent caused the observability he was used to from code running end to end to go dark. The problem grows as agents use more tools through MCP, because developers know less about what happens across the full execution. Benjamin says MCP.Run has had to build its own observability methods for clients and servers, while other teams appear to be solving the same problems independently. They frame this as both a production response problem and a developer experience problem. Without a quick way to understand why, where, and how a production failure happened, response becomes much slower.

02:36

W&B Weave can trace MCP calls when developers control both sides

Alex announces MCP support in W&B Weave. Developers using Python-based MCP clients can set the MCP trace list operation environment variable on the client and server. Weave then displays the MCP tool calls and their durations. The example shows client traces and calls to tools such as calculate BMI. This integration is useful when the developer owns both ends, but it does not reveal what happens inside a remote MCP server. Benjamin also points out that the approach is specific to Weave and its Python SDK rather than a vendor-neutral standard.

04:22

OpenTelemetry supplies shared primitives for MCP traces

Benjamin describes agents with MCP as another distributed system and explains the OpenTelemetry model. A trace is a tree of spans, with each span recording a duration and arbitrary metadata for one step. The step can be broad, such as an HTTP request, or narrow, such as a function call. Telemetry is sent to a sink, often a platform with dashboards, alerts, and monitoring. Because these systems use a shared schema and wire protocol, developers can change the destination through configuration rather than rewriting their application. A common sink also allows spans from separate services to be stitched into one distributed trace.

06:58

Ownership determines how much of an MCP call can be inspected

The speakers separate MCP calls into different administrative domains. When an agent uses a server hosted by another organization, such as a GitHub or Stripe server, the remote operation appears as a single service span and remains a black box. When the developer owns the server, even if it runs in another data center, the server can send spans to the same telemetry sink as the client. Context propagation then lets the sink connect the traces. Their example exposes an HTTP request taking roughly 350 milliseconds inside a fetch server, followed by work that creates markdown.

08:30

MCP context propagation currently requires a low-level workaround

To connect client and server traces, the client extracts its current OpenTelemetry span and passes the trace context through the MCP metadata payload when calling a tool. The server reads that context, makes it the current span, and exports its own spans to the same sink. The sink can then attach the server work to the client's parent span. Benjamin says this uses a lower-level protocol interface rather than a proper high-level observability feature. He describes the method as an abuse of the interface and says tooling should provide a better way.

11:37

An agent can use MCP observability to repair its own integration

Alex describes a demonstration built with a TypeScript agent and W&B Weave's OpenTelemetry endpoint. The agent was asked to add MCP observability, ran the code, and checked whether traces appeared in Weave. It noticed that an input or output parameter was wrong, then discovered that the Weave MCP server exposed a support bot. The coding agent queried that bot, received instructions, fixed the integration, and checked the result again. Alex says he did not touch the keyboard during this process. The example shows an MCP client using one MCP server to reason about traces from another piece of software.

13:10

MCP.Run plans to export telemetry for both servers and clients

Benjamin says MCP.Run will export telemetry to OpenTelemetry-compatible sinks. Its server-side profiles let users combine multiple MCP servers into one virtual server. Its MCP client, Task, is a single-prompt agent that can be triggered by a URL or schedule. Both fit the same observability model, and Benjamin says telemetry from them should be able to connect to Weights & Biases Weave. This extends the discussion beyond a custom SDK integration to a platform that runs MCP infrastructure itself.

14:10

The ecosystem needs shared conventions and higher-level SDKs

The speakers say MCP observability exists today, but it is uneven and still requires too much specialist knowledge. They ask AI engineers to check whether telemetry covers the full execution chain. Tool builders and platform providers should work on higher-level SDKs, instrumentation for clients with custom SDKs, and shared conventions. Semantic conventions give telemetry attributes agreed meanings, such as identifying an MCP tool call with an error or an HTTP request with a 200 status code. They point listeners to the OpenTelemetry team's GenAI semantic conventions effort and invite them to review RFCs and contribute to the Observable.tools initiative.

"Fortunately MCP powered agents are really just another distributed system and we've been doing that for decades."04:22
Who should watch
  • You run agents that call MCP servers and need to diagnose failures across the full client-server path.
  • Your team is building an MCP client, server, or platform and does not want to create a private tracing integration.
  • You maintain an observability product and want to understand the proposed OpenTelemetry and semantic-convention approach for MCP.