# OpenLLMetry is all you need

Nir, TraceLoop | AI Engineer Summit 2025 | 09:12

Source: https://www.youtube.com/watch?v=KVgbERRPU4M
Channel: AI Engineer (https://www.youtube.com/@aiDotEngineer). Summarised by AIE Talks.
Page: https://aietalks.com/talks/openllmetry-is-all-you-need
Published: 2025-02-22
Tags: observability, open-source, tracing

## TL;DR
- OpenTelemetry gives cloud applications a common format for logs, metrics, and traces that works across observability platforms.
- OpenLLMetry extends OpenTelemetry with automatic instrumentation for GenAI frameworks, foundation models, and vector databases.
- Teams can send the same GenAI observability data to platforms such as Datadog, Sentry, Grafana Tempo, Dynatrace, or LangSmith by changing configuration.

## Summary
Nir, CEO of TraceLoop, explains OpenTelemetry before showing how OpenLLMetry extends it for GenAI applications. OpenTelemetry standardizes logs, metrics, and traces, and its SDKs, instrumentations, and collectors work across major cloud observability platforms. Traces are especially useful for GenAI systems because chains, workflows, and agents often involve several steps and tool calls. OpenLLMetry adds automatic instrumentation for foundation models, frameworks, and vector databases. The instrumentation can capture events such as model calls, token usage, latency, errors, vector searches, indexing, returned distances, and scores in the OpenTelemetry format. Collectors can filter or obscure sensitive data and route observability data to multiple providers. Nir's main argument is practical: applications can keep their observability data in a standard format and choose the platform that receives it. Switching platforms then becomes a configuration change rather than an application rewrite.

## Key ideas
### OpenTelemetry standardizes cloud observability data
[00:18](https://www.youtube.com/watch?v=KVgbERRPU4M&t=18s)
OpenTelemetry is an open-source CNCF project that standardizes logging, metrics, and tracing for cloud applications. Logging records arbitrary events during an application's life. Metrics show aggregate behavior across days, users, or other dimensions, such as CPU usage, memory usage, latency, token usage, or error rate. Traces follow a multi-step request across microservices. Nir points out that tracing fits GenAI systems particularly well because chains, workflows, and agents often perform several steps and interact with tools.

### The OpenTelemetry ecosystem includes code libraries and collectors
[03:15](https://www.youtube.com/watch?v=KVgbERRPU4M&t=195s)
The protocol is only part of OpenTelemetry. SDKs let developers send logs, metrics, and traces from application code, while instrumentations create that data automatically. Nir says OpenTelemetry has SDKs for 11 languages, including Python, TypeScript, Go, and C++. Collectors are deployable components that process observability data before it reaches a platform. They can filter data, obscure personally identifiable information or other sensitive content, and send data to multiple providers.

### Instrumentations add visibility without manual event collection
[04:04](https://www.youtube.com/watch?v=KVgbERRPU4M&t=244s)
An instrumentation adds observability to a library or service used by the application. Nir uses a PostgreSQL client as an example. The instrumentation monkey-patches the client library and emits the relevant logs, metrics, and traces automatically. The work happens on the application side, and the instrumentations are designed to keep latency impact almost negligible. This gives developers visibility into a component without adding every event by hand.

### OpenLLMetry extends OpenTelemetry to GenAI systems
[06:11](https://www.youtube.com/watch?v=KVgbERRPU4M&t=371s)
OpenLLMetry takes the OpenTelemetry project and adds support for GenAI frameworks, foundation models, and vector databases. Nir names OpenAI, Anthropic, Cohere, Gemini, and Bedrock among the supported model providers. The project also supports Pinecone and Chroma, along with LangChain, LlamaIndex, CrewAI, and Haystack. These integrations emit logs, metrics, and traces through the same standard format.

### The Pinecone instrumentation captures search details
[07:54](https://www.youtube.com/watch?v=KVgbERRPU4M&t=474s)
Nir describes what a Pinecone instrumentation can expose. It can show queries sent to Pinecone, indexing activity, and the vectors returned from searches. The data can include vector distances, scores, and latencies. Because these details use the OpenTelemetry format, they can be viewed in the observability platform connected to the application instead of requiring a Pinecone-specific monitoring system.

### The same GenAI telemetry can reach different platforms
[08:34](https://www.youtube.com/watch?v=KVgbERRPU4M&t=514s)
OpenLLMetry sends GenAI observability data to the platform a team already uses. Nir names Datadog, Sentry, Grafana Tempo, Dynatrace, and other OpenTelemetry-compatible systems. The project description also names New Relic and LangSmith. Since the data uses a standard protocol and format, moving between supported platforms should require configuration changes rather than changes to the application's instrumentation.

## Notable quotes
- "Open Telemetry is a protocol first and foremost that standardizes the way to do logging, metrics and traces in your Cloud application." (01:03)
- "Agents are multi-step processes that interact and run tools." (02:48)
- "Because it's a standard protocol, you're never tied to a specific platform." (08:21)
- "The latency impact is almost negligible." (05:00)

## Tools & references mentioned
- OpenTelemetry
- OpenLLMetry
- CNCF
- Kubernetes
- Datadog
- New Relic
- Dynatrace
- Grafana
- Honeycomb
- Sentry
- Grafana Tempo
- LangSmith
- OpenAI
- Anthropic
- Cohere
- Gemini
- Amazon Bedrock
- Pinecone
- Chroma
- LangChain
- LlamaIndex
- CrewAI
- Haystack
- TraceLoop

## Who should watch
- You are instrumenting an agent, chain, or LLM application and need traces that cover model calls, tools, and vector searches.
- Your team already uses an observability platform and wants GenAI telemetry without committing its application to one vendor.
- You need to remove sensitive data or route the same telemetry to more than one monitoring provider.

## Related talks

- [LLM Observability, Evaluation, Experimentation Platform](https://aietalks.com/talks/llm-observability-evaluation-experimentation-platform) (Dat Ngo, Arize AI, 16:32)
- [What It Actually Takes to Deploy GenAI Applications to Enterprises](https://aietalks.com/talks/what-it-actually-takes-to-deploy-genai-applications-to-enterprises) (Arjun Bansal, Log10.io & Trey Doig, Echo AI, 21:30)
- [Mind the Gap (In Your Agent Observability)](https://aietalks.com/talks/mind-the-gap-in-your-agent-observability) (Amy Boyd & Nitya Narasimhan, Microsoft, 1:20:07)
- [The State of MCP Observability: Observable.tools](https://aietalks.com/talks/the-state-of-mcp-observability-observable-tools) (Alex Volkov, Weights & Biases & Benjamin Eckel, Dylibso, 16:56)
- [Local Models: Trust, Control, Optimization](https://aietalks.com/talks/local-models-trust-control-optimization) (Carter Abdallah, NVIDIA & Vincent Weisser, Prime Intellect & Lucas Atkins, Arcee AI & Chris Alexiuk, NVIDIA, 43:21)