Best LLM Observability Platform Guide 2026

Updated: September 10, 2026

For most production LLM stacks in 2026, Langfuse is the best default LLM observability platform. It's MIT-licensed, self-hostable, framework-agnostic, and ships trace, prompt-management, and LLM-as-judge features in a single project. LangSmith wins if you already live inside LangChain and want the deepest first-party integration. Helicone wins if you need a five-minute proxy install and cost caps. Arize Phoenix wins if you want OpenTelemetry-native local dev and embedding-drift analysis. I've run all four in production over the last eighteen months, and this guide is the dashboard-and-dial-tone comparison I wish someone had handed me on day one.

Langfuse (v3, released 2024, actively shipping through 2026) is the best all-round pick: MIT + EE license, self-hosted or cloud, native prompt registry, LLM-as-judge evals, and OpenTelemetry ingestion.
LangSmith remains closed-source and cloud-only for the free tier; self-hosting requires an enterprise contract. Its dataset and eval UX is still the smoothest if you have paying customers and no time to run infra.
Helicone is the fastest to install (a single base-URL swap), which is why every "quick MVP" I see in production runs on it. Its trace model is shallower than Langfuse's or Phoenix's for multi-agent flows, though.
Arize Phoenix is the only tool here built on the OpenInference OpenTelemetry semantic conventions, so it slots into existing OTEL collectors without a bespoke SDK.
None of these tools will save you if you don't wire session IDs, user IDs, and prompt version tags at the call site. That's the operator's job, not the vendor's.

What is LLM observability?

LLM observability is the practice of capturing, correlating, and analyzing every request an application sends to a language model, along with the model's response, latency, cost, and downstream side effects. It's not just APM with prettier icons. The traces are non-deterministic, the outputs are unstructured natural language, the "unit cost" is a function of both input and output tokens, and the failure modes are qualitative (hallucination, refusal, tool-loop, off-topic), not just HTTP 5xx.

So, a production-grade LLM observability platform has to do five things at once: (1) capture rich trace and span data for each generation, including tool calls and retrievals; (2) attach a token-and-dollar cost to every call across every model; (3) group calls into sessions and users so you can slice quality by cohort; (4) run automated evaluations, either rule-based or LLM-as-judge, on live traces; and (5) give you a prompt registry so a prompt change is a first-class deployable artifact, not a code commit hidden in a diff. Every tool in this comparison hits some subset of that list.

If you're new to the operational patterns behind these features, our companion piece on LLM observability in production walks through the how-to. This article is about which vendor to hand your traces to.

Head-to-head comparison table

Here's the shape of the market as of Q3 2026. All four projects ship weekly, so treat the specifics as directional rather than gospel. The strategic positioning, though, has been stable for a year.

Dimension	Langfuse	LangSmith	Helicone	Arize Phoenix
License	MIT + EE add-on	Proprietary	Apache 2.0	Elastic License 2.0
Self-host	Docker / Helm, free	Enterprise only	Docker / Helm, free	Docker / `pip install`, free
Instrumentation model	SDK + OTEL	SDK (LangChain-first)	Proxy or SDK	OpenInference OTEL
Prompt registry	Yes, versioned	Yes, versioned	Yes (newer)	Basic
LLM-as-judge evals	Native, scheduled	Native, dataset-driven	Native (2025+)	Native, notebook-style
Cost tracking	All major providers	All major providers	All major providers	All major providers
Free tier (cloud)	50k obs/mo	5k traces/mo	100k requests/mo	Local free; AX paid
Best for	General production ops	LangChain / LangGraph shops	Startups and MVPs	OTEL-native platforms

Langfuse: the pragmatic default

Langfuse is the tool I reach for when someone says "we need observability, we don't know what we need yet, and we can't afford a vendor lock-in argument." The core product is MIT-licensed and self-hostable with a single docker compose up. The SDKs are stable in Python, TypeScript, Java, and via OpenTelemetry. The feature set covers the whole loop: traces, sessions, users, cost, prompt management, datasets, and LLM-as-judge evaluations. The enterprise edition adds SSO, RBAC, and audit logs; the OSS core is production-usable on its own.

Instrumentation is a decorator or a context manager. Here's what a typical wrapping looks like:

from langfuse import observe, get_client
from langfuse.openai import openai

langfuse = get_client()

@observe(name="answer-question")
def answer(question: str, user_id: str) -> str:
    langfuse.update_current_trace(user_id=user_id, session_id=session_for(user_id))
    prompt = langfuse.get_prompt("qa-answer", label="production")
    resp = openai.chat.completions.create(
        model="gpt-4o-2024-11-20",
        messages=prompt.compile(question=question),
        metadata={"langfuse_prompt": prompt},  # links trace to prompt version
    )
    return resp.choices[0].message.content

What matters in that snippet: the langfuse_prompt metadata key is what ties every generation back to the exact prompt version that produced it. That's the single most important dashboard column in a production LLM app. "Did the regression start when we rolled out prompt v42?" is the question you want answered fast, and Langfuse makes it a one-liner. For a broader treatment of how prompt registries fit into a cost strategy, see our guide on LLM cost optimization. Full instrumentation and self-hosting docs live in the Langfuse documentation.

LangSmith: managed and integrated

LangSmith is LangChain's first-party observability platform and it's very good at what it does. If your stack is LangChain or LangGraph, tracing is automatic. Set two environment variables and every chain run shows up in the UI with the full graph, retriever hits, tool calls, and token cost. The dataset workflow is the smoothest in the market: capture a bad trace, one-click it into a regression dataset, then run scheduled evaluations against every new prompt version. For teams shipping a support-bot or a RAG product without infra engineers, LangSmith buys weeks of calendar time.

The tradeoffs are the ones you'd expect from a managed, proprietary product. Self-hosting is enterprise-only, so the free tier and Plus tier keep your traces in LangChain's cloud in the US or EU regions. Pricing scales with trace volume, not seats, which surprises teams that add tool calls, retrievals, and evals (each becomes a billable span). And the SDK, while it works with any provider, has clearly been designed with LangChain graphs as the first-class citizen. Instrumentation for a raw anthropic or openai call is a wrapper rather than a decorator. See the LangSmith documentation for the full API surface.

import os
os.environ["LANGSMITH_TRACING"] = "true"
os.environ["LANGSMITH_API_KEY"] = "ls_..."

from langsmith import traceable
from openai import OpenAI

client = OpenAI()

@traceable(run_type="chain", name="answer-question")
def answer(question: str) -> str:
    resp = client.chat.completions.create(
        model="gpt-4o-2024-11-20",
        messages=[{"role": "user", "content": question}],
    )
    return resp.choices[0].message.content

Helicone: proxy-based simplicity

Helicone's superpower is that it doesn't require an SDK at all. You change your OpenAI base URL from https://api.openai.com/v1 to https://oai.helicone.ai/v1, add an auth header, and every request is now logged, cost-tracked, and rate-limitable from Helicone's dashboard. That's the fastest observability integration in the market and it's why so many prototype apps ship on it. The proxy approach also gives you free features that SDK-based tools charge extra for: request caching keyed by prompt, per-user rate limits, and cost caps that return a 429 when a customer exceeds their budget.

The catch is that a pure proxy sees one HTTP request at a time. Multi-step agent traces (retriever → planner → tool call → generator) cannot be reconstructed from proxy hops alone. You either wear the shallow view or you switch to Helicone's async SDK, at which point the "no code changes" pitch is gone. Helicone shipped native trace grouping and LLM-as-judge evaluators in 2025, but the depth is still behind Langfuse and Phoenix for agent-shaped workloads. If your app is a single-shot RAG chatbot, Helicone is often enough. If it's a five-step agent, you'll outgrow it. Their Helicone documentation covers both proxy and async modes.

Arize Phoenix: OTEL-native and open

Phoenix is the outlier. It's built by Arize AI as the open-source counterpart to their enterprise platform, and it's the only tool in this comparison that treats OpenTelemetry as a first-class instrumentation model, not a compatibility layer. Instrumentation uses the OpenInference semantic conventions, which means the same spans that flow into Phoenix also flow into Grafana Tempo, Honeycomb, or Datadog with zero re-instrumentation. For teams that already run an OTEL collector, this is huge, because LLM traces stop being a bespoke pipeline.

Phoenix's other strong suit is embedding analysis. It ships built-in UMAP projections, cluster explorers, and drift detection for embedding vectors, which is invaluable for RAG teams debugging retrieval regressions. The pattern I recommend is running Phoenix as a local dev-loop tool during prompt iteration and shipping the same OpenInference spans to a cloud backend in production. You get the notebook-friendly Phoenix UI locally without paying to persist every request. The install is a one-liner:

pip install arize-phoenix openinference-instrumentation-openai
phoenix serve  # opens http://localhost:6006

# in your app
from phoenix.otel import register
from openinference.instrumentation.openai import OpenAIInstrumentor

tracer_provider = register(project_name="qa-service")
OpenAIInstrumentor().instrument(tracer_provider=tracer_provider)

How do you instrument once and switch backends?

Honestly, the single most valuable investment you can make in 2026 is instrumenting with OpenTelemetry and the OpenInference semantic conventions, then pointing the exporter at whichever backend you currently prefer. All four vendors accept OTLP: Langfuse via its native OTEL endpoint, Phoenix natively, LangSmith via its OTEL-compatible ingest (in beta as of Q2 2026), and Helicone through its async SDK bridge. This means the decision you're making today is reversible, which is a good property for a market that's still moving.

Here's a minimal OTEL setup that works with Langfuse and Phoenix out of the box, and can be redirected in one environment variable:

from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from openinference.instrumentation.openai import OpenAIInstrumentor
import os

endpoint = os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"]  # per-backend
headers  = os.environ.get("OTEL_EXPORTER_OTLP_HEADERS", "")

provider = TracerProvider()
provider.add_span_processor(
    BatchSpanProcessor(OTLPSpanExporter(endpoint=endpoint, headers=dict(
        h.split("=", 1) for h in headers.split(",") if h
    )))
)
OpenAIInstrumentor().instrument(tracer_provider=provider)

The exporter endpoint is https://cloud.langfuse.com/api/public/otel/v1/traces for Langfuse Cloud, http://localhost:6006/v1/traces for local Phoenix, or your self-hosted collector for either. If you build the instrumentation layer first and pick a backend second, you keep leverage. If you build against Langfuse's or LangSmith's proprietary SDK first, you're locked in until the next rewrite.

Cost tracking and token accounting

All four tools track cost by looking up the model name against a per-provider price sheet and multiplying by the token counts the API returns. In practice, the accuracy diverges in three places: (1) fine-tuned models and custom-priced enterprise contracts, where you need to override the rate card; (2) cached and batched tokens, which are billed at 10-50% of the base rate and require the vendor to know about caching semantics; and (3) tool calls and structured outputs, where token accounting depends on how the vendor counts the tool schema.

Langfuse and LangSmith both let you upload a custom model price table via API. That's critical if you negotiated a discount or if you're running self-hosted OSS models with no natural "price." Helicone's rate card is auto-updated but overrides live in the dashboard, not in code, which is a small papercut for GitOps teams. Phoenix defers to whatever cost your app reports in the span attribute, which is the purest OTEL-y approach but means you own the token-price map yourself.

If cost attribution is a first-class requirement (for example, if you charge customers for LLM usage), pair whichever platform you pick with the patterns from our LLM cost optimization guide and tag every trace with user_id, tenant_id, and feature. Vendors give you the dollar column; you have to give them the grouping key. I hit this exact bug shipping a multi-tenant RAG app last year, and retrofitting tenant_id onto six months of traces is not a fun weekend.

Evals, datasets, and LLM-as-judge

Every tool in this comparison shipped LLM-as-judge evaluators by mid-2025, and the surface area is now surprisingly similar: define a rubric, pick a judge model, score either live traces or a dataset. Where they diverge is in what you can automate. Langfuse and LangSmith both let you schedule evaluators to run continuously against a sample of production traces. 5% of live requests get graded, and a regression on faithfulness or hallucination is a dashboard-visible metric within minutes of a bad prompt shipping. Helicone and Phoenix support the same rubrics but their scheduling story is thinner; Phoenix's evals are notebook-first, which is either a feature or a limitation depending on your team.

Datasets are the other differentiator. LangSmith's dataset UX is the reason many teams stay on the platform even after outgrowing free-tier limits. The "capture bad trace → click into dataset → re-run against new prompt" loop is very well-tuned. Langfuse hit feature parity in v3 and is now equally usable. If you're new to this pattern, our writeup on LLM evaluation pipelines covers the mechanics.

Can I self-host these tools?

Langfuse, Helicone, and Phoenix all ship free, production-grade self-hosted deployments. Langfuse's Helm chart runs on any Kubernetes cluster; Helicone's Docker image is a one-liner with Postgres and ClickHouse; Phoenix runs as a single container or an in-process Python server. LangSmith is the exception: self-hosting is available only on the enterprise tier, and the deployment is more involved (Kubernetes-only, with a licensing key). If SOC 2 or air-gapped deployment is a hard requirement, that alone eliminates LangSmith from the shortlist for many teams.

The other consideration for self-hosting is the observability of your observability tool. Langfuse and Phoenix both export their own operational metrics in Prometheus format; Helicone's OSS build exposes structured logs but lightweight metrics. If you're going to run this thing in prod, you want to be able to page yourself when its ingestion queue backs up.

PII redaction and compliance

Every one of these tools accepts a redaction hook on the SDK side. You pass a function that mutates the prompt or completion before it leaves your process, and the platform never sees the raw payload. That's the only tenable posture for a compliance-heavy workload. Never trust a vendor to redact for you, because "redacted" and "hashed" and "not indexed" are three different things, and you can't easily audit which one they mean.

The practical pattern I use is a two-layer redaction: a fast regex pass for structured PII (emails, phone numbers, credit-card numbers, SSNs) and a slower NER pass for names and addresses on the traces that will be persisted to a dataset. Every tool here supports masking on ingest. Langfuse and Phoenix additionally let you opt out of storing the raw input/output and keep only the shape and metadata, which is a valid tradeoff for HIPAA-adjacent workloads.

Which one should you pick?

My decision tree, after running all four in production:

You're on LangChain or LangGraph and time-to-value matters more than $/trace: LangSmith. The zero-config tracing and dataset UX will pay for itself in the first month.
You want the flexibility to self-host later without a rewrite, and you don't want vendor lock-in: Langfuse. It's the modal answer to "which platform should we start with?" in 2026.
You're two weeks from a demo and need a dashboard now: Helicone. Change your base URL, get a dashboard. Revisit the choice in three months.
You already run OpenTelemetry, or you're building a RAG product where embedding drift is a first-order concern: Arize Phoenix. The OTEL-native model and embedding analysis are hard to replicate elsewhere.

Whichever you pick, wire session IDs, user IDs, and prompt-version tags at the call site from day one. The best vendor in the world can't slice your data by cohort if you never told it what cohort each request belongs to. That's the operator's job, and the one thing every dashboard you'll wish you had built first has in common.

Frequently Asked Questions

Is Langfuse really open source?

Yes. The Langfuse core is MIT-licensed and can be self-hosted for free with full trace, prompt-management, and evaluation features. Enterprise Edition add-ons (SSO, RBAC, audit logs) live in a separately licensed module, but the OSS core is production-usable on its own.

What is the best free LLM observability tool in 2026?

For self-hosted, Langfuse and Arize Phoenix are the two strongest free options. Pick Langfuse if you want a full product surface, Phoenix if you want OpenTelemetry-native local development. For managed free tiers, Helicone's 100k requests/month is the most generous, and LangSmith's 5k traces/month is enough to prototype but tight for real production traffic.

How does LangSmith differ from Langfuse?

LangSmith is proprietary, cloud-first, and deeply integrated with LangChain/LangGraph. Langfuse is MIT-licensed, self-hostable, and framework-agnostic. Feature-wise both cover traces, prompts, datasets, and LLM-as-judge evals; the pick usually comes down to licensing and how much of your stack is LangChain.

Can I self-host LangSmith?

Yes, but only on the enterprise tier. Self-hosting requires a paid license from LangChain and a Kubernetes deployment. If free self-hosting is a hard requirement, use Langfuse, Helicone, or Phoenix instead.

What is OpenInference and why does it matter?

OpenInference is an OpenTelemetry semantic-convention specification for LLM applications, maintained primarily by Arize. It standardizes span attribute names like llm.model_name and llm.token_count.completion so that traces are portable across backends. Instrumenting once with OpenInference means you can switch from Phoenix to Langfuse to a bespoke Grafana Tempo stack without re-instrumenting your app.

Do I need LLM observability if I already have Datadog or New Relic?

You need both. APM tools handle HTTP-level latency and errors well, but they don't natively understand prompts, completions, tool calls, or LLM-as-judge scores. In practice, you export OpenInference spans to both a general APM (for infrastructure correlation) and a dedicated LLM observability tool (for prompt, cost, and eval workflows).