What it is

Arize Phoenix is an open-source AI observability platform for experimentation, evaluation, and troubleshooting. It lives in the machine-learning infrastructure ecosystem and is written in Python. The project provides tracing, evaluation, datasets, experiments, prompt management, and a playground for LLM applications.

The concrete problem it solves is the difficulty of understanding and improving LLM application behavior at runtime. Teams need to inspect traces, benchmark response quality and retrieval quality, compare prompt and model changes, and keep versioned examples for experiments. Phoenix addresses these needs with OpenTelemetry-based tracing and OpenInference instrumentation, while remaining vendor and language agnostic.

Key capabilities

• Phoenix traces LLM application runtime behavior through OpenTelemetry-based instrumentation and OpenInference integrations.
• It evaluates applications with response and retrieval evals that use LLMs to benchmark performance.
• It supports versioned datasets of examples for experimentation, evaluation, and fine-tuning.
• It tracks changes to prompts, LLMs, and retrieval through experiments.
• It provides a playground for optimizing prompts, comparing models, adjusting parameters, and replaying traced LLM calls.
• It includes prompt management with version control, tagging, and experimentation.
• It exposes a remote MCP server at the /mcp endpoint so Claude Code, Cursor, and other MCP clients can query traces, datasets, and experiments.

Who uses it and how

• AI engineering teams use Phoenix to trace agent workflows built with frameworks such as LangGraph, CrewAI, LlamaIndex, DSPy, OpenAI Agents SDK, Claude Agent SDK, Vercel AI SDK, and Mastra.
• Teams use it to monitor calls to providers including OpenAI, Anthropic, Google GenAI, Google ADK, AWS Bedrock, OpenRouter, and LiteLLM.
• Developers use the playground and prompt management to test prompt changes and replay traced LLM calls.
• Data and evaluation workflows use datasets and experiments to compare prompt, model, and retrieval changes.
• MCP clients such as Claude Code and Cursor can connect to a Phoenix instance to query traces, datasets, and experiments.

Getting started

Users run Phoenix on a local machine, in a containerized deployment, or in the cloud, and they can use Arize AX for managed production workflows. The README points to environments and a tracing quickstart for setup.

When to use it — and when not to

Use Phoenix when a team wants an open-source, self-hosted observability and evaluation platform for LLM applications and can operate the deployment itself. Avoid it when a managed production platform is required, because Arize AX is the managed option offered by Arize. The repository lists 970 open issues and a NOASSERTION license, so teams should review issue volume and licensing terms before adoption.