Helicone: Open Source Machine Learning Infrastructure

What is Helicone?

Helicone is an open-source AI gateway and LLM observability platform for AI engineers who need to monitor, evaluate, and experiment with applications built on large language models.

What it is

Helicone is a TypeScript platform, licensed Apache-2.0, that sits between an application and its model providers. It combines two roles: an AI gateway that routes requests to more than 100 models through an OpenAI-compatible API, and an observability layer that logs, traces, and scores every request that passes through it. The project has 6,160 stars, 670 forks, and 156 open issues on GitHub, and its topics cover agent-monitoring, analytics, evaluation, llm-cost, llm-evaluation, llm-observability, and llmops. It is a YC W23 company and ships both a hosted service and a self-hostable open-source deployment.

The concrete problem it solves is fragmented instrumentation. Rather than wiring up separate SDKs and dashboards per provider, an application changes one value — the baseURL — and adds an API key. Where teams previously stitched together per-provider API keys, hand-rolled request logging, and ad-hoc cost spreadsheets, Helicone centralizes request capture, cost and latency tracking, session grouping, and prompt versioning behind a single endpoint. The gateway also removes the need for bespoke failover logic: intelligent routing and automatic fallbacks are handled upstream of the application.

Key capabilities

AI Gateway access to over 100 models through one API key, using baseURL: "https://ai-gateway.helicone.ai" with the OpenAI client format.
One-line integration for OpenAI, Anthropic, LangChain, Gemini, and the Vercel AI SDK.
Trace and session inspection for agents, chatbots, and document processing pipelines.
Cost and latency analytics, with one-line export to PostHog for custom dashboards.
Prompt Management that versions prompts against production data and deploys them through the gateway without code changes.
A playground for iterating on prompts, sessions, and traces in the UI.
Fine-tuning through partners OpenPipe and Autonomi, plus SOC 2 and GDPR compliance.

Who uses it and how

AI engineers shipping agents, chatbots, and document processing pipelines who need to debug a request after it fails in production.
Teams running multiple model providers that want one credential and automatic fallback instead of provider-specific client code.
Organizations under SOC 2 or GDPR obligations that need request-level logging and evaluation with a compliance story attached.
Enterprise workloads that self-host, using the Helm chart rather than the hosted dashboard.
Smaller teams starting on the hosted free tier, which covers 10,000 requests per month with no credit card required.

Getting started

Self-hosting uses Docker: clone the repository, cd docker, cp .env.example .env, then ./helicone-compose.sh helicone up. The hosted option is a signup at helicone.ai/signup followed by a baseURL change in application code.

How it compares

No competing observability platform is named in the facts, so Helicone stands alone in this registry. The tools it names — LangChain, LlamaIndex, the Vercel AI SDK, PostHog, OpenPipe, and Autonomi — are integration targets rather than substitutes. It is therefore best read as an additional layer beneath an existing stack, not a replacement for one.

When to use it — and when not to

A self-hoster takes on five services: Web (NextJS), Worker (Cloudflare Workers), Jawn (Express + Tsoa), Supabase for the application database and auth, ClickHouse for analytics, and Minio for object storage. That is a substantial operational footprint, and the README explicitly states that manual deployment is not recommended. The production-ready Helm chart is not public — it requires contacting [email protected] — so teams without an enterprise relationship are effectively limited to docker-compose or the hosted service. Anyone unwilling to operate ClickHouse, Supabase, and object storage should use the hosted tier instead of self-hosting.

project readme (upstream, from github) — read inline

🔍 Observability	🕸️ Agent Tracing	🚂 LLM Routing
💰 Cost & Latency Tracking	📚 Datasets & Fine-tuning	🎛️ Automatic Fallbacks

helicone logo

Docs • Changelog • Bug reports • See Helicone in Action! (Free)

Helicone is an AI Gateway & LLM Observability Platform for AI Engineers

🌐 AI Gateway: Access 100+ AI models with 1 API key through the OpenAI API with intelligent routing and automatic fallbacks. Get started in 2 minutes.
🔌 Quick integration: One-line of code to log all your requests from OpenAI, Anthropic, LangChain, Gemini, Vercel AI SDK, and more.
📊 Observe: Inspect and debug traces & sessions for agents, chatbots, document processing pipelines, and more
📈 Analyze: Track metrics like cost, latency, quality, and more. Export to PostHog in one-line for custom dashboards
🎮 Playground: Rapidly test and iterate on prompts, sessions and traces in our UI.
🧠 Prompt Management: Version prompts using production data. Deploy prompts through the AI Gateway without code changes. Your prompts remain under your control, always accessible.
🎛️ Fine-tune: Fine-tune with one of our fine-tuning partners: OpenPipe or Autonomi (more coming soon)
🛡️ Enterprise Ready: SOC 2 and GDPR compliant

🎁 Generous monthly free tier (10k requests/month) - No credit card required!

Open Sourced LLM Observability & AI Gateway Platform

Quick Start ⚡️

Get your API key by signing up here and add credits at helicone.ai/credits

Update the baseURL in your code and add your API key.

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://ai-gateway.helicone.ai",
  apiKey: process.env.HELICONE_API_KEY,
});

const response = await client.chat.completions.create({
  model: "gpt-4o-mini",  // claude-sonnet-4, gemini-2.0-flash or any model from https://www.helicone.ai/models
  messages: [{ role: "user", content: "Hello!" }]
});

🎉 You're all set! View your logs at Helicone and access 100+ models through one API.

Self-Hosting Open Source LLM Observability

Docker

Helicone is simple to self-host and update. To get started locally, just use our docker-compose file.

# Clone the repository
git clone https://github.com/Helicone/helicone.git
cd docker
cp .env.example .env

# Start the services
./helicone-compose.sh helicone up

Helm

For Enterprise workloads, we also have a production-ready Helm chart available. To access, contact us at [email protected].

Manual (Not Recommended)

Manual deployment is not recommended. Please use Docker or Helm. If you must, follow the instructions here.

Architecture

Helicone is comprised of five services:

Web: Frontend Platform (NextJS)
Worker: Proxy Logging (Cloudflare Workers)
Jawn: Dedicated Server for serving collecting logs (Express + Tsoa)
Supabase: Application Database and Auth
ClickHouse: Analytics Database
Minio: Object Storage for logs.

Integrations 🔌

Inference Providers

Integration	Supports	Description
AI Gateway	JS/TS, Python, cURL	Unified API for 100+ providers with intelligent routing, automatic fallbacks, and unified observability
Async Logging (OpenLLMetry)	JS/TS, Python	Asynchronous logging for multiple LLM platforms
OpenAI	JS/TS, Python	Inference provider
Azure OpenAI	JS/TS, Python	Inference provider
Anthropic	JS/TS, Python	Inference provider
Ollama	JS/TS	Run and use large language models locally
AWS Bedrock	JS/TS	Inference provider
Gemini API	JS/TS	Inference provider
Gemini Vertex AI	JS/TS	Gemini models on Google Cloud's Vertex AI
Vercel AI	JS/TS	AI SDK for building AI-powered applications
Anyscale	[JS/

Frequently asked questions

Is Helicone free to use?

Helicone is open source under the Apache-2.0 licence. There is no licence fee and no seat count — you can self-host it or, where the project offers one, pay a vendor for a managed version instead.

What does Helicone do?

Build reliable AI apps with comprehensive observability

What is Helicone written in?

Helicone is primarily written in TypeScript. Its source is publicly available at https://github.com/helicone/helicone, and it has 6,161 GitHub stars.