[opensourcedex]

LangWatch is a free, open source machine learning infrastructure project written in TypeScript and released under Apache-2.0. It has 5,003 GitHub stars, 424 forks and 939 open issues, and was last pushed 16 hours ago. On this registry it ranks #33 of 89 tracked projects in Machine Learning Infrastructure, with 5 head-to-head comparisons available. It gained 112 stars over the last 30 tracked days.

What is LangWatch?

What it is

LangWatch is an open-source TypeScript platform for LLM evaluations and AI agent testing, licensed under Apache-2.0. It lives in the machine-learning infrastructure ecosystem for teams that build LLM-powered agents and need to test, simulate, evaluate, and monitor agents before release and in production.

The problem it addresses is the gap between agent demos and reliable production systems. Teams need regression testing, simulations, trace analysis, prompt iteration, and cost governance, but custom tooling creates glue code and tool sprawl. LangWatch connects tracing, datasets, evaluation, prompt optimization, annotations, and an AI gateway in one loop.

Key capabilities

  • End-to-end agent simulations run scenarios against the full stack, including tools, state, a user simulator, and a judge, to pinpoint where an agent breaks down to each decision.
  • The evaluation loop links traces to datasets, offline evaluation, prompt and model optimization, and re-testing, reducing custom integration code between separate tools.
  • The platform is OpenTelemetry and OTLP native, and it is designed to be framework agnostic and LLM-provider agnostic, which supports open standards.
  • The AI Gateway provides an OpenAI-compatible and Anthropic-compatible proxy with virtual keys, hierarchical budgets, inline guardrails, automatic fallback across providers, and Anthropic cache_control passthrough; it ships as a separate Go binary in services/aigateway/ with a Helm sub-chart in charts/gateway/, and the README states about 700 ns hot-path overhead.
  • Collaboration features include run review, failure annotation, annotation queues, GitHub prompt integration, and links between prompt versions and traces, giving domain experts and engineers shared evidence.

Who uses it and how

  • Teams developing LLM agents use LangWatch to run regression tests and simulations before release, then monitor behavior and cost in production.
  • Prompt and model users trace runs, collect them into datasets, evaluate them, adjust prompts or models, and re-test the changed agent.
  • Domain experts and reviewers use annotations and queues to label edge cases, while engineering teams keep prompts in Git through the GitHub integration and link versions to traces.
  • Self-hosting and cloud teams run LangWatch locally with Node.js or Docker Compose, deploy it on Kubernetes with Helm, or use a free account and API key.

Getting started

The hosted path is to create a free account, create a project, and copy the API key. Local paths are npx @langwatch/server, Docker Compose from the repository, and Kubernetes Helm deployment, with the UI at http://localhost:5560.

When to use it — and when not to

LangWatch fits teams that want an open-source, self-hostable platform for agent evaluation, observability, prompt management, and LLM gateway governance. Self-hosters must operate Postgres, Redis, ClickHouse, the AI gateway binary, and optional evaluators or assistants. A weakness is that the project is young, the repository lists zero contributors and 939 open issues, and the local Langy assistant runs workers unsandboxed.

project readme (upstream, from github) — read inline
012d1688-24ae-4759-ae70-5f8f81a13c0e

Website · Docs · Discord · Self-hosting

chat on Discord langwatch Python package on PyPi langwatch npm package follow on X Open-core: Apache 2.0 floor + Enterprise extension

Why LangWatch?

The platform for LLM evaluations and AI agent testing. We help teams test, simulate, evaluate, and monitor LLM-powered agents end-to-end — before release and in production. Built for teams that need regression testing, simulations, and production observability without building custom tooling.

LangWatch gives you full visibility into agent behavior and the tools to systematically improve reliability, performance, and cost, while keeping you in control of your AI system

Getting Started

Cloud ☁️

The easiest way to get started with LangWatch.

Create a free account → create a project → get started/ copy your API key.

Local setup 💻

The fastest way to run LangWatch locally — only Node.js required:

npx @langwatch/server

The CLI installs uv, postgres, redis, clickhouse, the AI gateway binary, and the Langy assistant's runtime into ~/.langwatch/, scaffolds a .env with locally-generated secrets, then starts every service in parallel and opens http://localhost:5560. Everything lives under ~/.langwatch/; rm -rf ~/.langwatch is a clean reset.

Three pieces are yours to decide on, in ~/.langwatch/.env:

Variable Default What it changes
LANGWATCH_ENABLE_LANGY true The Langy assistant. Adds ~45MB for its runtime; the workers run unsandboxed as you, on your own machine.
LANGWATCH_ENABLE_PRESIDIO false The PII detection evaluator. Adds ~670MB of language model, larger than the rest of the evaluator environment put together. LangWatch's own secret and PII redaction of your traces does not depend on it.
LANGWATCH_ENABLE_LINGUA false The language detection evaluator. Adds ~95MB of language models.

Every other evaluator is installed either way. Change any of these in ~/.langwatch/.env and restart the server.

Prefer Docker? You can still use docker compose:

git clone https://github.com/langwatch/langwatch.git
cd platform/app
cp platform/app/.env.example platform/app/.env
docker compose up -d --wait --build

Once running, LangWatch will be available at http://localhost:5560, where you can create your first project and API key.

Deployment options ⚓️

Run LangWatch on your own infrastructure:

  • Docker Compose - Run LangWatch on your own machine.
  • Kubernetes (Helm) - Run LangWatch on a Kubernetes cluster using Helm.
  • OnPrem - Cloud-specific setups for AWS, Google Cloud, and Azure.
Hybrid (OnPrem data) 🔀

For companies that have strict data residency and control requirements, without needing to go fully on-prem.

Read more about it on our docs.

Local Development 👩‍💻

You can also run LangWatch locally without docker to develop and help contribute to the project.

Start just the databases using docker and leave it running:

docker compose up redis postgres opensearch

Then, on another terminal, install the dependencies and start LangWatch:

make install
make start

🚀 Quick Start

Ship safer agents in minutes. Create a free account, then dive into these guides:

🗺️ Integrations

LangWatch builds and maintains several integrations listed below. Our tracing platform is built on top of OpenTelemetry, so we support any OpenTelemetry-compatible library out of the box.

Frameworks:
LangChain · LangGraph · Vercel AI SDK · Mastra · CrewAI · Google ADK

Model Providers:
OpenAI · Anthropic · Azure OpenAI · Google Vertex AI · AWS Bedrock · Groq · Ollama

Platforms

LangFlow · Flowise · n8n

and many more…

Are you using a platform that could benefit from a direct LangWatch integration? We'd love to hear from you, please fill out this very quick form.

💬 Support

Have questions or need help? We're here to support you in multiple ways:

  • Documentation: Our comprehensive documentation covers everything from getting started to advanced features.
  • Discord Community: Join our Discord server for real-time help from our team and community.
  • X (Twitter): Follow us on X for updates and announcements.
  • GitHub Issues: Report bugs or request features through our GitHub repository.
  • Enterprise Support: Enterprise customers receive priority support with dedicated response times. Our pricing page contains more information.

🤝 Collaborating

Contributions are what make the open-source community such an amazing place to learn, inspire, and create. Any contributions you make are greatly appreciated.

Please read our Contribution Guidelines for details on our code of conduct, and the process for submitting pull requests.

✍️ License: open-core split

LangWatch is Apache 2.0 and free to use, for individuals and for companies, self-hosted or not. The enterprise modules (SSO, SCIM provisioning, audit logs, gateway webhooks, billing, back-office) live under platform/app/ee/ and are the one part that needs a commercial license in production. The SDKs (typescript-sdk, python-sdk, mcp-server) are MIT.

See Editions and licensing for what a license adds, and LICENSE.md, NOTICE, platform/app/ee/LICENSE.md for the full text and the per-folder breakdown.

👮‍♀️ Security + Compliance

As a platform that has access to data that is highly likely to be sensitive, we take security incredibly seriously and treat it as a core part of our culture.

Legal Framework Current Status
GDPR Compliant. DPA available upon request.
ISO 27001 Certified. Certification report available upon request on our Enterprise plan.

Please refer to our Security page for more information. Contact us at security@langwatch.ai if you have any further questions.

Vulnerability Disclosure

If you need to do a responsible disclosure of a security vulnerability, you may do so by email to security@langwatch.ai, or if you prefer you can reach out to one of our team privately on Discord.

Frequently asked questions

Is LangWatch free to use?

LangWatch is open source under the Apache-2.0 licence. There is no licence fee and no seat count — you can self-host it or, where the project offers one, pay a vendor for a managed version instead.

What does LangWatch do?

Simulation-based testing and evaluation for AI agents

What is LangWatch written in?

LangWatch is primarily written in TypeScript. Its source is publicly available at https://github.com/langwatch/langwatch, and it has 5,003 GitHub stars.