LLM Gateway: Open Source AI Development Platforms

What is LLM Gateway?

What it is

LLM Gateway is an open-source API gateway for large language models. It acts as middleware between applications and model providers, and it is categorized in the AI and Machine Learning / AI Development Platforms ecosystem. The project is written in TypeScript, and its core functionality is licensed under AGPLv3, while features in the ee directory require an Enterprise license.

The concrete problem it addresses is operational fragmentation across LLM providers. Teams often need to route requests to OpenAI, Anthropic, Google Vertex AI, and other providers, manage provider API keys in one place, track token usage and costs, and analyze performance metrics. LLM Gateway provides a unified API interface compatible with the OpenAI API format, so existing applications can migrate without rewriting every provider call.

Key capabilities

LLM Gateway routes LLM requests to multiple providers through one gateway interface.
It manages API keys for different providers in a single place.
It tracks token usage, request counts, response times, and costs across interactions.
It provides a unified API interface compatible with the OpenAI API format.
It supports chat completion requests, including the example model gpt-4o through the /v1/chat/completions endpoint.
It compares model performance and cost-effectiveness for optimization decisions.
It includes enterprise-oriented features such as advanced billing, extended data retention, custom provider key configurations, and team or organization management under a separate license.

Who uses it and how

Application developers use it as an OpenAI-compatible endpoint by sending a bearer token and a JSON body containing model and messages.
Teams with several providers use it to centralize API key management and compare usage costs across models.
Self-hosters deploy the unified Docker image with named volumes for PostgreSQL and Redis, set AUTH_SECRET and GATEWAY_API_KEY_HASH_SECRET, and expose the gateway ports listed in the README.
Hosted users create an account at llmgateway.io, obtain an API key, and call the service without installing the stack.
Contributors run pnpm i and pnpm run setup to install dependencies, start Docker services, sync the database schema, and seed initial data before using pnpm dev.

Getting started

The README offers a hosted version at llmgateway.io and a self-hosted Docker deployment using ghcr.io/theopenco/llmgateway-unified:latest with named volumes and required secrets. Local development uses pnpm i, pnpm run setup, pnpm dev, and pnpm build, with Docker Desktop integration recommended for WSL2 users.

When to use it — and when not to

Use LLM Gateway when you need a single OpenAI-compatible interface for multiple LLM providers, centralized key management, and usage analytics. Do not choose it if you want to avoid operating PostgreSQL, Redis, Docker volumes, secrets, and multiple exposed ports, because the self-hosted path requires that infrastructure. The metadata also shows a zero-year repository age, zero listed contributors, and seventy-seven open issues, so teams needing a mature, widely maintained gateway should evaluate that risk.

project readme (upstream, from github) — read inline

LLM Gateway

LLM Gateway is an open-source API gateway for Large Language Models (LLMs). It acts as a middleware between your applications and various LLM providers, allowing you to:

Route requests to multiple LLM providers (OpenAI, Anthropic, Google Vertex AI, and others)
Manage API keys for different providers in one place
Track token usage and costs across all your LLM interactions
Analyze performance metrics to optimize your LLM usage

Features

Unified API Interface: Compatible with the OpenAI API format for seamless migration
Usage Analytics: Track requests, tokens used, response times, and costs
Multi-provider Support: Connect to various LLM providers through a single gateway
Performance Monitoring: Compare different models' performance and cost-effectiveness

Getting Started

You can use LLM Gateway in two ways:

Hosted Version: For immediate use without setup, visit llmgateway.io to create an account and get an API key.
Self-Hosted: Deploy LLM Gateway on your own infrastructure for complete control over your data and configuration.

Self-Hosted With Docker

Use Docker-managed volumes for the unified image. Do not bind-mount a host directory directly to /var/lib/postgresql/data, because PostgreSQL initialization inside the container needs to set permissions on that directory and that can fail depending on the host filesystem and ownership.

export LLM_GATEWAY_SECRET="$(openssl rand -base64 32 | tr -d '\n')"
export GATEWAY_API_KEY_HASH_SECRET="$(openssl rand -base64 32 | tr -d '\n')"
./scripts/run-unified-container.sh

If you prefer a one-off docker run, use named volumes instead of ~/some-host-dir:

docker volume create llmgateway_postgres
docker volume create llmgateway_redis

docker run -d \
  --name llmgateway \
  --restart unless-stopped \
  -p 3002:3002 \
  -p 3003:3003 \
  -p 3005:3005 \
  -p 3006:3006 \
  -p 3007:3007 \
  -p 4001:4001 \
  -p 4002:4002 \
  -v llmgateway_postgres:/var/lib/postgresql/data \
  -v llmgateway_redis:/var/lib/redis \
  -e AUTH_SECRET="$LLM_GATEWAY_SECRET" \
  -e GATEWAY_API_KEY_HASH_SECRET="$GATEWAY_API_KEY_HASH_SECRET" \
  ghcr.io/theopenco/llmgateway-unified:latest

Using LLM Gateway API

curl -X POST https://api.llmgateway.io/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $LLM_GATEWAY_API_KEY" \
  -d '{
  "model": "gpt-4o",
  "messages": [
    {"role": "user", "content": "Hello, how are you?"}
  ]
}'

Development Setup

Install dependencies and set up the development environment:
```
pnpm i && pnpm run setup
```
This will install all dependencies, start Docker services, sync the database schema, and seed initial data.

Note for WSL2 users: Ensure Docker Desktop is running with WSL integration enabled.
Start development servers:
```
pnpm dev
```
Build for production:
```
pnpm build
```

Folder Structure

apps/ui: Next.js dashboard frontend
apps/playground: Lounge — consumer AI chat app (Next.js)
apps/code: Next.js Dev Plans + coding tools landing & dashboard
apps/airside: Airside — self-serve provider portal (Next.js)
apps/api: Hono backend
apps/gateway: API gateway for routing LLM requests
apps/docs: Documentation site
ee/admin: Internal admin dashboard
packages/db: Drizzle ORM schema and migrations
packages/models: Model and provider definitions
packages/shared: Shared types and utilities

License

LLMGateway is available under a dual license:

Open Source: Core functionality is licensed under AGPLv3 - see the LICENSE file for details.
Enterprise: Commercial features in the ee/ directory require an Enterprise license; multi-organization administration requires a white-label license. See ee/LICENSE for details.

Enterprise features include:

Advanced billing and subscription management
Extended data retention (unlimited vs 30 days)
Custom provider key configurations
Team and organization management
Priority support
And more to be defined

For enterprise licensing, please contact us at [email protected]

Frequently asked questions

Is LLM Gateway free to use?

LLM Gateway is open source. There is no licence fee and no seat count — you can self-host it or, where the project offers one, pay a vendor for a managed version instead.

What does LLM Gateway do?

Unified API for all LLM providers with analytics

What is LLM Gateway written in?

LLM Gateway is primarily written in TypeScript. Its source is publicly available at https://github.com/theopenco/llmgateway, and it has 1,643 GitHub stars.