Envoy AI Gateway: Open Source AI Development Platforms

What is Envoy AI Gateway?

What it is

Envoy AI Gateway is an open source control plane for AI and agent traffic, powered by Envoy. The README says the project is now Agent Router, an Agentic AI Foundation project, while the directory entry still uses the Envoy AI Gateway name. It lives in the Envoy, Envoy Gateway, Kubernetes, API gateway, LLM, and AI gateway ecosystem.

The project solves the problem of routing generative AI traffic through one consistent interface. Agent Router gives application teams one OpenAI-compatible API for hosted providers, self-hosted inference, and MCP servers. Platform teams keep credentials, routing, quotas, failover, and usage attribution in one place, with enforcement by Envoy and Envoy Gateway.

Key capabilities

It exposes an OpenAI-compatible API for models and tools across supported providers.
It supports OpenAI, Azure OpenAI, Google Gemini, Vertex AI, AWS Bedrock, Mistral, Cohere, Groq, Together AI, DeepInfra, DeepSeek, Hunyuan, SambaNova, Grok, Anthropic, and Tetrate Agent Router Service.
It uses Envoy and Envoy Gateway to enforce routing, credentials, quotas, failover, and usage attribution.
It keeps Kubernetes resources compatible after the rename: AIGatewayRoute, AIServiceBackend, BackendSecurityPolicy, and the aigateway.envoyproxy.io API group remain unchanged.
It provides a standalone CLI, still named aigw, and a local router endpoint at http://localhost:1975/v1.
It supports a two-tier gateway pattern, with Tier One handling authentication, top-level routing, and global rate limiting, and Tier Two handling self-hosted model access with endpoint picker support.

Who uses it and how

Application teams use it when they need one API surface for many LLM providers, self-hosted inference, and MCP servers.
Platform teams use it to centralize credentials, routing, quotas, failover, and usage attribution behind Envoy and Envoy Gateway.
Kubernetes operators deploy it as a gateway control plane for AI traffic, using the unchanged CRDs and API group.
Developers use the CLI to run a local OpenAI-compatible router with an OPENAI_API_KEY environment variable.
Teams using self-hosted model serving can place a Tier Two Gateway in front of the serving cluster for fine-grained access control and LLM inference optimization.

Getting started

The README mentions container images under docker.io/envoyproxy/ai-gateway-* and Helm charts; the quick start runs the standalone router with OPENAI_API_KEY=sk-your-key aigw run and points to Kubernetes deployment with Envoy Gateway.

When to use it — and when not to

Use this project when an open source Envoy-based gateway is needed for AI traffic and the team is willing to operate Kubernetes, Envoy Gateway, credentials, routing, quotas, and failover configuration. It is less suitable when a hosted service with no self-hosted control plane is required, or when features beyond the OpenAI-compatible routing and provider list are needed. The rename to Agent Router is a trade-off: code and deployment artifacts remain compatible, but users must follow the new repository and website locations.

project readme (upstream, from github) — read inline

The open source control plane for AI and agent traffic, powered by Envoy.
An Agentic AI Foundation project. Formerly Envoy AI Gateway.

Documentation · Quickstart · Blog · Release Notes · Talks

Agent Router

Agent Router gives application teams one consistent, OpenAI-compatible API for every model and tool, from hosted providers to self-hosted inference and MCP servers. Platform teams keep credentials, routing, quotas, failover, and usage attribution in one place, enforced by Envoy and Envoy Gateway.

Agent Router controls. Envoy carries.

Formerly Envoy AI Gateway

Envoy AI Gateway is now Agent Router, an Agentic AI Foundation project. Same code, same maintainers, same release cadence and Apache 2.0 license. Nothing you deploy is renamed:

CRDs and API group are unchanged: AIGatewayRoute, AIServiceBackend, BackendSecurityPolicy, and the aigateway.envoyproxy.io API group stay as they are.
The CLI is still aigw, and the envoy-ai-gateway-system namespace is unchanged.
Container images, Helm charts, and the Go module path are unchanged (docker.io/envoyproxy/ai-gateway-*, github.com/envoyproxy/ai-gateway).
The repository moved to theagentrouter/agent-router; old envoyproxy/ai-gateway links redirect.
The website moved to theagentrouter.ai; every aigateway.envoyproxy.io link redirects to the same page.

Your manifests from yesterday apply tomorrow.

Quick start

Run Agent Router as a standalone, OpenAI-compatible router on your laptop with one command:

OPENAI_API_KEY=sk-your-key aigw run

Then point any OpenAI-compatible client at http://localhost:1975/v1. See the CLI guide for installation and provider auto-configuration, or the Getting Started guide to deploy on Kubernetes with Envoy Gateway.

Usage

When using Agent Router, we refer to a two-tier gateway pattern. The Tier One Gateway functions as a centralized entry point, and the Tier Two Gateway handles ingress traffic to a self-hosted model serving cluster.

The Tier One Gateway handles authentication, top-level routing, and global rate limiting
The Tier Two Gateway provides fine-grained control over self-hosted model access, with endpoint picker support for LLM inference optimization.

Supported AI Providers

Agent Router supports a wide range of AI providers, making it easy to integrate with your preferred LLM services:

_OpenAI	_{Azure OpenAI}	_{Google Gemini}	_{Vertex AI}	_{AWS Bedrock}
_Mistral	_Cohere	_Groq	_{Together AI}	_DeepInfra
_DeepSeek	_Hunyuan	_SambaNova	_Grok	_{Tetrate Agent Router Service}
_Anthropic

Documentation

Documentation for Agent Router.
Quickstart to use Agent Router in a few simple steps.
CLI guide to run Agent Router standalone with aigw run.
Concepts to understand the architecture and resources of Agent Router.
Blog for announcements, features, and reference architectures.
Talks and Presentations about Agent Router.

Contact

Chat: join the Agent Router Discord to ask questions and collaborate with the community.
Weekly meeting: the maintainers and community meet every Monday. Register, add agenda points, and get involved via the public meeting document.

Get Involved

We follow the LF Projects Code of Conduct, as an Agentic AI Foundation project.

To contribute to the project via pull requests, please read the CONTRIBUTING.md file which includes information on how to build and test the project.

Background

The proposal of using Envoy Gateway as a Cloud Native LLM Gateway inspired the initiation of this project.

Agent Router is an Agentic AI Foundation project, powered by Envoy. Copyright © Agent Router a Series of LF Projects, LLC. For web site terms of use, trademark policy and other project policies please see .

Frequently asked questions

Is Envoy AI Gateway free to use?

Envoy AI Gateway is open source under the Apache-2.0 licence. There is no licence fee and no seat count — you can self-host it or, where the project offers one, pay a vendor for a managed version instead.

What does Envoy AI Gateway do?

Route GenAI traffic with confidence using open source

What is Envoy AI Gateway written in?

Envoy AI Gateway is primarily written in Go. Its source is publicly available at https://github.com/envoyproxy/ai-gateway, and it has 2,115 GitHub stars.