[opensourcedex]

openlake is a free, open source machine learning infrastructure project written in Rust and released under Apache-2.0. It has 2,662 GitHub stars, 424 forks and 138 open issues, and was last pushed 3 days ago. On this registry it ranks #42 of 57 tracked projects in Machine Learning Infrastructure, with 5 head-to-head comparisons available. It gained 55 stars over the last 3 tracked days.

What is openlake?

What it is

OpenLake is a Rust storage engine for AI and machine learning infrastructure, aimed at LLM inference and GPU training. It provides distributed storage for GPU workloads and uses io_uring to keep accelerators fed during serving and training.

The problem it addresses is GPU idle time caused by slow or small I/O storage paths. OpenLake gives cache-like performance while remaining persistent and durable, so inference engines can offload KV cache to host RAM and disk, read it back quickly, and avoid repeated prefill.

Key capabilities

  • OpenLake is a distributed storage engine written in Rust and built on io_uring, with claimed million-plus iops within one millisecond.
  • It offloads LLM KV cache to GPU host RAM and disk, so an inference engine can write KV once and read it back in milliseconds for long or repeated prompts.
  • It supports checkpointing for reinforcement learning and machine learning workloads, and reports a top MLPerf Storage v3.0 2026 object checkpointing result.
  • It includes ExANS, a lossless GPU codec for BF16 KV cache, available in version 0.8 with a claimed 1.51x cost saving.
  • It targets large KV-cache workloads, including 100 TB KV cache management and 8x inference throughput with deferred materialization.
  • It integrates with vLLM through the openlake-vllm connector and a kv-transfer configuration, without code changes for the basic KV pool setup.

Who uses it and how

  • vLLM operators can install openlake-vllm and openlaked, then run vLLM with OpenLakeConnector enabled to offload KV cache locally.
  • Multi-host GPU cluster operators can start openlaked with kv_rdma.toml and a per-node self_id, then point vLLM workers at a unified cluster over InfiniBand.
  • Kubernetes administrators can use the Helm KV deployment guide to deploy one OpenLake instance per selected node and generate ordered vLLM peer configuration.
  • Training, reinforcement learning, and agentic inference teams can use checkpointing, context storage, and massive conversation retrieval workflows.

Getting started

Typical setup uses pip install openlake-vllm openlaked, then runs vllm serve with an OpenLake kv-transfer-config. For fleet or Kubernetes use, operators start openlaked with a configuration file or use the Helm KV deployment guide.

When to use it — and when not to

OpenLake is useful when teams need persistent KV offload, checkpointing, or GPU-workload storage in self-hosted AI infrastructure with vLLM. It is less suitable when teams want to avoid operating storage nodes, because they must run openlaked, configure host RAM and disk, manage node peers, and integrate the connector. The young repository, 138 open issues, and missing contributor list make it less attractive for teams needing mature operational support.

project readme (upstream, from github) — read inline
OpenLake

Fast, easy and efficient storage for LLM Inference and Training

| Blog | Documentation | Comparision | Discord/X | Website |

License Rust Discord Web

🔥 Distributed storage for GPU workloads. Built on Rust on io_uring, OpenLake is a state of the art storage engine delivering million+ iops within 1ms.

Updates

  • [2026/09] 🔥 OpenLake tops the MLPerf Storage v3.0 2026 object checkpointing, leading NVIDIA and Nebius (blog).
  • [2026/08] ExANS: a lossless GPU codec for BF16 KV cache: 1.51× cost savings, now available in OpenLake v0.8 (blog).
  • [2026/07] Breaking the KV Wall: managing 100 TB of KV cache and 8× inference throughput with deferred materialization (blog).
Older updates
  • [2026/07] Introducing OpenLake: open source storage that saturates the GPUs: 8× throughput and 600 µs reads (blog).

Why OpenLake?

OpenLake is a storage engine for AI infrastructure. With OpenLake you get high throughput for small I/O and cache like performance while being fully persistent and durable. Keep GPUs fed during training and inference reducing idle time and getting more from your accelerators.

OpenLake is fast with:

  • KV Cache Offload. Reduced LLM Inference costs by having Petabyte scale KV cache store co-located on GPU hosts.
  • VectorDB: Fast index building and vector serving.
  • Checkpointing: Ultra fast checkpoint storage and retrieval for RL and ML workloads.
  • Model Training: Small file I/O and fast random reads, reduced GPU costs/training time.
  • Context Storage: Store massive conversations, memories and context for fast agentic retrieval.

Learn more: Blogs → | Benchmarks → | KV Offload | Object Store


66× speedup on time to first token first token when cached. (128K context window)

GPU nodes contribute to create an OpenLake cluster. The inference engine writes KV once and reads it back in milliseconds (using the host RAM and disk), saving prefill for long and repeated prompts.

Quickstart:

1) Setup KV Pool on GPU nodes:

Drop OpenLake into your existing setup. No code changes:

a. Install the connector and start the store:
pip install openlake-vllm
openlaked
b. Run vLLM with OpenLake enabled:
export PYTHONHASHSEED=0
vllm serve <model_name> --kv-transfer-config '{"kv_connector":"OpenLakeConnector","kv_connector_module_path":"openlake_client.openlake_connector","kv_role":"kv_both","kv_connector_extra_config":{"openlake_nodes":["127.0.0.1:9400"],"openlake_device":"local"}}'

Note: By default OpenLake offloads to the same host. To enable OpenLake across your GPU fleet, please start openlaked with a --config.

For Kubernetes clusters, use the Helm KV deployment guide to place one OpenLake instance on each selected node and generate the ordered vLLM peer configuration.

OpenLake enabled vs disabled:

openlake-video

OpenLake and vLLM serving Gemma4-31B on H100 (256K context window)

Multi host GPU Cluster (IB)

Run OpenLake on existing GPU cluster. Run OpenLake with (kv_rdma.toml) and node's self_id: (0, 1, 2...):

openlaked --config kv_rdma_0.toml   # gpu 1: ids = 0
openlaked --config kv_rdma_1.toml   # gpu 2: ids = 1

Point your vLLM workers at the unified cluster, in id order:

cat > /tmp/openlake-kv.json <<'EOF'
{
  "kv_connector": "OpenLakeConnector",
  "kv_connector_module_path": "openlake_client.openlake_connector",
  "kv_role": "kv_both",
  "kv_connector_extra_config": {
    "openlake_nodes": ["10.0.0.1:9400", "10.0.0.2:9400"],
    "openlake_device": "mlx5_ib0"
  }
}
EOF

export PYTHONHASHSEED=0
vllm serve <model_name> --kv-transfer-config "$(cat /tmp/openlake-kv.json)"

A prefix computed on one GPU host is served to any other from the shared pool.

2) PB scale object store for GPU Fleet

Build from source and have an S3 compatible store running in four steps. Install the dependencies and build the OpenLake binary locally.

a. Clone and build:
sudo apt-get install -y build-essential pkg-config clang cmake libhwloc-dev libudev-dev curl git awscli
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && . "$HOME/.cargo/env"

git clone https://github.com/openlake-project/openlake.git && cd openlake
cargo build --release --bin openlaked
b. Start the store (single node, default config):
mkdir -p data/d0 data/d1 data/d2 data/d3
./target/release/openlaked --config crates/openlake_server/configs/storage-tcp-local.toml
Talk to it with any S3 client:
export AWS_ACCESS_KEY_ID=openlakeadmin
export AWS_SECRET_ACCESS_KEY=openlakeadmin
export AWS_DEFAULT_REGION=us-east-1

aws --endpoint-url http://127.0.0.1:9000 s3 mb s3://demo
aws --endpoint-url http://127.0.0.1:9000 s3 cp ./checkpoint.safetensors s3://demo/
aws --endpoint-url http://127.0.0.1:9000 s3 ls s3://demo/

Build from source:

To build from the source, please follow the platform specific build guides:

Ubuntu / Debian

Produce binaries for your deployment or test code changes.

# Install system dependencies
sudo apt-get update
sudo apt-get install -y --no-install-recommends ca-certificates build-essential pkg-config clang cmake libhwloc-dev libudev-dev curl git

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && . "$HOME/.cargo/env"

# Clone and build OpenLake
git clone https://github.com/openlake-project/openlake.git
cd openlake && cargo build --release --locked -p openlake_server --bin openlaked

# Create the local storage directories
mkdir -p data/d0 data/d1 data/d2 data/d3

# Start OpenLake in TCP mode. Please switch the config path for RDMA.
RUST_LOG=info ./target/release/openlaked --config crates/openlake_server/configs/storage-tcp-local.toml
macOS (development)

Install Homebrew first if brew is unavailable.

# Install system dependencies
xcode-select --install
brew install cmake pkg-config

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && . "$HOME/.cargo/env"

# Clone and build OpenLake
git clone https://github.com/openlake-project/openlake.git
cd openlake && cargo build --release --locked -p openlake_server --bin openlaked

# Create local storage directories
mkdir -p data/d0 data/d1 data/d2 data/d3

# Start OpenLake in TCP mode. (macOS does not provide the Linux RDMA interfaces)
RUST_LOG=info ./target/release/openlaked --config crates/openlake_server/configs/storage-tcp-local.toml
Windows (WSL2)

Build OpenLak

readme truncated — read the full docs on github

Frequently asked questions

Is openlake free to use?

openlake is open source under the Apache-2.0 licence. There is no licence fee and no seat count — you can self-host it or, where the project offers one, pay a vendor for a managed version instead.

What does openlake do?

OpenLake is a high performance storage engine for efficient LLM inference and GPU Training

What is openlake written in?

openlake is primarily written in Rust. Its source is publicly available at https://github.com/openlake-project/openlake, and it has 2,662 GitHub stars.