Flide is an Apache-2.0, Python-based inference engine for Jev-style decision models that serves the /v1/systemone API and uses exact state forking to return, for each question, exactly the answer the model gives that question alone.

What it is

Flide sits in the machine-learning infrastructure layer, specifically in model serving. It is written in Python, licensed under Apache-2.0, and at the time of writing carries 505 stars, 33 forks and 2 open issues. Rather than acting as a general-purpose chat server, it targets decision models in the Jev family and exposes them through the Jev /v1/systemone API. Its distinguishing mechanism is exact state forking: the engine forks model state per question instead of reprocessing a shared context, which the project reports makes it up to 9.5 times faster than vLLM and SGLang while keeping every answer identical to the one the model produces for that question in isolation. The method is documented in the paper "Flide: Accelerating Decision Inference with Exact State Forking" on arXiv.

The concrete problem it solves is the shape of decision workloads, where one input is asked many questions. The project's email triage demo illustrates the pattern: Flide tags 10,000 emails with eight questions each in 28 seconds on a single H200. General-purpose serving stacks such as vLLM and SGLang are the layer it replaces for this kind of work, and the topic list — deterministic-inference, kv-cache, linear-attention, cuda, ai-agents, llm-agents — shows where the engineering effort goes. Determinism is the point: agent and decision pipelines need repeatable answers, not sampled ones.

Key capabilities

• Serves the Jev /v1/systemone API, which since version 0.1.9 accepts several requests in a single body.
• Implements exact state forking, with the flide verify command to check that forking is exact on your own model and your own requests.
• Reports up to 9.5 times the speed of vLLM and SGLang on decision inference, per the project's own benchmarks.
• Ships precompiled CUDA kernels for RTX 40 series, RTX 50 series and RTX PRO 6000 GPUs, with the 40 series and 50 series support added in versions 0.1.13 and 0.1.12 respectively.
• Offers FP8 quantization through the --quantize fp8 flag, which lets Gemma 4 12B fit on a single 24 GB card.
• Accepts image input for several model families, including GLiDE-27B and the KnowLine family, as well as Lumma and mammoth.
• Covers a wide model roster, among them d1-3B from LiquidAI, GLiDE-9B and GLiDE-27B, Sieve-31B-v3, JEV-35B, pplx-decider-v1.1-27b, Drex, Xor, and Gemma 4 mixture-of-experts models such as KnowLine-26B-A4B.

Who uses it and how

• Teams building LLM agents and automated decision systems, where the same record is evaluated against a fixed set of questions and answer stability matters more than open-ended generation.
• Teams already serving decision models on vLLM or SGLang who want to benchmark an engine built specifically for forked, per-question inference.
• Single-GPU deployments: a 24 GB card running Gemma 4 12B under FP8, or one H200 handling the 10,000-email, eight-question triage batch in 28 seconds.
• Operators on workstation-class hardware, who get precompiled kernels for RTX 40 series, RTX 50 series and RTX PRO 6000 cards rather than building kernels themselves.
• Model publishers, since new decision model families have been added at a rapid pace across releases 0.1.5 through 0.1.15, frequently several families per release.

Getting started

The README's Quick start section, presented alongside Models and Options, is the entry point, with fuller documentation hosted at docs.flide.org. The excerpt given here does not spell out a package, image or compose file name beyond the flide command-line tool itself, which is also used for flide verify and accepts serving flags such as --quantize fp8.

How it compares

The README benchmarks Flide directly against vLLM and SGLang, positioning it as a faster engine for decision inference rather than as a replacement for general-purpose serving. Both of those projects serve broad text-generation workloads, whereas Flide is shaped around the Jev /v1/systemone contract and exact state forking. It is Apache-2.0 licensed and self-hosted, so the trade is operational: you run the GPU capacity and the engine yourself.

When to use it — and when not to

Choose Flide when the workload is many questions over the same input, you need deterministic answers, and you have CUDA GPUs to run it on — the topic list and precompiled kernel notes are entirely GPU-oriented, so there is no evident CPU-only path. Do not pick it as a drop-in replacement for a general chat or completions server, since its API surface is the Jev /v1/systemone endpoint rather than a broad text-generation interface. The project is also young at version 0.1.x with frequent releases, so teams wanting a settled, slow-moving serving stack should weigh that against the speed claims, which come from the project's own benchmarks and its arXiv paper.

uv venv -p 3.12 && source .venv/bin/activate
uv pip install flide torchvision
flide serve --model OmniJev/OneJev1.6-0.8B

flide serve --model perplexity-ai/pplx-decider-flash-v1.2

curl http://localhost:8000/v1/systemone -H "Content-Type: application/json" -d '{
  "state": "Order #1182 arrived with a cracked screen. The customer asks for a refund and attached two photos.",
  "questions": {
    "action": {"type": "choice", "instructions": "What should support do next?",
               "criteria": {"refund": "refund the order", "replace": "send a replacement", "escalate": "ask a human"}},
    "urgent": {"type": "noul", "instructions": "Does this need a reply within the hour?"}
  }
}'

@article{li2026flide,
  title   = {Flide: Accelerating Decision Inference with Exact State Forking},
  author  = {Li, Bobo and Song, Jiajia and Fei, Hao and Hsu, Wynne and Lee, Mong Li},
  journal = {arXiv preprint arXiv:2610.42148},
  year    = {2026}
}