ollaya

Run open decision models locally, the way Ollama runs LLMs.

System 1 for AI agents: fast, typed, calibrated decisions for routing, guardrails, triage and tool approval.

Website · Models · Bench · Docs · Releases · Hugging Face

Created and maintained by Mert Cobanov (@mertcobanov).

A decision model reads a state (a message, an email, a ticket, any JSON) plus typed questions (choice, score, noul) and returns calibrated probabilities in a single forward pass, in milliseconds. It never generates text. Ollaya pulls these models by name and serves them from a local daemon behind a self-hosted, Jev-compatible API (TypeSafe's /v1/systemone wire format), so the TypeSafe SDK and existing Jev clients work unchanged once they point at Ollaya and name an open model.

curl -fsSL https://ollaya.dev/install.sh | sh
ollaya run winnow:e4b --preset triage "Third time this year you've double-charged me. Refund it today or I'm cancelling and moving to a competitor."

intent            refund                                ███████████████░ 0.91
is_urgent         yes                                   ███████████████░ 0.92
frustration       2.89 / 3  very angry or using stron…  ██████████████░░ 0.86
refund_requested  yes                                   ████████████████ 0.99
churn_risk        yes                                   ████████████████ 0.99

Or ask your own questions: `ollaya run kev --ask "Is a refund requested?" --ask "Which team?" --choice billing --choice sales

Install

• Linux (x86_64 or arm64, glibc ≥ 2.38, e.g. Ubuntu 24.04+): curl -fsSL https://ollaya.dev/install.sh | sh. When an NVIDIA GPU is present (driver R525+), the installer adds the CUDA runtime: CUDA 13 for R580+, CUDA 12 for older drivers.
• macOS (Apple silicon): the same command.
• Windows (x64): irm https://ollaya.dev/install.ps1 | iex in PowerShell. When an NVIDIA GPU is present (driver R527+), the installer adds the CUDA runtime, as on Linux.
• Desktop app for macOS, Windows and Linux: start and stop the server, download models and run them in one window. On macOS it lives in the menu bar. Get it from ollaya.dev/download.
• Docker: docker run -d --gpus=all -p 11435:11435 ghcr.io/ollaya-dev/ollaya:cuda (:cuda12 for host drivers older than R580), or ghcr.io/ollaya-dev/ollaya for CPU only.

Configuration is through environment variables: OLLAYA_HOST, OLLAYA_MODELS, OLLAYA_KEEP_ALIVE, OLLAYA_DEVICE, OLLAYA_API_KEY and others, listed in docs/api.md §15.

Repository

Development

cargo test --workspace
cargo build --release -p ollaya --features cuda   # CUDA build (x86-64 Linux and Windows)

Tests come in three tiers, all with the same parity tolerances:

• K1, every change: the tests, and smoke parity for the families the change touches. A parity example run with --smoke checks only the cases listed in crates/ollaya-runner/examples/smoke/cases.txt, about 40 questions per family: every question type, refused requests, a long state and many options.

  cargo run --release -p ollaya-runner --example parity --   cpu --smoke
  

• K2, every release: the full goldens on every GPU device a release ships, and on the CPU the full set for models of 4B and under and --smoke for larger ones. CPU goldens count only on the machine and build that made them.
• K3, major releases, a llama.cpp or ONNX Runtime update, a new device or a new model: full CPU parity of the large models and the quality benchmarks. The site, this README and the reports are updated from those results.

convert/ rebuilds models. It exports them, checks parity against the PyTorch reference, generates golden fixtures, and packages the result into registry/. See the module docstrings. cd convert && uv sync installs it: with CUDA 13 torch on Linux and Windows, and with the CPU and MPS build from PyPI on Apple silicon Macs, where exports and parity run on the CPU.

License

Apache-2.0. Each model keeps its own license: laya (Convai Innovations), decider (Mapika), kev (Jared Palmer, on Qwen3.5 by the Qwen team), decision (the vLLM Semantic Router contributors, on Qwen3.5), qwen3guard (Qwen team), gliclass (Knowledgator), von (Victor Hugo Panisa), winnow (EldanRing, on Gemma 4 by Google DeepMind), jevk5 (alibiserikbay, on Qwen3.5), nimble (Bespoke Labs, on Qwen3.5), pplx-decider (Perplexity, on Qwen3.5), strands-decider (Strands Agents, on Qwen3.5), jeeves (PostHog, on Qwen3.5), clef (Cloudflare, on Qwen3.5), jeb (Jason Brashear, on Qwen3.5 and Qwen3.8), cygnet (Gemma 4 by Google DeepMind; the Cygnet recipe is MIT), snap (logitlab, on MiniCPM5 by OpenBMB; snap's prompt is MIT), decima (A. M. Madani, on mmBERT-base by JHU CLSP and multilingual-e5-small by intfloat, both MIT), largha (Phocinae, on mmBERT-small by JHU CLSP, MIT) and nli:modernbert-large are Apache-2.0, and nli:deberta-v3-large (Moritz Laurer) is MIT. arbiter's LoRA adapter and head (Codekins Pvt Ltd, Zyot Lab) are Apache-2.0; its base model, Gemma 3 4B IT by Google DeepMind, is under the Gemma Terms of Use. nimble:9b-v3's adapter (Bespoke Labs) is CC BY-NC 4.0, which does not allow commercial use; its base, Qwen3.5-9B, is Apache-2.0. d1 (Liquid AI) is under the LFM Open License v1.0: commercial use by an organization with US$10 million or more in annual revenue is not covered by it. llama.cpp, which Ollaya ships for GGUF models, is MIT.

Ollaya is an independent project. It is not affiliated with or endorsed by Ollama or TypeSafe.