[opensourcedex]

best-of ranking · updated nightly

Best Open Source Machine Learning Infrastructure Tools

Ranked by GitHub stars — actual adoption, not editorial taste. All 10 are free, open source and self-hostable; license and last activity on every entry.

  1. 1
    Ollama ★ 186K

    Run open-source LLMs locally on your own machine

    MIT Go pushed 2026-11-02 details →
  2. 2
    llama.cpp ★ 133K

    Run LLMs locally with minimal setup, maximum hardware support

    MIT C++ pushed 2026-11-02 details →
  3. 3
    vllm ★ 96K

    A high-throughput and memory-efficient inference and serving engine for LLMs

    Apache-2.0 Python pushed 2026-11-02 details →
  4. 4
    GPT4All ★ 77K

    Run open-source AI models privately on your own device

    MIT C++ pushed 2025-05-27 details →
  5. 5
    serviette ★ 59K

    Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. 🐳Docker-friendly.⚡Always in sync with Sharepoint, Google Drive, S3, K

    MIT Python pushed 2026-11-02 details →
  6. 6
    llama_index ★ 53K

    LlamaIndex is the document processing platform for AI

    MIT Python pushed 2026-10-26 details →
  7. 7
    LocalAI ★ 50K

    Self-hosted AI runtime for text, voice, vision, and agents

    MIT Go pushed 2026-11-02 details →
  8. 8
    faiss ★ 41K

    A library for efficient similarity search and clustering of dense vectors.

    MIT C++ pushed 2026-11-01 details →
  9. 9
    PageIndex ★ 40K

    📑 PageIndex: Document Index for Vectorless, Reasoning-based RAG

    MIT Python pushed 2026-11-02 details →
  10. 10
    Langfuse ★ 37K

    Open source LLM engineering platform for AI-powered applications

    NOASSERTION TypeScript pushed 2026-11-02 details →
method stars desc · snapshot from github api refresh nightly related AI Security & Privacy, AI Interaction & Interfaces, AI Development Platforms

Related best-of lists

best AI Security & Privacy →best AI Interaction & Interfaces →best AI Development Platforms →

last updated nightly · ranking method: github stars, live-tracked

Looking for a replacement for one specific commercial product? Browse alternatives by tool →

Frequently asked questions

What is the best open source Machine Learning Infrastructure tool in 2026?

Ollama leads the ranking with 185,702 GitHub stars, ahead of llama.cpp. Rankings use live GitHub star counts, refreshed nightly.

How is this list of Machine Learning Infrastructure tools ranked?

By GitHub stars — a proxy for adoption and community size. Each entry also shows its licence, primary language and last commit date, so you can judge maintenance activity rather than trusting the ranking alone.

Are the best Machine Learning Infrastructure tools free?

Every project on this list is open source and free to self-host. Some also offer paid managed hosting, but the software itself carries no licence fee.