Open Source Machine Learning Infrastructure Tools

Ollama ★ 181K

Run open-source LLMs locally on your own machine

llama.cpp ★ 129K

Run LLMs locally with minimal setup, maximum hardware support

vllm ★ 92K

A high-throughput and memory-efficient inference and serving engine for LLMs

GPT4All ★ 77K

Run open-source AI models privately on your own device

llama_index ★ 52K

LlamaIndex is the document processing platform for AI

LocalAI ★ 49K

Self-hosted AI runtime for text, voice, vision, and agents

PageIndex ★ 36K

📑 PageIndex: Document Index for Vectorless, Reasoning-based RAG

Langfuse ★ 35K

Open source LLM engineering platform for AI-powered applications

cognee ★ 31K

Cognee is the open-source AI memory platform for agents. Give your AI agents persistent long-term memory across sessions with a self-hosted knowledge graph engi

taipy ★ 19K

Turns Data and AI algorithms into production-ready web applications in no time.

dagster ★ 16K

An orchestration platform for the development, production, and observation of data assets.

zvec ★ 16K

A lightweight, lightning-fast, in-process vector database

langchain4j ★ 13K

LangChain4j is an idiomatic, open-source Java library for building LLM-powered applications on the JVM. It offers a unified API over popular LLM providers and v

txtai ★ 13K

💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows

claude-context ★ 13K

Code search MCP for Claude Code. Make entire codebase the context for any coding agent.

OpenLLM ★ 13K

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

great_expectations ★ 12K

Always know what to expect from your data.

Arize Phoenix ★ 12K

Open-source LLM tracing & evaluation for AI optimization

lancedb ★ 11K

Developer-friendly OSS embedded retrieval library for multimodal AI. Search More; Manage Less.

paradedb ★ 9.3K

One Postgres for your application data, full-text search, vector retrieval, and aggregations. Home of the pg_search extension.

deep-searcher ★ 8.3K

Open Source Deep Research Alternative to Reason and Search on Private Data. Written in Python.

server ★ 8.2K

MariaDB server is a community developed fork of MySQL server. Started by core members of the original MySQL team, MariaDB actively works with outside developers

OpenLLMetry ★ 7.4K

Monitor LLM performance with open-source observability

vllm-omni ★ 6.9K

A framework for efficient model inference with omni-modality models

genkit ★ 6.4K

Open-source framework for building agentic apps in JavaScript, Go, Dart, and Python, built and used in production by Google

Helicone ★ 6.2K

Build reliable AI apps with comprehensive observability

gpustack ★ 5.7K

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

MineContext ★ 5.5K

MineContext is your proactive context-aware AI partner（Context-Engineering+ChatGPT Pulse）

Olares ★ 5.3K

Open-Source Personal Cloud OS for Always-On Agents

LangWatch ★ 4.8K

Simulation-based testing and evaluation for AI agents

Agenta ★ 4.8K

Build, automate, and improve AI agents with your team

infinity ★ 4.7K

The AI-native database built for LLM applications, providing incredibly fast hybrid search of dense vector, sparse vector, tensor (multi-vector), and full-text.

Latitude ★ 4.7K

Full observability for AI agents in production

LongMemory ★ 4.5K

Local persistent memory store for LLM applications including claude desktop, github copilot, codex, antigravity, etc.

lorax ★ 3.8K

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

FastDeploy ★ 3.7K

High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle

knowhere ★ 3.3K

Knowhere extracts, parses, and outputs structured chunks ready for AI Agents and RAG.

Laminar ★ 3.3K

AI-powered platform for engineering LLM products

chitu ★ 3.0K

High-performance inference framework for large language models, focusing on efficiency, flexibility, and availability.

datachain ★ 2.8K

The Context Layer for unstructured data: typed, versioned datasets over S3, GCS, Azure

OpenLIT ★ 2.8K

Monitor, debug, and scale LLM applications with ease

openlake ★ 2.7K

OpenLake is a high performance storage engine for efficient LLM inference and GPU Training

burr ★ 2.6K

Build applications that make decisions (chatbots, agents, simulations, etc...). Monitor, trace, persist, and execute on your own infrastructure.

dstack ★ 2.3K

GPU orchestration across clouds, Kubernetes, and on-prem

envd ★ 2.2K

🏕️ Reproducible development environment for humans and agents

Beam ★ 1.8K

Serverless GPU compute with sub-second cold starts

InferenceX ★ 1.7K

Open Source Continuous Inference Benchmark Research Platform — Kimi K3 2.8T, MiniMax M3, DeepSeekv4, GLM5 - GB200 NVL72 vs MI355X vs B200 vs GB300 NVL72 & soon™

gpu-hot ★ 1.6K

🔥 Real-time NVIDIA GPU dashboard

kitops ★ 1.4K

An open source DevOps tool from the CNCF for packaging and versioning AI/ML models, datasets, code, and configuration into an OCI Artifact.

sglang-omni ★ 1.2K

SGLang-Omni is a high-performance serving framework for audio models (TTS, ASR) and unified multimodal models.

lightly-studio ★ 889

LightlyStudio - The Unified Data Platform for Multimodal ML

ServerlessLLM ★ 715

Serverless LLM Serving for Everyone.

pegainfer ★ 704

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

ome ★ 509

Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton

mlop ★ 392

Experiment tracking platform for ML teams

Spanlens ★ 13

LLM observability: cost, latency, and traces in one place

Grepture ★ 8

Trace, evaluate, and protect every LLM call in one SDK