Open Source Machine Learning Infrastructure Tools

Ollama ★ 186K

Run open-source LLMs locally on your own machine

llama.cpp ★ 133K

Run LLMs locally with minimal setup, maximum hardware support

vllm ★ 96K

A high-throughput and memory-efficient inference and serving engine for LLMs

GPT4All ★ 77K

Run open-source AI models privately on your own device

serviette ★ 59K

Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. 🐳Docker-friendly.⚡Always in sync with Sharepoint, Google Drive, S3, K

llama_index ★ 53K

LlamaIndex is the document processing platform for AI

LocalAI ★ 50K

Self-hosted AI runtime for text, voice, vision, and agents

faiss ★ 41K

A library for efficient similarity search and clustering of dense vectors.

PageIndex ★ 40K

📑 PageIndex: Document Index for Vectorless, Reasoning-based RAG

Langfuse ★ 37K

Open source LLM engineering platform for AI-powered applications

cognee ★ 33K

Cognee is the open-source AI memory platform for agents. Give your AI agents persistent long-term memory across sessions with a self-hosted knowledge graph engi

taipy ★ 19K

Turns Data and AI algorithms into production-ready web applications in no time.

xerj ★ 19K

XERJ is the new way for AI to search data. Its autoindex capability activates agents to know your data without the token waste of grep and sed. One command inde

dagster ★ 16K

An orchestration platform for the development, production, and observation of data assets.

zvec ★ 16K

A lightweight, lightning-fast, in-process vector database

langchain4j ★ 13K

LangChain4j is an idiomatic, open-source Java library for building LLM-powered applications on the JVM. It offers a unified API over popular LLM providers and v

txtai ★ 13K

💡 All-in-one AI framework for semantic search, LLM orchestration and language model workflows

claude-context ★ 13K

Code search MCP for Claude Code. Make entire codebase the context for any coding agent.

OpenLLM ★ 13K

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

Arize Phoenix ★ 12K

Open-source LLM tracing & evaluation for AI optimization

great_expectations ★ 12K

Always know what to expect from your data.

lancedb ★ 12K

Developer-friendly OSS embedded retrieval library for multimodal AI. Search More; Manage Less.

paradedb ★ 9.5K

One Postgres for your application data, full-text search, vector retrieval, and aggregations. Home of the pg_search extension.

server ★ 8.5K

MariaDB server is a community developed fork of MySQL server. Started by core members of the original MySQL team, MariaDB actively works with outside developers

deep-searcher ★ 8.4K

Open Source Deep Research Alternative to Reason and Search on Private Data. Written in Python.

OpenLLMetry ★ 7.5K

Monitor LLM performance with open-source observability

vllm-omni ★ 7.5K

A framework for efficient model inference with omni-modality models

genkit ★ 6.6K

Open-source framework for building agentic apps in JavaScript, Go, Dart, and Python, built and used in production by Google

Helicone ★ 6.3K

Build reliable AI apps with comprehensive observability

gpustack ★ 5.9K

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

MineContext ★ 5.5K

MineContext is your proactive context-aware AI partner（Context-Engineering+ChatGPT Pulse）

Olares ★ 5.3K

Open-Source Personal Cloud OS for Always-On Agents

LangWatch ★ 5.0K

Simulation-based testing and evaluation for AI agents

Agenta ★ 4.9K

Build, automate, and improve AI agents with your team

infinity ★ 4.8K

The AI-native database built for LLM applications, providing incredibly fast hybrid search of dense vector, sparse vector, tensor (multi-vector), and full-text.

Latitude ★ 4.7K

Full observability for AI agents in production

LongMemory ★ 4.6K

Local persistent memory store for LLM applications including claude desktop, github copilot, codex, antigravity, etc.

knowhere ★ 3.9K

Knowhere extracts, parses, and outputs structured chunks ready for AI Agents and RAG.

FastDeploy ★ 3.7K

High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle

Laminar ★ 3.5K

AI-powered platform for engineering LLM products

examples ★ 3.1K

Jupyter Notebooks to help you get hands-on with Pinecone vector databases

chitu ★ 3.0K

High-performance inference framework for large language models, focusing on efficiency, flexibility, and availability.

autoflow ★ 3.0K

pingcap/autoflow is a Graph RAG based and conversational knowledge base tool built with TiDB Serverless Vector Storage. Demo: https://tidb.ai

OpenLIT ★ 2.9K

Monitor, debug, and scale LLM applications with ease

datachain ★ 2.8K

The Context Layer for unstructured data: typed, versioned datasets over S3, GCS, Azure

openlake ★ 2.6K

OpenLake is a high performance storage engine for efficient LLM inference and GPU Training

burr ★ 2.6K

Build applications that make decisions (chatbots, agents, simulations, etc...). Monitor, trace, persist, and execute on your own infrastructure.

vearch ★ 2.3K

Distributed vector search for AI-native applications

dstack ★ 2.3K

GPU orchestration across clouds, Kubernetes, and on-prem

JustHireMe ★ 2.3K

Local-first AI job intelligence workbench for scraping roles, ranking fit, and generating tailored application materials.

envd ★ 2.2K

🏕️ Reproducible development environment for humans and agents

neuron-ai ★ 2.2K

The Agentic Framework of the PHP ecosystem to build production-ready AI driven applications. Connect components (LLMs, Tools, vector DBs, memory) to agents that

matrixone ★ 2.1K

AI-native HTAP database with Git-for-Data and built-in vector search, serving as the data and memory backbone for intelligent agents and applications.

mcp-memory-service ★ 2.1K

Open-source persistent memory for AI agent pipelines (LangGraph, CrewAI, AutoGen) and Claude. REST API + knowledge graph + autonomous consolidation.

kvcached ★ 2.0K

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

InferenceX ★ 2.0K

Open Source Continuous Inference Benchmark Research Platform — Kimi K3 2.8T, MiniMax M3, DeepSeekv4, GLM5 - GB200 NVL72 vs MI355X vs B200 vs GB300 NVL72 & soon™

Beam ★ 1.8K

Serverless GPU compute with sub-second cold starts

opencode-mem ★ 1.8K

OpenCode plugin that gives coding agents persistent memory using local vector database

LLPhant ★ 1.7K

LLPhant - A comprehensive PHP Generative AI Framework using OpenAI GPT 4. Inspired by Langchain

dingo ★ 1.7K

A multi-modal vector database that supports upserts and vector queries using unified SQL (MySQL-Compatible) on structured and unstructured data, while meeting t

sglang-omni ★ 1.7K

SGLang-Omni is a high-performance serving framework for audio models (TTS, ASR) and unified multimodal models.

gpu-hot ★ 1.6K

🔥 Real-time NVIDIA GPU dashboard

kitops ★ 1.4K

An open source DevOps tool from the CNCF for packaging and versioning AI/ML models, datasets, code, and configuration into an OCI Artifact.

ollaya ★ 1.4K

Jev-compatible API server for open decision models. Ollama for decision models: pull and serve 25 model families (Laya, pplx-decider, Clef, Winnow, coba...) loc

qdrant-client ★ 1.4K

Python client for Qdrant vector search engine

infinispan ★ 1.4K

Infinispan is an open source data grid platform and highly scalable NoSQL cloud data store.

SeaGOAT ★ 1.3K

local-first semantic code search engine

arcadedb ★ 1.2K

ArcadeDB Multi-Model Database, one DBMS that supports SQL, Cypher, Gremlin, HTTP/JSON, MongoDB and Redis. ArcadeDB is a conceptual fork of OrientDB, the first M

VectorDBBench ★ 1.2K

Benchmark for vector databases.

Agent_Memory_Techniques ★ 1.1K

Agent memory for LLMs: 30 runnable Jupyter notebooks covering conversation buffers, vector stores, knowledge graphs, episodic and semantic memory, MemGPT, Mem0,

kividb ★ 1.1K

A fast, multi-threaded Redis® and Valkey alternative written in Rust: drop-in compatible, with full-text and vector search built in

selfhost-ai ★ 953

🚀 Self-hosted AI automation platform. Deploy n8n, Ollama, Flowise, RAG, Supabase & 30+ tools with one command. Auto HTTPS. Free Zapier/Make alternative.

rag_api ★ 920

ID-based RAG FastAPI: Integration with Langchain and PostgreSQL/pgvector

lightly-studio ★ 902

LightlyStudio - The Unified Data Platform for Multimodal ML

llamafarm ★ 836

Deploy any AI model, agent, database, RAG, and pipeline locally or remotely in minutes

pegainfer ★ 731

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

ServerlessLLM ★ 718

Serverless LLM Serving for Everyone.

RAGLight ★ 673

RAGLight is a modular framework for Retrieval-Augmented Generation (RAG). It makes it easy to plug in different LLMs, embeddings, and vector stores, and now inc

next-plaid ★ 608

NextPlaid, ColGREP: Multi-vector search, from database to coding agents.

embedJs ★ 601

A NodeJS RAG framework to easily work with LLMs and embeddings

Compartment ★ 580

Encrypted, fully offline agentic memory. One click install, GUI w/ memory map, all OS and agents. Superior memory creation, storage and retrieval.

langtest ★ 561

Deliver safe & effective language models

unbody ★ 524

The Supabase of AI era. A modular, open-source backend for building AI-native software — designed for knowledge, not static data.

ome ★ 523

Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton

popmon ★ 512

Monitor the stability of a Pandas or Spark dataframe ⚙︎

mlop ★ 394

Experiment tracking platform for ML teams

Spanlens ★ 13

LLM observability: cost, latency, and traces in one place

Grepture ★ 12

Trace, evaluate, and protect every LLM call in one SDK