What it is

Langchain-Chatchat (formerly Langchain-ChatGLM) is an open-source RAG and Agent application built on the Langchain framework and designed to run against local large language models such as ChatGLM, Qwen, and Llama. It lives in the Python ecosystem, is distributed under the Apache-2.0 license, and is published on PyPI as langchain-chatchat. The project targets Chinese-language scenarios and open-source models specifically, with the stated goal of providing a knowledge-base question-answering solution that can run entirely offline.

The concrete problem it solves is retrieval-augmented generation over private documents without sending data to a hosted API. The pipeline loads files, reads text, splits it, vectorizes chunks and the incoming question, matches the top-k most similar vectors, and injects the matched text as context into the prompt before the LLM generates an answer. This lets an operator point a local model at an internal corpus and get grounded answers, rather than relying on a model's parametric memory or a third-party service.

Key capabilities

• Retrieval-augmented generation over a local knowledge base, with the full load, split, embed, match, and prompt pipeline implemented in Langchain.
• Agent application support alongside the RAG workflow, as stated in the project description.
• Model access through Xinference and Ollama, covering GLM-4-Chat, Qwen2-Instruct, and Llama3.
• A model_provider abstraction for local models and oneapi for online models, introduced in the 0.3.x line.
• FastAPI-based API service for programmatic calls and a Streamlit WebUI for interactive use.
• Support for mainstream open-source LLM, embedding, and vector database backends, with FAISS listed among the project topics.
• Optional OpenAI GPT API calls in addition to the fully local path.

Who uses it and how

• Teams that need an internal document question-answering system deployed on private infrastructure, using only open-source models.
• Chinese-language deployments, since the project explicitly optimizes for Chinese scenarios and open-source model support.
• Operators who want to swap model backends, using Xinference or Ollama to serve GLM-4-Chat, Qwen2-Instruct, or Llama3 behind the same application.
• Developers who need an HTTP interface, calling the FastAPI service rather than the Streamlit UI.
• Users who prefer a prebuilt environment, via the AutoDL image whose 0.3.0 version tracks the project's v0.3.0 release.

Getting started

The README documents three deployment paths: pip installation, source or development installation, and Docker deployment, with Python 3.8 through 3.11 supported. An AutoDL image for version 0.3.0 is available, and a Docker image update is noted as forthcoming.

When to use it — and when not to