Chonkie is a lightweight, MIT-licensed Python ingestion library that turns raw documents into retrieval-ready chunks for RAG pipelines, and it is built for developers who are tired of writing yet another custom chunker or absorbing the overhead of a large framework just to split text.

What it is

Chonkie lives in the Python AI and machine learning ecosystem, where it occupies the ingestion stage of a retrieval-augmented generation stack. It provides chunkers that split text into pieces suited to embedding and similarity search, refiners that post-process those chunks, and a chonkie.Pipeline abstraction that chains the steps together. A pipeline can fetch, chunk, refine, embed and ship straight to a vector database, and the library states that it works out of the box with over 32 integrations covering common tools and vector stores. Semantic chunking, similarity search, and recursive splitting are all part of the shipped feature set.

The concrete problem it solves is the repeated, unrewarding work of hand-rolling a chunker for every project. The README opens by asking whether the reader is tired of making their gazillionth chunker and sick of the overhead of large libraries, which frames the project as a direct replacement for ad hoc splitting code and for heavyweight frameworks used only for their text-splitting utilities. It also targets efficiency: the package is 505KB, and installation is deliberately modular so only the components a project actually needs are pulled in.

Key capabilities

• Chunkers exposed as importable classes, such as RecursiveChunker, which is called on a string and returns chunk objects carrying .text and .token_count.
• chonkie.Pipeline for composing workflows, for example chunk_with("recursive", tokenizer="gpt2", chunk_size=2048, recipe="markdown") followed by chunk_with("semantic", chunk_size=512).
• Refinement stages including refine_with("overlap", context_size=128) and refine_with("embeddings", embedding_model="sentence-transformers/all-MiniLM-L6-v2").
• Asynchronous execution through pipe.arun() for high-throughput applications.
• Over 32 integrations with tools and vector databases, plus end-to-end fetch, chunk, refine, embed and ship behaviour.
• Out-of-the-box multilingual support covering 56 languages.
• Choice of execution location, either locally or in Chonkie Cloud at labs.chonkie.ai.

Who uses it and how

• RAG teams that need a complete ingestion path, from fetching source documents through to shipping embedded chunks into a vector database.
• Projects that avoid the bloat of large libraries by installing only the extras each chosen chunker requires.
• Multilingual corpora, since the library ships support for 56 languages without extra configuration.
• Services processing text asynchronously at high throughput, using the arun entry point on a shared pipeline object.
• Documentation and markdown ingestion, where the recipe="markdown" option drives recursive chunking against document structure.

Getting started

Install with pip install chonkie or the faster uv pip install chonkie. To pull in every component at once, install pip install "chonkie[all]", although the README notes that the full install is not recommended for production environments; documentation lives at docs.chonkie.ai.

How it compares

Chonkie stands alone in this registry: no other entry is listed as a comparable tool, and the project's own materials name no direct alternatives. Its positioning is defined against the general problem of repeated custom chunkers and the import overhead of large libraries, rather than against a named set of competing products.

When to use it — and when not to

A team adopting Chonkie must choose its optional dependencies deliberately, because the library follows a rule of minimum installs and expects the reader to consult external documentation to pick the right extras; the chonkie[all] shortcut is explicitly discouraged for production. Anyone wanting a single batteries-included install with no per-chunker decisions, or an ingestion service that runs entirely without writing code, is better served elsewhere, though the Cloud option at labs.chonkie.ai narrows that gap. The repository is otherwise healthy and actively maintained, with a permissive MIT licence and a substantial integration surface.

pip install chonkie

uv pip install chonkie

pip install "chonkie[all]"

uv pip install "chonkie[all]"

# First import the chunker you want from Chonkie
from chonkie import RecursiveChunker

# Initialize the chunker
chunker = RecursiveChunker()

# Chunk some text
chunks = chunker("Chonkie is the goodest boi! My favorite chunking hippo hehe.")

# Access chunks
for chunk in chunks:
    print(f"Chunk: {chunk.text}")
    print(f"Tokens: {chunk.token_count}")

from chonkie import Pipeline

# Create a pipeline with multiple chunking and refinement steps
pipe = (
    Pipeline()
    .chunk_with("recursive", tokenizer="gpt2", chunk_size=2048, recipe="markdown")
    .chunk_with("semantic", chunk_size=512)
    .refine_with("overlap", context_size=128)
    .refine_with("embeddings", embedding_model="sentence-transformers/all-MiniLM-L6-v2")
)

# CHONK some Texts!
doc = pipe.run(texts="Chonkie is the goodest boi! My favorite chunking hippo hehe.")

# Access the processed chunks in the `doc` object
for chunk in doc.chunks:
    print(chunk.text)

# Run asynchronously for high-throughput applications
import asyncio

async def main():
    doc = await pipe.arun(texts="Chonkie runs fast!")
    print(len(doc.chunks))

asyncio.run(main())

# Install with API dependencies (includes catsu for multi-provider embeddings)
pip install "chonkie[api,semantic,code,catsu]"

# Start the server using the CLI
chonkie serve

# Or with custom options
chonkie serve --port 3000 --reload --log-level debug

# Or directly with uvicorn
uvicorn chonkie.api.main:app --host 0.0.0.0 --port 8000

docker compose up

# Create a reusable pipeline
curl -X POST http://localhost:8000/v1/pipelines \
  -H "Content-Type: application/json" \
  -d '{
    "name": "rag-chunker",
    "steps": [
      {"type": "chunk", "chunker": "semantic", "config": {"chunk_size": 512}},
      {"type": "refine", "refinery": "embeddings", "config": {"embedding_model": "text-embedding-3-small"}}
    ]
  }'

# List your pipelines
curl http://localhost:8000/v1/pipelines