# ID-based RAG FastAPI

Overview

This project integrates Langchain with FastAPI in an Asynchronous, Scalable manner, providing a framework for document indexing and retrieval, using PostgreSQL/pgvector.

Files are organized into embeddings by file_id. The primary use case is for integration with LibreChat, but this simple API can be used for any ID-based use case.

The main reason to use the ID approach is to work with embeddings on a file-level. This makes for targeted queries when combined with file metadata stored in a database, such as is done by LibreChat.

The API will evolve over time to employ different querying/re-ranking methods, embedding models, and vector stores.

Features

• Document Management: Methods for adding, retrieving, and deleting documents.
• Vector Store: Utilizes Langchain's vector store for efficient document retrieval.
• Asynchronous Support: Offers async operations for enhanced performance.

Retrieval scope

Chunks are owned. Every route that reads or removes stored content resolves the caller's owner set from the verified token and puts it into the store query before ranking, so a chunk outside that set is never read into the process. The owner set is built in one place — app/scope.py — rather than re-derived per route.

Before this release these routes addressed the store by caller-supplied file_id alone, or authorized a whole result set from the first hit returned:

• GET /ids listed every file id in the deployment.
• POST /query_multiple performed no authorization at all, so pairing it with GET /ids disclosed the content of every file to any authenticated caller.
• POST /query authorized the whole result set from documents[0], so any hit behind the first was never checked. A file_id is chosen by whoever uploads, so an attacker's own row ranking first authorized the rows behind it.
• GET /documents, GET /documents/{id}/context and DELETE /documents read or deleted the chunks of any file id the caller could name.
• A chunk with no recorded user_id read as "belongs to everyone".
• On the synchronous store path, a failed ingestion rolled back by file_id alone, so an upload under someone else's file id destroyed their chunks. The async pgvector pipeline already scopes its rollback to the ingestion attempt.

What changes for callers. A caller reads and deletes only what it owns. A file id outside the caller's scope answers "not found" rather than "found but refused", so none of these routes is an existence oracle. Chunks with no user_id are owned by nobody and are no longer readable — if a deployment holds such rows and still needs them, stamp an owner on them before upgrading:

UPDATE langchain_pg_embedding
SET cmetadata = jsonb_set(cmetadata, '{user_id}', '"<owner>"')
WHERE cmetadata->>'user_id' IS NULL;

If this deployment ever ran without JWT_SECRET, check for public too. With no signing key configured there is no caller identity to record, so every chunk written in that period is owned by the literal string public. Once a signing key is set, callers arrive with their own ids and none of them owns public, so that content stops being readable. Routes other than /query returned it to everybody before this release, which is exactly the hole being closed — but if the content is still wanted, give it a real owner first:

-- inspect before rewriting: this is content nobody was ever identified as owning
SELECT count(*) FROM langchain_pg_embedding WHERE cmetadata->>'user_id' = 'public';

Deployments that never set JWT_SECRET are unaffected: with no key configured the read scope is public as well, so what was written is what is read.

atlas-mongo deployments must add user_id to the vector search index first; see Use Atlas MongoDB as Vector Database.

Deleting entity-owned files requires entity_id. Chunks embedded under an entity_id — an agent knowledge base, for instance — are owned by that entity rather than by the uploading user, so DELETE /documents needs the same entity_id that the upload used, as a query parameter alongside the JSON body of file ids. A delete that omits it resolves to the caller's own scope, matches nothing, and answers 404 with the chunks left in place. Because a 404 is indistinguishable from "already deleted", a caller that treats it as success will orphan those chunks silently.

Upgrade the client first. Deploy order matters, in one direction only:

• A client that sends entity_id against an older build is inert — the parameter is simply undeclared there, so the request behaves exactly as before.
• An older client against this build orphans every agent knowledge-base file it tries to delete.

So upgrade the client first, or both together — never this service first. LibreChat carries the matching change: it records the owner each embed was made under and sends it on delete, with npm run migrate:embed-owners to backfill files embedded before that.

entity_id is unchanged and still caller-asserted. Agent knowledge bases are owned by an agent id rather than a user id, so a caller reading one names it via entity_id. That id now widens the owner set rather than replacing the caller's identity — the caller's own scope always remains — but nothing in a token minted today proves the caller may act for the entity it names. A caller that knows another owner's id can still name it — on read, to reach that owner's chunks, and on the ingestion routes, where entity_id is what gets stamped as the owner, to write into that owner's namespace. Deployments exposing this API to untrusted callers must continue to authorize entity access upstream. Closing this requires the token to carry the entity authorization, which is a coordinated change with the callers that mint those tokens and is tracked separately from this release.

Setup

Getting Started

• Configure .env file based on section below
• Setup pgvector database:
  • Run an existing PSQL/PGVector setup, or,
  • Docker: docker compose up (also starts RAG API)
    • or, use docker just for DB: docker compose -f ./db-compose.yaml up
• Run API:
  • Docker: docker compose up (also starts PSQL/pgvector)
    • or, use docker just for RAG API: docker compose -f ./api-compose.yaml up
  • Local:
    • Make sure to setup DB_HOST to the correct database hostname
    • Run the following commands (preferably in a virtual environment)

pip install -r requirements.txt
uvicorn main:app

Clean Install (Local Development)

To do a clean reinstall of all dependencies (e.g., after updating requirements.txt):

# Remove existing virtual environment and recreate it
rm -rf venv
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

For the lite version (without sentence_transformers/huggingface):

rm -rf venv
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.lite.txt

For Docker, rebuild without cache:

docker compose build --no-cache

Environment Variables

The following environment variables are required to run the application:

• RAG_OPENAI_API_KEY: The API key for OpenAI API Embeddings (if using default settings).
  • Note: OPENAI_API_KEY will work but RAG_OPENAI_API_KEY will override it in order to not conflict with LibreChat setting.
• RAG_OPENAI_BASEURL: (Optional) The base URL for your OpenAI API Embeddings
• RAG_OPENAI_PROXY: (Optional) Proxy for OpenAI API Embeddings
  • Note: When using with LibreChat, you can also set HTTP_PROXY and HTTPS_PROXY environment variables in the docker-compose.override.yml file (see Proxy Configuration section below)
• VECTOR_DB_TYPE: (Optional) select vector database type, default to pgvector.
• POSTGRES_USE_UNIX_SOCKET: (Optional) Set to "True" when connecting to the PostgreSQL database server with Unix Socket.
• POSTGRES_DB: (Optional) The name of the PostgreSQL database, used when VECTOR_DB_TYPE=pgvector.
• POSTGRES_USER: (Optional) The username for connecting to the PostgreSQL database.
• POSTGRES_PASSWORD: (Optional) The password for connecting to the PostgreSQL database.
• DB_HOST: (Optional) The hostname or IP address of the PostgreSQL database server.
• DB_PORT: (Optional) The port number of the PostgreSQL database server.
• PGVECTOR_CREATE_EXTENSION: (Optional) Set to "False" to skip the CREATE EXTENSION IF NOT EXISTS vector call on startup. Default is "True". Use this when the vector extension is already installed on a managed Postgres (e.g. RDS, Azure Da