aws-genai-llm-chatbot is an MIT-0 licensed TypeScript blueprint from aws-samples that deploys a multi-LLM, multi-RAG chatbot inside a customer's own AWS account, built for teams that want generative AI chat grounded in private data without assembling the surrounding infrastructure themselves.

What it is

It is a reference solution, defined as infrastructure as code, that stands up a complete retrieval-augmented generation chatbot on AWS. A single deployment brings together Amazon Bedrock for large language model access, Amazon OpenSearch for vector storage, Amazon S3 for document storage, Amazon Cognito for authentication, AWS Lambda for serverless processing, Amazon API Gateway for API access, and a React-based web interface. The whole stack is expressed in TypeScript and provisioned through AWS CDK, which the README pairs with SeedFarmer to make deployment fully automated.

The concrete problem it solves is the assembly work that otherwise sits in front of every RAG chatbot project on AWS. Getting a working system requires model access, a vector index, an identity provider, document ingestion, conversation persistence, and some accounting of what the inference costs, and those pieces are normally wired together by hand per project. The blueprint replaces that bespoke glue with one deployable definition, and it lives squarely in the AWS ecosystem, so it assumes and consumes AWS-native services rather than abstracting over them.

Key capabilities

• Multiple LLM backends through Amazon Bedrock, including Claude and Llama 2, plus SageMaker and custom model endpoints.
• GenAIEH Gateway integration for access to additional models beyond those configured directly.
• Retrieval Augmented Generation over connected data sources, with vector storage in Amazon OpenSearch and the opensearch-serverless and pgvector topics indicating the supported index options.
• Conversation memory with full history kept in persistent storage.
• A React web interface alongside API endpoints for programmatic integration.
• Enterprise security controls covering fine-grained access control, audit logging, and data encryption, with Amazon Cognito handling authentication.
• Cost optimization through token usage tracking and cost management features.

Who uses it and how

• Organizations that already hold an AWS account with appropriate permissions deploy the chatbot into their own tenancy, keeping data and model traffic inside their account boundary.
• Teams that need answers grounded in private documents point the RAG layer at their own sources and rely on Amazon S3 and Amazon OpenSearch as the storage backends.
• Integrators who do not want the bundled React interface consume the API endpoints instead and embed chat into their own applications.
• Groups evaluating multiple providers can route between Amazon Bedrock, SageMaker-hosted models, custom endpoints, and the GenAIEH Gateway without rebuilding the surrounding stack.
• Operators sensitive to inference spend use the token usage and cost management features to track consumption.

Getting started

Install the CDK CLI with npm install -g aws-cdk@latest and confirm it with cdk --version, using a version compatible with aws-cdk-lib 2.206.0, alongside the stated prerequisites of Node.js 18+, Python 3.8+, and configured AWS credentials. Deployment is fully automated through AWS CDK and SeedFarmer, and complete documentation is linked from the project homepage.

How it compares

The facts provided name no paid products that this project replaces, and they name no directly comparable tools, so it stands alone in this registry. No alternative implementation, hosted service, or commercial equivalent is listed for contrast.

When to use it — and when not to

A self-hoster takes on the operation of Amazon OpenSearch, Amazon S3, Amazon Cognito, AWS Lambda, Amazon API Gateway, and Amazon Bedrock access, plus the Node.js and Python toolchain and the CDK CLI version compatibility the README warns about. Teams without an AWS account, or those wanting a managed service with no infrastructure of their own, should not pick it. The README excerpt is high level and defers full documentation to the repository, GenAIEH Gateway is named without explanation of how it is configured, and the project carries 32 open issues.