riot is a free, open source machine learning infrastructure project written in Go and released under Apache-2.0. It has 6,068 GitHub stars, 469 forks and 0 open issues, and was last pushed 12 hours ago. On this registry it ranks #31 of 94 tracked projects in Machine Learning Infrastructure, with 5 head-to-head comparisons available.

What is riot?

Riot is an Apache-2.0 Go library for text and vector indexing and search, aimed at Go developers who need to embed search capabilities in an application instead of adopting a separate search server.

What it is

Riot is an open-source indexing and search library written in Go and published as github.com/vcaesar/riot under the Apache-2.0 licence. It supports Text, Numeric, Date, Boolean, IP, Geo Point and Vector field types, and it provides a query surface that includes Term, Phrase, Match, Match Phrase, Prefix, Regexp, Wildcard and Fuzzy queries, Boolean combinations, numeric, date, term and IP ranges, geo bounding box, geo distance and geo polygon queries, plus ANN and KNN vector queries. The README describes it as a text indexing library in Go and points to zedsearch as the distributed search engine built on riot.

The concrete problem is adding search to a Go application without taking on a full Elasticsearch deployment. Riot provides writers and readers opened from a configuration such as DefaultConfig("./riot_index"), an in-memory option through InMemoryOnlyConfig(), BM25 scoring with pluggable interfaces, and HNSW approximate vector search with int8 quantization. In the Go ecosystem it supplies the embedded indexing layer, while the README names zedsearch, built on riot, as the distributed search engine and lightweight alternative to full Elasticsearch.

Key capabilities

  • Field types cover Text, Numeric, Date, Boolean, IP, Geo Point and Vector.
  • Query types include Term, Phrase, Match, Match Phrase, Prefix, Regexp, Wildcard, Fuzzy, Boolean combinations, numeric, date, term and IP ranges, geo bounding box, geo distance, geo polygon, ANN and KNN.
  • BM25 similarity and scoring are exposed with pluggable interfaces.
  • HNSW approximate vector search uses int8 quantization, and its graphs persist with segments and are memory-mapped from disk.
  • Search results support match highlighting.
  • Extensible aggregations include bucketing by terms, numeric range and date range; metrics such as Min, Max, Count and Sum; Avg and Weighted Avg; cardinality estimation through HyperLogLog++; and quantile approximation through T-Digest.
  • The gse package wraps riot for Chinese and Japanese use, while InMemoryOnlyConfig() and writer.Reader() support in-memory and same-process indexing and search workflows.

Who uses it and how

  • Go applications that need embedded text search can open a writer with DefaultConfig("./riot_index"), update documents, open a reader, and execute a TopNSearch request with a NewMatchQuery.
  • Projects that need vector search in Go can issue ANN and KNN queries and rely on HNSW graphs persisted with segments.
  • Chinese and Japanese search applications can use the gse package, which wraps riot for CJK tokenization.
  • Teams that need bucketing, metrics, cardinality estimates or quantile approximations can use WithStandardAggregations() on a search request.
  • Deployments that need a distributed search engine are directed by the README to zedsearch, which is built on riot and described as a lightweight alternative to full Elasticsearch.

Getting started

Install the library with go get -u github.com/vcaesar/riot. Runnable indexing, querying and CJK programs live in test/readme_demo, and struct indexing and vector search demos live in examples.

How it compares

The README names Elasticsearch as the comparison point: riot is presented as a Go indexing library for embedding search, while zedsearch, built on riot, is described as a lightweight alternative to full Elasticsearch. No other similar tools are named in the README or topic list.

When to use it — and when not to

Use riot when a Go application needs text or vector indexing, BM25 scoring, HNSW vector search, aggregations or CJK analysis without running a separate search server; the library can write to an on-disk index such as ./riot_index or use an in-memory configuration. Teams that need a ready-made distributed search service should not choose riot alone, because the README directs that use case to zedsearch. Self-hosters using the default on-disk configuration must manage the index directory and the HNSW segment files that are persisted and memory-mapped from disk.

project readme (upstream, from github) — read inline

Riot

PkgGoDev Tests Lint codecov GitHub release Join the Discord chat at https://discord.gg/535uZPakJw

English | 简体中文 | 繁體中文 | 日本語 | 한국어 | Français | Deutsch | Español | Русский | Português

A fastest, modern text indexing library in Go.

For a distributed search engine and a lightweight alternative to full Elasticsearch built on riot, see zedsearch.

Features

  • Supported field types:
    • Text, Numeric, Date, Boolean, IP, Geo Point, Vector
  • Supported query types:
    • Term, Phrase, Match, Match Phrase, Prefix, Regexp, Wildcard, Fuzzy
    • Conjunction, Disjunction, Boolean
    • Numeric Range, Date Range, Term Range, IP Range
    • Geo Bounding Box, Geo Distance, Geo Polygon, ANN, KNN
  • BM25 Similarity/Scoring with pluggable interfaces
  • HNSW approximate vector search with int8 quantization; graphs persist with segments and are memory-mapped from disk
  • Search result match highlighting
  • Extensible Aggregations:
    • Bucketing
      • Terms
      • Numeric Range
      • Date Range
    • Metrics
      • Min/Max/Count/Sum
      • Avg/Weighted Avg
      • Cardinality Estimation (HyperLogLog++)
      • Quantile Approximation (T-Digest)

Installation

go get -u github.com/vcaesar/riot

Usage

Runnable versions of the basic indexing, querying, and CJK programs live in test/readme_demo. Struct indexing and vector search demos live in examples; run their commands below from the repository root.

Indexing

Save as write/main.go and run with go run ./write:

package main

import (
	"log"

	riot "github.com/vcaesar/riot"
)

func main() {
	config := riot.DefaultConfig("./riot_index")
	writer, err := riot.OpenWriter(config)
	if err != nil {
		log.Fatalf("error opening writer: %v", err)
	}
	defer writer.Close()

	doc := riot.NewDocument("example").
		AddField(riot.NewTextField("name", "riot"))

	err = writer.Update(doc.ID(), doc)
	if err != nil {
		log.Fatalf("error updating document: %v", err)
	}
	log.Printf("indexed: %s", doc.ID())
}

Querying

Save as read/main.go and run with go run ./read against the index written above. If you index and search in the same process, use writer.Reader() instead of riot.OpenReader (or riot.InMemoryOnlyConfig() for an in-memory index):

package main

import (
	"context"
	"fmt"
	"log"

	riot "github.com/vcaesar/riot"
)

func main() {
	config := riot.DefaultConfig("./riot_index")
	reader, err := riot.OpenReader(config)
	if err != nil {
		log.Fatalf("error opening reader: %v", err)
	}
	defer reader.Close()

	query := riot.NewMatchQuery("riot").SetField("name")
	request := riot.NewTopNSearch(10, query).
		WithStandardAggregations()
	documentMatchIterator, err := reader.Search(context.Background(), request)
	if err != nil {
		log.Fatalf("error executing search: %v", err)
	}

	match, err := documentMatchIterator.Next()
	for err == nil && match != nil {
		err = match.VisitStoredFields(func(field string, value []byte) bool {
			if field == "_id" {
				fmt.Printf("match: %s\n", string(value))
			}
			return true
		})
		if err != nil {
			log.Fatalf("error loading stored fields: %v", err)
		}
		match, err = documentMatchIterator.Next()
	}
	if err != nil {
		log.Fatalf("error iterator document matches: %v", err)
	}
}

Output:

match: example

Chinese / Japanese with gse

The gse package wraps riot with a gse tokenizer for CJK text, plus query-string search and highlighting. Save as cjk/main.go and run with go run ./cjk:

package main

import (
	"fmt"
	"log"
	"os"

	"github.com/vcaesar/riot/gse"
)

func main() {
	opt := gse.Option{
		Index: "test.riot",
		Dicts: "embed, ja", // or "embed, zh"
		Opt:   "search-hmm",
	}
	// Option{Lang: "en"} skips gse and uses a riot analysis/lang analyzer
	// instead ("en", "cjk", "de", ...; see gse.Langs()).
	defer os.RemoveAll(opt.Index)

	index, err := gse.New(opt)
	if err != nil {
		log.Fatalf("error opening gse index: %v", err)
	}
	defer index.Close()

	text := `見解では、謙虚なヴォードヴィリアンのベテランは、運命の犠牲者と悪役の両方の変遷として代償を払っています`
	docs := map[string]string{
		"1": text,
		"3": text + "浮き沈み",
		"4": `In view, a humble vaudevillian veteran cast vicariously as both victim and villain vicissitudes of fate.`,
		"2": `It's difficult to understand the sum of a person's life.`,
		"5": `Riot 是用 Go 语言编写的全文搜索引擎`,
	}
	for id, doc := range docs {
		if err = index.Index(id, doc); err != nil {
			log.Fatalf("error indexing %s: %v", id, err)
		}
	}

	for _, query := range []string{"運命の犠牲者", "搜索引擎", "vaudevillian"} {
		req := gse.QueryString(query, true)
		res, err := index.Search(req)
		if err != nil {
			log.Fatalf("error searching %q: %v", query, err)
		}
		fmt.Printf("query %q: %d hits in %v\n", query, res.Total, res.Took)
		for _, hit := range res.Hits {
			fmt.Printf("  %s (%.3f) %v\n", hit.ID, hit.Score, hit.Fragments["text"])
		}
	}
}

Example output (timings may vary):

query "運命の犠牲者": 2 hits in 14.5µs
  1 (1.899) [見解では、謙虚なヴォードヴィリアンのベテランは、<mark>運命</mark><mark>の</mark><mark>犠</mark><mark>牲</mark><mark>者</mark>と悪役<mark>の</mark>両方<mark>の</mark>変遷として代償を払っています]
  3 (1.837) [見解では、謙虚なヴォードヴィリアンのベテランは、<mark>運命</mark><mark>の</mark><mark>犠</mark><mark>牲</mark><mark>者</mark>と悪役<mark>の</mark>両方<mark>の</mark>変遷として代償を払っています浮き沈み]
query "搜索引擎": 1 hits in 19.792µs
  5 (1.531) [Riot 是用 Go 语言编写的全文<mark>搜索</mark><mark>引擎</mark>]
query "vaudevillian": 1 hits in 1.958µs
  4 (0.654) [In view, humble <mark>vaudevillian</mark> veteran cast vicariously as both victim and villain vicissitudes of fate.]

Indexing structs with gse

package main

import (
	"errors"
	"fmt"
	"log"

	"github.com/vcaesar/riot/gse"
)

type Article struct {
	Title string `json:"title"`
	Text  string `json:"text"`
}

func main() {
	if err := run(); err != nil {
		log.Fatal(err)
	}
}

func run() (err error) {
	index, err := gse.New(gse.Option{Lang: "en"}) // Empty Index uses memory only.
	if err != nil {
		return err
	}
	defer func() { err = errors.Join(err, index.Close()) }()

	if err := index.Index("article-1", Article{
		Title: "Getting started",
		Text:  "Riot is a search engine written in Go.",
	}); err != nil {
		return err
	}

	request := gse.QueryString("started", true)
	request.Field = "title"
	result, err := index.Search(request)
	if err != nil {
		return err
	}
	for _, hit := range result.Hits {
		fmt.Printf("%s: %s %v\n", hit.ID, hit.Fields["title"], hit.Fragments["title"])
	}
	return nil
}

Output:

article-1: Getting started [Getting <mark>started</mark>]

Vector search: exact KNN and approximate ANN

Run examples/knn with go run ./examples/knn:

package main

import (
	"context"
	"errors"
	"fmt"
	"log"

	riot "github.com/vcaesar/riot"
)

func main() {
	if err := run(); err != nil {
		log.Fatal(err)
	}
}

func run() (err error) {
	writer, err := riot.OpenWriter(riot.InMemoryOnlyConfig())
	if err != nil {
		return err
	}
	defer func() { err = errors.Join(err, writer.Close()) }()

	for _, item := range []struct {
		id     string
		vector []float32
	}{
		{"east", []float32{1, 0}},
		{"northeast", []float32{1, 1}},
		{"north", []float32{0, 1}},
	} {
		field, err := riot.NewVectorField("embedding", item.vector)
		if err != nil {
			return err
		}
		doc := riot.NewDocument(item.id).AddField(field)
		if err := writer.Update(doc.ID(), doc); err != nil {
			return err
		}
	}

	reader, err := writer.Reader()
	if err != nil {
		return err
	}
	defer func() { err = errors.Join(err, reader.Close()) }()

	for _, mode := range []string{"KNN", "ANN"} {
		query := riot.NewKNNQuery("embedding", []float32{1, 0}, 2).
			SetMetric(riot.Cosine)
		if mode == "ANN" {
			query.SetANN(riot.ANNParams{EfSearch: 100})
		}
		matches, err := reader.Search(context.Background(), riot.NewTopNSearch(2, query))
		if err != nil {
			return err
		}
		for {
			match, err := matches.Next()
			if err != nil {
				return err
			}
			if match == nil {
				break
			}
			var id string
			if err := match.VisitStoredFields(func(name string, value []byte) bool {
				if name == "_id" {
					id = string(value)
				}
				return true
			}); err != nil {
				return err
			}
			fmt.Printf("%s: %s (%.3f)\n", mode, id, match.Score)
		}
	}
	return nil
}

Output:

KNN: east (1.000)
KNN: northeast (0.707)
ANN: east (1.000)
ANN: northeast (0.707)

Persisted vector indexes

HNSW graphs are saved with on-disk segments, so a reopened index searches them without rebuilding. WithVectorStorage chooses where their vectors live: riot.VectorStorageDisk (the default) memory-maps each graph with its vectors, so they sit in the OS page cache instead of the Go heap; riot.VectorStorageMemory stores links only and loads vectors onto the heap when a graph opens (smaller files, more memory).

config := riot.DefaultConfig("./riot_vectors").WithVectorStorage(riot.VectorStorageDisk)

examples/ann_disk indexes vectors on disk and searches them after reopening: go run ./examples/ann_disk.

Benchmarks

Riot vs Bleve and Bluge (the engine it forked)

All three engines run the same workload: 100,000 synthetic documents (body text with positions, stored; numeric id; category keyword). Bleve v2.6.2 (scorch, BM25) and Bluge v0.2.2 run against current riot (Go 1.27.3) in its default stream indexing mode, one process at a time on one core (GOMAXPROCS=1), median of 5 rotated rounds (Apple M1, load average 2.7 to 6.1). In every run all three returned the same match counts and match sets as a per-document oracle.

Workload Bleve v2.6.2 Bluge v0.2.2 Riot Riot vs Bleve Riot vs Bluge
Initial indexing, 100,000 docs 6,546 ms 5,029 ms 500 ms 13.1× faster 10.1× faster
1-doc commit + visible reader 0.151 ms 0.313 ms 0.101 ms§ 1.5× faster 3.1× faster
Index size on disk 76.71 MB 61.06 MB 5.76 MB 13.3× smaller 10.6× smaller
Peak memory (RSS) 1,931 MiB 1,974 MiB 140 MiB 13.8× less 14.1× less
Total CPU, whole run 17.84 s 19.50 s 1.05 s 17.0× less 18.6× less
Rare term 11.90 µs 16.98 µs 2.19 µs 5.4× faster 7.8× faster
Common term, top 10 + count 5,848 µs 10,861 µs 7.17 µs† 816× faster 1,516× faster
Boolean AND 1,788 µs 1,990 µs 23.20 µs† 77× faster 86× faster
Boolean OR 27.18 µs 43.61 µs 4.46 µs 6.1× faster 9.8× faster
Phrase 60,645 µs 61,758 µs 2,079 µs 29× faster 30× faster
Prefix term12* (constant score in riot) 3,133 µs 3,597 µs 35.13 µs 89× faster 102× faster
Fuzzy serch, distance 1 6,798 µs 12,035 µs 23.70 µs† 287× faster 508× faster
Typed numeric range 166.2 µs 164.6 µs 7.84 µs 21× faster 21× faster
Keyword / facet term 385.1 µs 645.7 µs 2.69 µs 143× faster 240× faster
Numeric stats‡ 45,375 µs 105,356 µs 433.2 µs 105× faster 243× faster
  • Peak memory and CPU cover the whole benchmark process (10K and 100K builds plus every query loop), taken from wait4 rusage. Index size comes from a separate on-disk build (median of 3).

Scores differ from Bluge's because riot fixed Bluge's BM25 idf to Lucene's log(1 + (N − n + 0.5)/(n + 0.5)). Bluge has no vector search; Bleve's needs the vectors build tag and FAISS through cgo. These are single-machine numbers on synthetic data, not a formal benchmark.

Riot (Go) is already on par with Tantivy: it is faster on 8 of 10 query workloads in the same run. An even faster Zig version of riot is also in development. It is still beta and not published yet. In the same benchmark run it was about 2–5× faster than riot (Go) on most queries and 2.8× faster at indexing, and it was at least 1.5× faster than Tantivy on every query workload (2–8× on most).

License

Apache License Version 2.0

Frequently asked questions

Is riot free to use?

riot is open source under the Apache-2.0 licence. There is no licence fee and no seat count — you can self-host it or, where the project offers one, pay a vendor for a managed version instead.

What does riot do?

Go Open Source, Distributed, Simple and fastest efficient text/vector Search Engine

What is riot written in?

riot is primarily written in Go. Its source is publicly available at https://github.com/vcaesar/riot, and it has 6,068 GitHub stars.