SeaGOAT: Open Source Machine Learning Infrastructure

What is SeaGOAT?

SeaGOAT is a local-first semantic code search engine that lets developers query a codebase in natural language, built for developers and teams who want AI-style code search without sending source code to a remote service.

What it is

SeaGOAT is a Python code search engine, licensed under MIT, that combines vector embeddings with conventional regex search to find code from plain-language queries. It runs as a local server started against a repository, uses ChromaDB as a local vector database with a local vector embedding engine and telemetry disabled by default, and pairs that with ripgrep for regular-expression and keyword matches. Results are queried through the gt or seagoat command. The project lives in the AI and machine learning infrastructure ecosystem, with topics including embeddings, code-search-engine, llm, grep, grep-like, ripgrep, and regular-expression.

The concrete problem it solves is the gap between two imperfect options. A traditional registry of code search tools — grep and ripgrep — requires the searcher to already know exact identifiers or write a regex that predicts the code. Hosted semantic search sends a private codebase to someone else's service. SeaGOAT instead answers queries like gt "Where are the numbers rounded" against a repository on the developer's own machine, while still accepting regex-style queries such as gt "function calc_.* that deals with taxes".

Key capabilities

Installs as the seagoat Python package via pipx install seagoat, with Python 3.11 or newer and ripgrep as hard dependencies and bat as an optional, recommended dependency.
Runs a per-repository server with seagoat-server start /path/to/your/repo, queried through the gt or seagoat commands, and stopped with seagoat-server stop /path/to/your/repo.
Answers natural-language queries and regex queries through the same interface, for example gt "Where are the numbers rounded" and gt "function calc_.* that deals with taxes".
Uses ChromaDB as a local vector database with a local vector embedding engine and telemetry disabled by default, with no third-party or remote APIs required.
Uses ripgrep alongside the embedding search so regex and keyword matches appear in addition to AI-based matches.
Renders results with bat when installed and color is enabled, falls back to pygments highlighting when bat is absent, and emits a grep-line output format when used as part of a pipeline.
Configures through YAML, globally or per project in a .seagoat.yml file, including settings such as server: port: 31134.

Who uses it and how

Individual developers searching a single repository locally, on hardware the README describes as working on any decent laptop.
Linux users primarily, since Linux is tested, with macOS partly tested and Windows explicitly marked as needing help.
Teams with code that cannot leave the machine, because SeaGOAT does not rely on third-party or remote APIs and executes functionality locally through its own server.
Developers working inside shell pipelines, where SeaGOAT switches to a grep-line output format instead of colored output.
Contributors, given the hacktoberfest and hacktoberfest2023 topics and a Poetry-based development setup using poetry run ptw and poetry run pytest . --testmon.

Getting started

Install the seagoat package with pipx install seagoat after installing Python 3.11 or newer and ripgrep, then start the server against a repository with seagoat-server start /path/to/your/repo and query it with gt. Project-specific behavior is set in a .seagoat.yml file, and documentation is hosted at the project homepage.

How it compares

Among similar tools named in the facts, SeaGOAT is closest to ripgrep and grep, but where those match literal patterns the user must already know, SeaGOAT layers embeddings and a language model oriented query over them. It also sits alongside zeitgrep, another search tool by the same author that is referenced in the README. The distinguishing property is that all matching happens locally, with no remote API calls.

When to use it — and when not to

A self-hoster takes on running a Python 3.11+ environment, installing ripgrep and optionally bat, and starting and stopping a local server per repository, which is more operational overhead than running grep. Windows users should not pick it yet, since support is marked as needing help, and macOS is only partly tested. The FAQ states that its points are indications of how SeaGOAT works rather than a legal contract, and that optional features sending data to remote servers might be added in the future, so anyone with strict data-handling requirements should verify against the source, which the README invites.

project readme (upstream, from github) — read inline

[!TIP] Check out zeitgrep, another search tool I am working on!

SeaGOAT

A code search engine for the AI age. SeaGOAT is a local search tool that leverages vector embeddings to enable you to search your codebase semantically.

Getting started

Install SeaGOAT

In order to install SeaGOAT, you need to have the following dependencies already installed on your computer:

Python 3.11 or newer
ripgrep
bat (optional, highly recommended)

When bat is installed, it is used to display results as long as color is enabled. When SeaGOAT is used as part of a pipeline, a grep-line output format is used. When color is enabled, but bat is not installed, SeaGOAT will highlight the output using pygments. Using bat is recommended.

To install SeaGOAT using pipx, use the following command:

pipx install seagoat

System requirements

Hardware

Should work on any decent laptop.

Operating system

SeaGOAT is designed to work on Linux (tested ✅), macOS (partly tested, help 🙏) and Windows (help needed 🙏).

Start SeaGOAT server

In order to use SeaGOAT in your project, you have to start the SeaGOAT server using the following command:

seagoat-server start /path/to/your/repo

Search your repository

If you have the server running, you can simply use the gt or seagoat command to query your repository. For example:

gt "Where are the numbers rounded"

You can also use Regular Expressions in your queries, for example

gt "function calc_.* that deals with taxes"

Stopping the server

You can stop the running server using the following command:

seagoat-server stop /path/to/your/repo

Configuring SeaGOAT

SeaGOAT can be tailored to your needs through YAML configuration files, either globally or project-specifically with a .seagoat.yml file. For instance:

# .seagoat.yml

server:
  port: 31134  # Specify server port

Check out the documentation for more details!

Development

Requirements:

Poetry
Python 3.11 or newer
ripgrep

Install dependencies

After cloning the repository, install dependencies using the following command:

poetry install

Running tests

Watch mode (recommended)

poetry run ptw

Test changed files

poetry run pytest .  --testmon

Test all files

poetry run pytest .

Manual testing

You can test any SeaGOAT command manually in your local development environment. For example to test the development version of the seagoat-server command, you can run:

poetry run seagoat-server start ~/path/an/example/repository

FAQ

The points in this FAQ are indications of how SeaGOAT works, but are not a legal contract. SeaGOAT is licensed under an open source license and if you are in doubt about the privacy/safety/etc implications of SeaGOAT, you are welcome to examine the source code, raise your concerns, or create a pull request to fix a problem.

How does SeaGOAT work? Does it send my data to ChatGPT?

SeaGOAT does not rely on 3rd party APIs or any remote APIs and executes all functionality locally using the SeaGOAT server that you are able to run on your own machine.

Instead of relying on APIs or "connecting to ChatGPT", it uses the vector database called ChromaDB, with a local vector embedding engine and telemetry disabled by default.

Apart from that, SeaGOAT also uses ripgrep, a regular-expression based code search engine in order to provider regular expression/keyword based matches in addition to the "AI-based" matches.

While the current version of SeaGOAT does not send your data to remote servers, it might be possible that in the future there will be optional features that do so, if any further improvement can be gained from that.

Why does SeaGOAT need a server?

SeaGOAT needs a server in order to provide a speedy response. SeaGOAT heavily relies on vector embeddings and vector databases, which at the moment cannot be replace with an architecture that processes files on the fly.

It's worth noting that you are able to run SeaGOAT server entirely locally, and it works even if you don't have an internet connection. This use case does not require you to share data with a remote server, you are able to use your own SeaGOAT server locally, albeit it's also possible to run a SeaGOAT server and allow other computers to connect to it, if you so wish.

Does SeaGOAT create AI-derived work? Is SeaGOAT ethical?

If you are concerned about the ethical implications of using AI tools keep in mind that SeaGOAT is not a code generator but a code search engine, therefore it does not create AI derived work.

That being said, a language model is being used to generate vector embeddings. At the moment SeaGOAT uses ChromaDB's default model for calculating vector embeddings, and I am not aware of this being an ethical concern.

What programming languages are supported?

Currently SeaGOAT is hard coded to only process files in the following formats:

Text Files (*.txt)
Markdown (*.md)
Python (*.py)
C (*.c, *.h)
C++ (*.cpp, *.cc, *.cxx, *.hpp)
TypeScript (*.ts, *.tsx)
JavaScript (*.js, *.jsx)
HTML (*.html)
Go (*.go)
Java (*.java)
PHP (*.php)
Ruby (*.rb)

Why is SeaGOAT processing files so slowly while barely using my CPU?

Since processing files for large repositories can take a long time, SeaGOAT is designed to allow you to use your computer while processing files. It is an intentional design choice to avoid blocking/slowing down your computer.

This design decision does not affect the performance of queries.

By the way, you are able to use SeaGOAT to query your repository while it's processing your files! When you make a query, and the files are not processed yet, you will receive a warning with an estimation of the accuracy of your results. Also, regular expression/full text search based results will be displayed from the very beginning!

What character encodings are supported?

The preferred character encoding is UTF-8. Most other character encodings should also work. Only text files are supported, SeaGOAT ignores binary files.

Where does SeaGOAT store it's database/cache?

Where SeaGOAT stores databases and cache depends on your operating system. For your convenience, you can use the seagoat-server server-info command to find out where these files are stored on your system.

Can I host SeaGOAT server on a different computer?

Yes, if you would like to use SeaGOAT without having to run the server on the same computer, you can simply self-host SeaGOAT server on a different computer or in the cloud, and configure the seagoat/gt command to connect to this remote server through the internet.

Keep in mind that SeaGOAT itself does not enforce any security as it is primarily designed to run locally. If you have private code that you do not wish to leak, you will have to make sure that only trusted people have access to the SeaGOAT server. This could be done by making it only available through a VPN that only your teammates can access.

Can I ignore files/directories?

SeaGOAT already ignores all files/directories ignored in your .gitignore. If you wish to ignore additional files but keep them in git, you can use the ignorePatterns attribute from the server configuration. Learn more

Frequently asked questions

Is SeaGOAT free to use?

SeaGOAT is open source under the MIT licence. There is no licence fee and no seat count — you can self-host it or, where the project offers one, pay a vendor for a managed version instead.

What does SeaGOAT do?

local-first semantic code search engine

What is SeaGOAT written in?

SeaGOAT is primarily written in Python. Its source is publicly available at https://github.com/kantord/SeaGOAT, and it has 1,316 GitHub stars.