Bacalhau is an open-source, Apache-2.0 licensed distributed compute orchestrator written in Go that brings computation to the data rather than moving data to computation, built for data engineers, data scientists, bioinformatics teams and infrastructure operators who run batch, edge and cross-organizational workloads at petabyte scale.

What it is

Bacalhau is a globally distributed compute orchestrator built around the Compute over Data (CoD) model. It is a community-driven framework that ships as a single self-contained binary able to act as a client, an orchestrator and a compute node at the same time. The architecture follows an orchestrator-compute split, in which a dedicated orchestrator coordinates job scheduling while compute nodes execute the tasks, and execution engines and storage providers are wired in through clean interfaces, so the system stays modular rather than monolithic. It is written in Go and released under the Apache-2.0 licence except where noted, and it lives in the infrastructure, orchestration and scheduling ecosystem.

The concrete problem it solves is data movement. Conventional pipelines ship large datasets across the network to a central compute cluster, which adds latency, inflates ingress and egress costs and forces sensitive data outside its security boundary. Bacalhau inverts that pattern: jobs run where the data was created and are parallel by default, so petabytes can be processed without massive transfers. Data scrubbing and security can be applied before any migration takes place, and a granular, code-based permission model governs what a job is allowed to touch. That same design makes cross-organizational computation possible, because vetted computations can run against protected datasets without exposing the raw data.

Key capabilities

• Single binary simplicity: one self-contained binary functions as client, orchestrator and compute node, which keeps setup and scaling simple.
• Multiple execution engines through modular interfaces, including Docker and WebAssembly.
• Flexible storage integrations covering S3, HTTP/HTTPS, IPFS and local storage systems.
• Multiple job types: batch, ops, daemon and service, covering different workflow requirements.
• Declarative and imperative submission, where a job is defined in YAML or given as arguments through the bacalhau CLI.
• Publisher support that writes results to local volumes, S3 or other storage backends.
• Orchestrator-compute model, in which a dedicated orchestrator schedules jobs and compute nodes run them.

Who uses it and how

• Log processing teams run distributed jobs directly at the source instead of centralizing log volumes first.
• Analytics teams build distributed data warehousing that queries data across multiple regions without moving the underlying datasets.
• Operators use it for fleet management across multiple environments, with distributed nodes coordinated from one orchestrator.
• Machine learning groups train and deploy models across a distributed compute fleet.
• Edge and genomics workloads, including gene sequencing and bioinformatics pipelines, run close to the data source where low latency and data sovereignty matter.
• Organizations sharing protected data run specific vetted computations without exposing raw records to the counterparty.

Getting started

Install the CLI on Linux or macOS with curl -sL https://get.bacalhau.org/install.sh | bash and confirm it with bacalhau version; the quick start guide at https://bacalhau.org/docs/getting-started/quick-start/ covers running a first job.

How it compares

This registry entry names no paid products that Bacalhau replaces, and no comparable tools are named in the supplied facts. On the evidence available it stands alone in this registry.

When to use it — and when not to

A self-hoster takes on operating the orchestrator and compute nodes, wiring storage integrations such as S3, HTTP/HTTPS, IPFS or local volumes, and enforcing the code-based permission model; there is no hosted option named in the facts to fall back on. Teams that only need to move modest datasets into an existing managed cluster, or that cannot run a distributed fleet, gain little. The repository shows 27 open issues and the README excerpt is cut off mid-sentence at its commercial terms, so the licence situation outside Apache-2.0 is worth verifying before adoption.

# Install Bacalhau CLI (Linux/macOS)
curl -sL https://get.bacalhau.org/install.sh | bash

# Verify installation
bacalhau version