[opensourcedex]

orioledb is a free, open source databases project written in C and released under Apache-2.0. It has 4,195 GitHub stars, 187 forks and 131 open issues, and was last pushed 2 hours ago. On this registry it ranks #163 of 203 tracked projects in Databases, with 5 head-to-head comparisons available.

What is orioledb?

OrioleDB is an open-source storage engine for PostgreSQL, delivered as an extension built on the table access method framework and standard Postgres extension interfaces, intended for database operators, platform engineers, and PostgreSQL contributors who need better vertical scalability and lower maintenance burden than the default heap engine offers.

What it is

OrioleDB is a new storage engine for PostgreSQL, written in C and packaged as a PostgreSQL extension named orioledb. It builds on the table access method framework and other standard extension interfaces, and its stated goal is to open the door to more powerful storage models tuned for cloud and modern hardware architectures. It is named after the golden oriole, pronounced OR-ee-ohl-DEE-BEE (IPA /ˈɔːr.i.oʊl diː biː/). The project lives in the PostgreSQL ecosystem and is dual-licensed under the Apache License 2.0 and the PostgreSQL License, with the SPDX identifier Apache-2.0 OR PostgreSQL.

The concrete problem it attacks is the default heap storage engine under cloud and many-core conditions. OrioleDB replaces the standard table access method that PostgreSQL ships with, and it also removes the operational need for dedicated garbage collection: the undo log and page-mergins mechanisms mean dedicated VACUUMing of tables is not needed. It eliminates the transaction ID wraparound problem by implementing default 64-bit transaction identifiers, and it targets the legacy CPU bottlenecks that appear on servers with dozens or hundreds of cores. Its row-level write-ahead log with parallel apply is designed for raft consensus replication and the active-active multimaster model.

Key capabilities

  • Runs as an additional table access method inside PostgreSQL rather than as a fork or a separate database server.
  • MVCC built on the undo log: old tuple versions are evicted into undo chains instead of bloating main storage, and page-level undo records allow space from deleted tuples to be reclaimed quickly.
  • Page-mergins combined with the undo log eliminate bloat in most cases, so dedicated table VACUUMing is not required.
  • Default 64-bit transaction identifiers remove the wraparound problem.
  • Copy-on-write checkpoints produce a structurally consistent snapshot of the data at every moment, which suits modern SSDs.
  • Row-level WAL logging with parallel apply is compact, already implemented, and optimized for raft consensus replication.
  • No buffer mapping and lock-less page reading: in-memory pages are linked directly to storage pages, and in-memory page reads involve no atomic operations.

Who uses it and how

  • Operators running cloud-native PostgreSQL on servers with dozens to hundreds of CPU cores, where the default engine's bottlenecks bite hardest.
  • Deployments on modern storage such as SSD and NVRAM that want the engine's storage-aware design rather than legacy paths.
  • Teams that have suffered system performance deterioration or database outages caused by VACUUM and table bloat, and want the garbage collection concern removed.
  • Groups planning raft consensus replication and eventual active-active multimaster, who need the row-level WAL with parallel apply.
  • Contributors building the extension locally against PostgreSQL, following the development quickstart at doc/contributing/local-builds.mdx and the project structure notes at doc/contributing/structure.mdx.

Getting started

The repository publishes Docker images to Docker Hub under orioledb/orioledb, and the source build path is documented in doc/contributing/local-builds.mdx. Usage afterwards follows doc/usage/getting-started.mdx, with architecture detail in doc/architecture/overview.mdx and introduction in doc/intro.mdx.

How it compares

No comparable or competing products are named in the facts available here, and no list of paid products that this project replaces is provided. On the evidence given, OrioleDB stands alone in this registry.

When to use it — and when not to

A self-hoster must still operate PostgreSQL itself, including storage, checkpoints, backups, and extension upgrades, and must build or pull the extension rather than consume a managed hosted option. The active-active multimaster capability is documented as planned rather than done, so replication-dependent designs should not assume it today. The project carries 131 open issues and a README that points at external documentation files rather than explaining everything inline, so teams wanting a fully managed, zero-operations database should look elsewhere.

project readme (upstream, from github) — read inline

OrioleDB – a cloud-native storage engine for PostgreSQL

(A solution to PostgreSQL’s wicked problems)

check status Coverage Status dockerhub

OrioleDB is pronounced as OR-ee-ohl-DEE-BEE (IPA /ˈɔːr.i.oʊl diː biː/) and is named after the golden oriole, a bird which inhabits a range of habitats and migrates in spring symbolizing renewal.

OrioleDB is a new storage engine for PostgreSQL, bringing a modern approach to database capacity, capabilities and performance to the world's most-loved database platform.

OrioleDB consists of an extension, building on the innovative table access method framework and other standard Postgres extension interfaces. By extending and enhancing the current table access methods, OrioleDB opens the door to a future of more powerful storage models that are optimized for cloud and modern hardware architectures.

  1. Designed for modern hardware. OrioleDB design avoids legacy CPU bottlenecks on modern servers containing dozens and hundreds CPU cores, providing optimized usage of modern storage technologies such as SSD and NVRAM.

  2. Reduced maintenance needs. OrioleDB implements the concepts of undo log and page-mergins, eliminating the need for dedicated garbage collection processes. Additionally, OrioleDB implements default 64-bit transaction identifiers, thus eliminating the well-known and painful wraparound problem.

  3. Designed to be distributed. OrioleDB implements a row-level write-ahead log with support for parallel apply. This log architecture is optimized for raft consensus-based replication allowing the implementation of active-active multimaster.

The key technical differentiations of OrioleDB are as follows:

  1. No buffer mapping and lock-less page reading. In-memory pages in OrioleDB are connected with direct links to the storage pages. This eliminates the need for in-buffer mapping along with its related bottlenecks. Additionally, in OrioleDB in-memory page reading doesn't involve atomic operations. Together, these design decisions bring vertical scalability for Postgres to the whole new level.

  2. MVCC is based on the UNDO log concept. In OrioleDB, old versions of tuples do not cause bloat in the main storage system, but eviction into the undo log comprising undo chains. Page-level undo records allow the system to easily reclaim space occupied by deleted tuples as soon as possible. Together with page-mergins, these mechanisms eliminate bloat in the majority of cases. Dedicated VACUUMing of tables is not needed as well, removing a significant and common cause of system performance deterioration and database outages.

  3. Copy-on-write checkpoints and row-level WAL. OrioleDB utilizes copy-on-write checkpoints, which provides a structurally consistent snapshot of data every moment of time. This is friendly for modern SSDs and allows row-level WAL logging. In turn, row-level WAL logging is easy to parallelize (done), compact and suitable for active-active multimaster (planned).

See introduction, getting started, and architecture documentation as well as PostgresBuild 2021 slides. To start the development see OrioleDB development quickstart, and project structure.

License

OrioleDB is dual-licensed under the Apache License 2.0 and the PostgreSQL License.

You may choose either license to govern your use of this work.

  1. Apache License 2.0
  2. PostgreSQL License

SPDX-License-Identifier: Apache-2.0 OR PostgreSQL

All contributions to OrioleDB are made under both the Apache License 2.0 and the PostgreSQL License.

See LICENSE-APACHE.txt and LICENSE-POSTGRESQL.txt for details.

Patent Grant:. Supabase provides a separate patent grant for OrioleDB. See PATENTS.txt for details.

Status

OrioleDB now has public beta status. It is recommended for experiments, testing, benchmarking, etc., but is not recommended for production usage. If you are interested in OrioleDB's benefits in production, please contact us.

Installation

Use docker container

We provide docker images for amd64 and arm64v8 architectures under Alpine Linux.

docker pull orioledb/orioledb:latest-pg17

For example it can be started same as postgres server:

# !Don't forget to set default locale to C, POSIX or use icu-locale
docker run --name some-postgres -e POSTGRES_PASSWORD=... -e POSTGRES_INITDB_ARGS="--locale=C" -d -p5432:5432 orioledb/orioledb:latest-pg17

See our dockerhub for details on our docker container usage. See the docker build guide for information on how to build the docker images locally.

Build from source

Before building and installing OrioleDB, one should ensure to have the following:

  • PostgreSQL with extensibility patches checked out at the exact commit/tag listed for your major version in .pgtags (16: … and 17: …). The build verifies this and refuses to compile against any other patchset.
  • Development package of libzstd;
  • python 3.5+ with testgres package.

Typical installation procedure may look like this:

 # 1. Clone this repo first -- .pgtags pins the exact patched-PostgreSQL
 # commit you must build against.
 $ git clone https://github.com/orioledb/orioledb
 $ cd orioledb
 $ cat .pgtags
 17: 7050019fd74d0d841040edc49ee1e16364a8b589
 16: e5ef9e6a6d21f8e69e1d4bddcd4cdb83f054ca0b

 # 2. Build patched PostgreSQL at the commit listed for your major
 # version (substitute the value from .pgtags, do not copy-paste this
 # one -- it moves).
 $ cd ..
 $ git clone https://github.com/orioledb/postgres orioledb-postgres
 $ cd orioledb-postgres
 $ git checkout $(awk -F': ' '/^17: /{print $2}' ../orioledb/.pgtags)
 $ ./configure --prefix=/path/to/pg --enable-debug --enable-cassert --with-icu
 $ make -j install
 $ make -C contrib -j install
 $ export PATH=/path/to/pg/bin:$PATH

 # 3. Build the extension.
 $ cd ../orioledb
 # IS_DEV=1 is required for `make installcheck` to actually run the tests
 $ make USE_PGXS=1 IS_DEV=1
 $ make USE_PGXS=1 IS_DEV=1 install
 $ make USE_PGXS=1 IS_DEV=1 installcheck

If the configured PostgreSQL is at the wrong commit, the build prints

Wrong orioledb patchset version: expected <hash>, got <hash>
Rebuild and install patched orioledb/postgres using commit '<hash>'

— in that case re-check out orioledb-postgres at the commit from .pgtags and rebuild it.

Before starting working with OrioleDB, adding the following line to postgresql.conf is required. This change requires a restart of the PostgreSQL database server.

shared_preload_libraries = 'orioledb.so'

Collations

OrioleDB tables support ICU, C, POSIX, and PostgreSQL 17 or later builtin collations.

So that you don't have to write COLLATE for every "text" field of tables you have options:

Create whole cluster with one of these collations:

initdb --locale=C -D..
# OR
initdb --locale=POSIX -D..
# OR
initdb --locale-provider=icu --icu-locale=en -D...
# OR
initdb --locale-provider=builtin --builtin-locale=C.UTF-8 -D...

Create new database with default collation from template0

createdb --locale=C --template template0 ...
# OR
createdb --locale=POSIX --template template0 ...
# OR
createdb --locale-provider=icu --icu-locale=en --template template0 ...
# OR
createdb --locale-provider=builtin --builtin-locale=C.UTF-8 --template template0 ...

Or use CREATE DATABASE with the LOCALE, ICU_LOCALE, or BUILTIN_LOCALE parameters.

Setup

Run the following SQL query on the database to enable the OrioleDB engine.

CREATE EXTENSION orioledb;

Once the above steps are complete, you can start using OrioleDB's tables. See getting started documentation for details.

CREATE TABLE table_name (...) USING orioledb;

Frequently asked questions

Is orioledb free to use?

orioledb is open source under the Apache-2.0 licence. There is no licence fee and no seat count — you can self-host it or, where the project offers one, pay a vendor for a managed version instead.

What does orioledb do?

OrioleDB – building a modern cloud-native storage engine (... and solving some PostgreSQL wicked problems)

What is orioledb written in?

orioledb is primarily written in C. Its source is publicly available at https://github.com/orioledb/orioledb, and it has 4,195 GitHub stars.