versitygw: Open Source Data Warehousing & Processing

What is versitygw?

VersityGW (the Versity S3 Gateway) is an Apache-2.0 licensed, Go-based translation service that presents existing storage — a generic POSIX filesystem, Versity's open source ScoutFS, Azure Blob Storage, or another S3 server — through a standard S3 API, and it is aimed at teams that already run storage infrastructure and want S3-compatible access to it without migrating data into a separate object store.

What it is

VersityGW is a stateless server that sits inline between S3-reliant applications and a backend storage system. It translates incoming AWS S3 API requests and transforms them into equivalent operations against the backend service, so applications can speak the S3-compatible API on top of storage that was not built to speak it. It lives in the S3-compatible ecosystem: the same protocol AWS clients, SDKs, and tools already use, applied to POSIX filesystems and other backends. It is written in Go, carries the Apache-2.0 licence, and lists binary release builds for Linux, macOS, BSD, and Windows on both amd64 and arm64. The project is modular by design, with a simplified interface for adding new storage system support, and it ships an optional WebGUI for management and exploration.

The concrete problem it solves is the gap between applications that assume S3 and data that already lives on a filesystem or another storage system. Rather than standing up a separate S3 object store and copying data into it, an operator points the gateway at the storage they already own, and the gateway substitutes for the dedicated S3 server that would otherwise have to be deployed in front of that data. Because it is completely stateless, several instances can be clustered to raise aggregate throughput, and because posix mode offers protocol compatibility, the same files can be reached through either POSIX or S3. What it replaces is the storage migration step: existing infrastructure investments stay in place while S3 compatibility is layered on top.

Key capabilities

Inline translation of AWS S3 object commands into equivalent backend operations, with a stateless design so multiple gateway instances can be deployed in a cluster to increase aggregate throughput.
Backend support for generic POSIX file storage, Versity's open source ScoutFS filesystem, Azure Blob Storage, and other S3 servers, including a proxy use case that forwards S3 requests to S3 storage.
Protocol compatibility in posix, which allows common access to the same files through either POSIX or S3.
S3 over RDMA, bypassing the kernel network stack for high-throughput, low-latency transfers, with vgwrdma exposing a standard S3 API accelerated by NVIDIA's cuObject (GPUDirect Storage for Objects) protocol.
Static website hosting with index documents, custom error pages, and routing rules, enabled through flags such as --website :8090 --website-domain example.com, supporting virtual-host style routing and a catch-all mode when --website-domain is omitted.
An optional WebGUI management and explorer interface, documented on the project wiki.
A simplified interface for adding new storage system support, keeping the backend layer extensible.

Who uses it and how

HPC and data-intensive workloads where network overhead is a bottleneck, using S3 over RDMA or the vgwrdma service.
Teams with data already on a POSIX filesystem or ScoutFS who need S3 access without copying it into a new object store, using posix mode so both access paths reach the same files.
Operators publishing buckets as static websites, configuring index documents, custom error pages, and virtual-host routing for bucket-per-hostname layouts.
Deployments that run several stateless gateway instances in a cluster to raise aggregate throughput, or that proxy S3 requests through the gateway to Azure Blob Storage or an upstream S3 server.
Organisations that want commercial enterprise support, arranged by contacting Versity Sales, with community questions handled through the project's GitHub discussions and the versitygw mailing list.

Getting started

The README's installation path is the binary release builds: download the latest release from the GitHub releases page, which provides builds for Linux, macOS, BSD, and Windows on amd64 and arm64. Once the binary is in place, the project's headline workflow is turning a local filesystem into an S3 server with a single command.

How it compares

The facts provided here do not name any comparable or competing gateways, and no list of paid products that this project replaces is supplied, so VersityGW stands alone in this registry. It should be read as a translation layer placed between S3 clients and the storage backends it supports — generic POSIX storage, ScoutFS, Azure Blob Storage, and other S3 servers — rather than as an alternative to them.

When to use it — and when not to

A self-hoster must already operate the backend storage the gateway fronts, plus whatever infrastructure is needed to reach it, because the gateway itself is stateless and holds no data of its own. It is a poor fit for anyone who wants a hosted S3 service with no storage to run, and the open-source offering does not include enterprise support, which the README routes through Versity Sales. Two honest signals for prospective users: the README is largely a hub of links, with operational detail such as S3 RDMA and WebGUI setup living on the wiki, and the repository carries 154 open issues, so some rough edges should be expected.

project readme (upstream, from github) — read inline

The Versity S3 Gateway:
A High-Performance S3 Translation Service

Binary release builds

Download latest release

Linux amd64/arm64	MacOS amd64/arm64	BSD amd64/arm64	Windows amd64/arm64
✔️	✔️	✔️	✔️

Use Cases

Turn your local filesystem into an S3 server with a single command!
Proxy S3 requests to S3 storage
Simple to deploy S3 server with a single command
Protocol compatibility in posix allows common access to files via posix or S3
Simplified interface for adding new storage system support

WebGUI

Get more details about the new (optional) WebGUI management/explorer here: https://github.com/versity/versitygw/wiki/WebGUI

admin-explorer

S3 RDMA

VersityGW supports S3 over RDMA (Remote Direct Memory Access), enabling high-throughput, low-latency object transfers that bypass the kernel network stack. This is particularly useful for HPC and data-intensive workloads where network overhead is a bottleneck. See the S3 RDMA wiki page for setup and usage details.

vgwrdma is a VersityGW-based service that exposes a standard S3 API accelerated with NVIDIA's cuObject (GPUDirect Storage for Objects) protocol.

Static Website Hosting

Serve S3 buckets as static websites with index documents, custom error pages, and routing rules. Enable a separate website endpoint with --website :8090 --website-domain example.com for virtual-host style routing (blog.example.com serves bucket blog, example.com serves bucket example.com). When --website-domain is omitted, catch-all mode is used: the full hostname becomes the bucket name (name your buckets as FQDNs, e.g. blog.example.com). See Global Options for all --website-* flags.

News

Check out latest wiki articles: https://github.com/versity/versitygw/wiki/Articles

Mailing List

Keep up to date with latest gateway announcements by signing up to the versitygw mailing list.

Documentation

See project documentation on the wiki.

Need help?

Ask questions in the community discussions.
Contact Versity Sales to discuss enterprise support.

Overview

Versity Gateway, a simple to use tool for seamless inline translation between AWS S3 object commands and storage systems. The Versity Gateway bridges the gap between S3-reliant applications and other storage systems, enabling enhanced compatibility and integration while offering exceptional scalability.

The server translates incoming S3 API requests and transforms them into equivalent operations to the backend service. By leveraging this gateway server, applications can interact with the S3-compatible API on top of already existing storage systems. This project enables leveraging existing infrastructure investments while seamlessly integrating with S3-compatible systems, offering increased flexibility and compatibility in managing data storage.

The Versity Gateway is focused on performance, simplicity, and expandability. The Versity Gateway is designed with modularity in mind, enabling future extensions to support additional backend storage systems. At present, the Versity Gateway supports any generic POSIX file backend storage, Versity’s open source ScoutFS filesystem, Azure Blob Storage, and other S3 servers.

The gateway is completely stateless. Multiple Versity Gateway instances may be deployed in a cluster to increase aggregate throughput. The Versity Gateway’s stateless architecture allows any request to be serviced by any gateway thereby distributing workloads and enhancing performance. Load balancers may be used to evenly distribute requests across the cluster of gateways for optimal performance.

The S3 HTTP(S) server and routing is implemented using the Fiber web framework. This framework is actively developed with a focus on performance. S3 API compatibility leverages the official aws-sdk-go-v2 whenever possible for maximum service compatibility with AWS S3.

Getting Started

See the Quickstart documentation.

Run the gateway with posix backend:

mkdir /tmp/vgw /tmp/vers
ROOT_ACCESS_KEY="testuser" ROOT_SECRET_KEY="secret" ./versitygw --port :10000 --iam-dir /tmp/vgw posix --versioning-dir /tmp/vers /tmp/vgw

This will enable an S3 server on the current host listening on port 10000 and hosting the directory /tmp/vgw with older object versions in /tmp/vers. It's fine if both of these directories are within the same filesystem. The --iam-dir option enables simple JSON flat file accounts for testing.

To get the usage output, run the following:

./versitygw --help

The command format is

versitygw [global options] command [command options] [arguments...]

The global options are specified before the backend type and the backend options are specified after.

Testing & Production Readiness

VersityGW is battle-tested and production-ready. Every pull request must pass our comprehensive test suite before it can be reviewed or merged. All code reviews are done by at least one human in the loop. LLMs may be used to augment the review process, but are never the sole reviewer or decision maker. See Testing for high level testing documentation.

Comprehensive Test Coverage

Our multi-layered testing strategy includes:

Go Unit Test Files - Extensive unit tests with race detection and code coverage analysis covering core functionality, edge cases, and error handling.
Integration Test Scripts - Real-world scenario testing across multiple backends (POSIX, S3, Azure) and configurations.
Functional/Regression Tests - End-to-end SDK tests validating complete workflows including full-flow operations, POSIX-specific behavior, and IAM functionality populated with regression tests as issues are addressed.
Static Analysis - Static Analysis checks using staticcheck.
System Tests - Protocol-level validation using industry-standard S3 clients:
- AWS CLI - Official AWS command-line tools
- s3cmd - Popular S3 client
- Direct REST API testing with curl for request/response validation
Security Testing - Both HTTP and HTTPS configurations tested. Vulnerability scanning with govulncheck. And regular dependency updates with dependabot.
Compatibility Testing - Multiple backends, versioning scenarios, static bucket modes, and various authentication methods.

Run the gateway in Docker

Use the published image like the native binary by passing CLI arguments:

docker run --rm versity/versitygw:latest --version

See Docker for more documentation for running within Docker.

Run on Kubernetes

A Helm chart is provided to easily run Versity in Kubernetes environments:

helm install versitygw oci://ghcr.io/versity/versitygw/charts/versitygw

Please refer to the chart's README for more information and configuration parameters.

Versity gives you clarity and control over your archival storage, so you can allocate more resources to your core mission.

Contact

versity logo [email protected]
+1 844 726 8826

@versitysoftware

Frequently asked questions

Is versitygw free to use?

versitygw is open source under the Apache-2.0 licence. There is no licence fee and no seat count — you can self-host it or, where the project offers one, pay a vendor for a managed version instead.

What does versitygw do?

A simple to deploy but feature rich S3 object storage server for your filesystem

What is versitygw written in?

versitygw is primarily written in Go. Its source is publicly available at https://github.com/versity/versitygw, and it has 2,933 GitHub stars.