[opensourcedex]

vulcain is a free, open source networking & connectivity project written in Go and released under AGPL-3.0. It has 3,595 GitHub stars, 103 forks and 28 open issues, and was last pushed 1 months ago. On this registry it ranks #30 of 41 tracked projects in Networking & Connectivity, with 5 head-to-head comparisons available.

What is vulcain?

Vulcain is an open-source protocol and gateway server, written in Go and released under AGPL-3.0, that turns any existing web API into a client-driven REST API by using Preload hints and the 103 Early Hints status code; it is intended for API developers and platform teams who want precise control over which related resources a client fetches without abandoning REST, HTTP caching, or their existing HTTP infrastructure.

What it is

Vulcain is a protocol specification plus a reference implementation. The specification is published as an IETF Internet Draft, draft-dunglas-vulcain, and is maintained in the repository at spec/vulcain.md. The implementation is a production-grade gateway server written in Go, available as a module for the Caddy web server, as a Docker image, and as a legacy standalone server for deployments that do not run Caddy.

The problem it solves is the over-fetching and under-fetching that affects REST APIs, together with the n+1 request pattern that follows from it. Existing answers to that problem, including GraphQL and JSON:API's embedded resources and sparse fieldsets, are described in the project's documentation as smart network hacks for HTTP/1 that carry drawbacks for HTTP caching, logs, and even security. Vulcain replaces those application-level workarounds with native HTTP/2 and HTTP/3 mechanisms: a Preload header the client uses to ask for related resources, 103 Early Hints responses, and server push.

Key capabilities

  • Client-driven relation pushing through the Preload HTTP header, so a request for /books/1 can ask the server to also send a relation such as /authors/1.
  • 103 Early Hints support, allowing preload hints to be sent before the final response body is produced.
  • HTTP/2 Server Push and HTTP/3 support, the transport features the protocol is designed around.
  • A Caddy web server module, documented in docs/gateway/caddy.md, which is placed in front of an existing API.
  • A legacy standalone gateway server with its own installation and configuration documentation (docs/gateway/install.md, docs/gateway/config.md), alongside a provided Docker image.
  • Mapping of non-hypermedia "legacy" APIs to Vulcain relations using OpenAPI (docs/gateway/openapi.md), next to native support for hypermedia APIs such as those created with API Platform.
  • GraphQL usable as a query language on top of Vulcain (docs/graphql.md), and resource filtering as a documented feature.

Who uses it and how

  • Teams running a hypermedia or HATEOAS API, for example one built with API Platform, can put the Caddy module in front of the existing API and expose Vulcain relations without rewriting their resources.
  • Teams with legacy REST APIs that publish no hypermedia links can document the relations with OpenAPI and let the gateway answer preload requests from that description.
  • Developers who want GraphQL-style selection of related data but need to preserve the HTTP cache, logging, and security behaviour that the project argues application-level query languages compromise.
  • Operators who prefer the gateway and the query mechanism in a single component, deployed either as a Caddy module or as the standalone server behind existing infrastructure.

Getting started

The gateway is installed either as the Caddy web server module or as the legacy standalone server, documented in docs/gateway/caddy.md and docs/gateway/install.md respectively, with the Go package available at github.com/dunglas/vulcain/gateway. A Docker image is provided, and the homepage at vulcain.rocks links to the documentation and to the docs/help.md support page.

How it compares

The closest alternatives named in the project's own material are GraphQL and JSON:API's embedded resources and sparse fieldsets, both characterised there as network hacks for HTTP/1. Vulcain's position is that the same fetching control can be expressed with standard HTTP semantics, namely preload hints and early hints, so caching, logging, and security continue to behave as HTTP intends. GraphQL is not excluded: the project documents using GraphQL as a query language for Vulcain.

When to use it — and when not to

Adopters must run a gateway, either a Caddy instance or the standalone server, in front of their API, terminate HTTP/2 or HTTP/3, and describe relations through hypermedia links or an OpenAPI document; the project keeps separate guidance on cache considerations. Anyone who cannot accept the AGPL-3.0 licence, or who wants an in-process library rather than an additional network hop, should look elsewhere. A further caveat is that the protocol is still an Internet Draft rather than a finished standard, so adopters depend on this reference implementation.

project readme (upstream, from github) — read inline

Vulcain is a brand new protocol using Preload hints and the 103 Early Hints status code to create fast and idiomatic client-driven REST APIs.

An open source gateway server (a module for the Caddy web server), which you can put on top of any existing web API to instantly turn it into a Vulcain-compatible API is also provided!

It supports hypermedia APIs (e.g. any API created with API Platform) but also any "legacy" API by documenting its relations using OpenAPI.

Plant Tree PkgGoDev Build Status codecov Go Report Card

[tabs]

Preload

Vulcain Schema

Preload + Early Hints

Vulcain Schema

Server push

Vulcain Schema

[/tabs]

Grab What You Need... Burn The REST!

The protocol has been published as an Internet Draft that is maintained in this repository.

A reference, production-grade, implementation gateway server is also available in this repository. It's free software (AGPL) written in Go. A Docker image is provided.

Introduction

Over the years, several formats have been created to fix performance bottlenecks impacting web APIs: over fetching, under fetching, the n+1 problem...

Current solutions for these problems (GraphQL, JSON:API's embedded resources and sparse fieldsets, ...) are smart network hacks for HTTP/1. But these hacks come with (too) many drawbacks when it comes to HTTP cache, logs and even security.

Fortunately, thanks to the new features introduced in HTTP/2, it's now possible to create true REST APIs fixing these problems with ease and class! Here comes Vulcain!

See also the comparison between Vulcain and GraphQL and other API formats.

Pushing Relations

[tabs]

Preload

Preload Schema

Preload + Early Hints

Preload Schema

Server push

Preload Schema

[/tabs]

Considering the following resources:

/books

{
    "member": [
        "/books/1",
        "/books/2"
    ]
}

/books/1

{
    "title": "1984",
    "author": "/authors/1"
}

/books/2

{
    "title": "Homage to Catalonia",
    "author": "/authors/1"
}

/authors/1

{
    "givenName": "George",
    "familyName": "Orwell"
}

The Preload HTTP header introduced by Vulcain can be used to ask the server to immediately push resources related to the requested one using 103 Early Hints or HTTP/2 Server Push:

GET /books/ HTTP/2
Preload: "/member/*/author"

In addition to /books, a Vulcain server will push the /books/1, /books/2 and /authors/1 resources!

Example in 

const bookResp = await fetch("/books/1", { headers: { Preload: `"/author"` } });
const bookJSON = await bookResp.json();

// Returns immediately, the resource has been pushed and is already in the push cache
const authorResp = await fetch(bookJSON.author);
// ...

Full example, including collections, see also use GraphQL as query language for Vulcain.

Thanks to HTTP/2+ multiplexing, pushed responses will be sent in parallel.

When the client will follow the links and issue a new HTTP request (for instance using fetch()), the corresponding response will already be in cache, and will be used instantly!

For non-hypermedia APIs (when the identifier of the related resource is a simple string or int), use an OpenAPI specification to configure links between resources. Tip: the easiest way to create a hypermedia API is to use the API Platform framework (by the same author as Vulcain).

When possible, we recommend using Early Hints (the 103 HTTP status code) to push the relations. Vulcain allows to gracefully fallback to preload links in the headers of the final response or to HTTP/2 Server Push when the 103 status code isn't supported.

Query Parameter

Alternatively to HTTP headers, the preload query parameter can be used:

[tabs]

Preload

Preload Query Schema

Preload + Early Hints

Preload Query Schema

Server push

Preload Query Schema

[/tabs]

Filtering Resources

[tabs]

Preload

Filter Schema

Preload + Early Hints

Filter Schema

Server push

Filter Schema

[/tabs]

The Fields HTTP header allows the client to ask the server to return only the specified fields of the requested resource, and of the preloaded related resources.

Multiple Fields HTTP headers can be passed. All fields matching at least one of these headers will be returned. Other fields of the resource will be omitted.

Considering the following resources:

/books/1

{
    "title": "1984",
    "genre": "novel",
    "author": "/authors/1"
}

/authors/1

{
    "givenName": "George",
    "familyName": "Orwell"
}

And the following HTTP request:

GET /books/1 HTTP/2
Preload: "/author"
Fields: "/author/familyName", "/genre"

A Vulcain server will return a response containing the following JSON document:

{
    "genre": "novel",
    "author": "/authors/1"
}

It will also push the following filtered /authors/1 resource:

{
    "familyName": "Orwell"
}

Query Parameter

Alternatively to HTTP headers, the fields query parameter can be used to filter resources:

[tabs]

Preload

Fields Schema

Preload + early hints

Fields Schema

Server push

Fields Schema

[/tabs]

See Also

License and Copyright

tl;dr:

  • proprietary software can implement the Vulcain specification
  • proprietary software can be used behind the Vulcain Gateway Server without having to share their sources
  • modifications made to the Vulcain Gateway Server must be shared
  • alternatively, a commercial license is available for the Vulcain Gateway Server

The specification is available under the IETF copyright policy. The Vulcain specification can be implemented by any software, including proprietary software.

The Vulcain Gateway Server is licensed under AGPL-3.0. This license implies that if you modify the Vulcain Gateway Server, you must share those modifications. However, the AGPL-3.0 license applies only to the gateway server itself, not to software used behind the gateway.

For companies not wanting, or not able to use AGPL-3.0 licensed software, commercial licenses are also available. Contact us for more information.

Treeware

This package is Treeware. If you use it in production, then we ask that you buy the world a tree to thank us for our work. By contributing to the Treeware forest you’ll be creating employment for local families and restoring wildlife habitats.

Credits

Created by Kévin Dunglas. Sponsored by Les-Tilleuls.coop.

Some ideas and code used in Vulcain's reference implementation have been taken from Hades by Gabe Sullice, an HTTP/2 reverse proxy for JSON:API backend.

See also the prior arts.

Frequently asked questions

Is vulcain free to use?

vulcain is open source under the AGPL-3.0 licence. There is no licence fee and no seat count — you can self-host it or, where the project offers one, pay a vendor for a managed version instead.

What does vulcain do?

🔨 Fast and idiomatic client-driven REST APIs.

What is vulcain written in?

vulcain is primarily written in Go. Its source is publicly available at https://github.com/dunglas/vulcain, and it has 3,595 GitHub stars.