gnostic: Open Source API Development & Testing

What is gnostic?

gnostic is a Go command line tool from Google that compiles OpenAPI descriptions written in JSON or YAML into Protocol Buffer representations and back again, aimed at developers who need type-safe, language-neutral access to API descriptions and at tool authors who build code generators and other API tooling on top of a plugin interface.

What it is

gnostic is a compiler for APIs described by the OpenAPI Specification, distributed as a Go command line tool with a plugin interface for code generation and other API support tasks. It reads OpenAPI descriptions into generated Protocol Buffer data structures, reports errors, resolves internal dependencies, and writes the results in a binary form usable from any language supported by the Protocol Buffer tools. The compilation code and the OpenAPI Protocol Buffer models are themselves generated from an OpenAPI JSON Schema, with the generator source kept in the generate-gnostic directory.

The concrete problem it solves is working with OpenAPI documents in a type-safe way. Raw JSON and YAML descriptions carry no compile-time guarantees, and hand-rolled parsers for them are brittle inside strongly-typed languages. gnostic replaces ad hoc JSON and YAML handling with generated data structures that have explicit fields for the elements of an OpenAPI description, which matters most in languages such as Go and Dart. It lives in the OpenAPI ecosystem and is published under the Apache-2.0 licence, with 2302 stars, 279 forks, and 119 open issues.

Key capabilities

Converts JSON and YAML OpenAPI descriptions to and from equivalent Protocol Buffer representations, for example gnostic --pb-out=. examples/v2.0/json/petstore.json, which writes a binary Protocol Buffer description named petstore.pb into the current directory.
Compiles files specified by local path or by URL.
Exposes a plugin interface that simplifies integration with API tools written in a variety of different languages; make builds all plugins and associated tools in the repository.
Re-exports Protocol Buffer OpenAPI descriptions as JSON or YAML when needed.
Generates data structures with explicit fields for OpenAPI elements, enabling type-safe work in strongly-typed languages.
Ships protoc-gen-openapi in cmd/protoc-gen-openapi to go from protobuf to OpenAPI.
Registry topics classify it under code-generation, linters, openapi, openapi3, and protocol-buffers.

Who uses it and how

Go projects that need only the models import packages from gnostic-models instead of gnostic, for a low-dependency integration.
Teams wanting a Go client for a described API can use the experimental gnostic-go-generator plugin.
Users who need an API that, when transcoded, conforms to a specified OpenAPI document use the gnostic-grpc plugin to generate an annotated Protocol Buffer description.
Tool authors wire gnostic into API tooling written in other languages through the plugin interface, rather than reimplementing OpenAPI parsing.
Dependent projects are asked to refer to tagged releases to keep builds stable, given the prerelease status.

Getting started

Install by cloning with git clone https://github.com/google/gnostic, verifying a local protoc installation, then building with make and checking the result with make test; gnostic runs in any environment that supports Go and the Protocol Buffer Compiler.

How it compares

No list of paid products it replaces is provided, and the registry names no direct competitor, so gnostic stands alongside its own sibling projects rather than against an established alternative. Those siblings mark its boundaries: gnostic-models offers the protobuf models alone for lightweight integration, gnostic-grpc and gnostic-go-generator extend it as plugins, and protoc-gen-openapi covers the protobuf-to-OpenAPI direction. Where a project needs only generated models or only one direction of translation, the smaller sibling repository is the better fit.

When to use it — and when not to

A self-hoster must supply a Go toolchain and the Protocol Buffer Compiler, and must build the tool from source with make; this is a compiler and code generation dependency, not a hosted service. gnostic is explicitly prerelease software and work in progress until a 1.0 release, with 119 open issues, so teams that require a frozen, stable API surface or long-term support guarantees should not depend on it directly, and should at minimum pin tagged releases. Projects that only need a lightweight set of the generated models are better served by gnostic-models.

project readme (upstream, from github) — read inline

⨁ gnostic

This repository contains a Go command line tool which converts JSON and YAML OpenAPI descriptions to and from equivalent Protocol Buffer representations.

Protocol Buffers provide a language-neutral, platform-neutral, extensible mechanism for serializing structured data. gnostic's Protocol Buffer models for the OpenAPI Specification can be used to generate code that includes data structures with explicit fields for the elements of an OpenAPI description. This makes it possible for developers to work with OpenAPI descriptions in type-safe ways, which is particularly useful in strongly-typed languages like Go and Dart.

gnostic reads OpenAPI descriptions into these generated data structures, reports errors, resolves internal dependencies, and writes the results in a binary form that can be used in any language that is supported by the Protocol Buffer tools. A plugin interface simplifies integration with API tools written in a variety of different languages, and when necessary, Protocol Buffer OpenAPI descriptions can be reexported as JSON or YAML.

gnostic compilation code and OpenAPI Protocol Buffer models are automatically generated from an OpenAPI JSON Schema. Source code for the generator is in the generate-gnostic directory.

Related Repositories

google/gnostic-models contains a lightweight distribution of the protobuf models generated by this project. Where a low-dependency integration of just these models is needed, Go projects can import packages from gnostic-models instead of gnostic.

google/gnostic-grpc contains a gnostic plugin that can generate an annotated Protocol Buffer description of an API that, when transcoded, produces an API that conforms to a specified OpenAPI document. To go from protobuf to OpenAPI, see the protoc-gen-openapi tool in this project.

google/gnostic-go-generator contains an experimental gnostic plugin that generates a Go client for an API described by a specified OpenAPI document.

Disclaimer

Feedback and contributions are welcome! Until there is a 1.0 release, please consider this prerelease software and work in progress. To ensure stable builds, we request that dependent projects always refer to tagged releases of gnostic.

Requirements

gnostic can be run in any environment that supports Go and the Protocol Buffer Compiler.

Installation and Getting Started

The following instructions are for installing gnostic using Go modules, supported by Go 1.11 and later.

Get this package by downloading it with git clone.

git clone https://github.com/google/gnostic
cd gnostic

Verify that you have a local installation of protoc. You can get protoc here.
Build gnostic with make. This uses go generate to build support code including code generated by protoc and the Go protoc plugin, which is automatically downloaded from github.com/golang/protobuf by the COMPILE-PROTOS.sh script. This also builds all plugins and associated tools in this repo.
Verify gnostic with make test. These tests are run by gnostic's continuous integration, so you should expect them to pass for all release versions.
Run gnostic. This sample invocation creates a file in the current directory named petstore.pb that contains a binary Protocol Buffer description of a sample API.
```
    gnostic --pb-out=. examples/v2.0/json/petstore.json
```
You can also compile files that you specify with a URL. Here's another way to compile the previous example. This time we're creating petstore.text, which contains a textual representation of the Protocol Buffer description. This is mainly for use in testing and debugging.
```
    gnostic --text-out=petstore.text https://raw.githubusercontent.com/google/gnostic/master/examples/v2.0/json/petstore.json
```
For a sample application, see apps/report. This reads a binary Protocol Buffer encoding created by gnostic.
```
go install ./apps/report ## automatically installed by the top-level Makefile
report petstore.pb
```
gnostic also supports plugins. gnostic's plugin interface is modeled on protoc's plugin.proto and is described in plugins/plugin.proto. Several plugins are implemented in the plugins directory. Others, like gnostic-grpc and gnostic-go-generator, are published in their own repositories. One such plugin is gnostic-vocabulary, which produces a summary of the word usage in an APIs interfaces. You can run gnostic-vocabulary with the following:
```
    gnostic examples/v2.0/json/petstore.json --vocabulary_out=.
```
This will produce files named vocabulary.pb and vocabulary.json in examples/v2.0/json. For the format of vocabulary.pb, see metrics/vocabulary.proto.
[Optional] A large part of gnostic is automatically-generated by the generate-gnostic tool. This uses JSON schemas to generate Protocol Buffer language files that describe supported API specification formats and Go-language files of code that will read JSON or YAML API descriptions into the generated protocol buffer models. Pre-generated versions of these files are checked into the openapiv2, openapiv3, and discovery directories. You can regenerate this code with the following:
```
go install ./generate-gnostic
generate-gnostic --v2
generate-gnostic --v3
generate-gnostic --discovery
```

Copyright

License

Released under the Apache 2.0 license.

Frequently asked questions

Is gnostic free to use?

gnostic is open source under the Apache-2.0 licence. There is no licence fee and no seat count — you can self-host it or, where the project offers one, pay a vendor for a managed version instead.

What does gnostic do?

A compiler for APIs described by the OpenAPI Specification with plugins for code generation and other API support tasks.

What is gnostic written in?

gnostic is primarily written in Go. Its source is publicly available at https://github.com/google/gnostic, and it has 2,302 GitHub stars.