runn is open source under the MIT licence. There is no licence fee and no seat count — you can self-host it or, where the project offers one, pay a vendor for a managed version instead.

What is runn written in?

runn is primarily written in Go. Its source is publicly available at https://github.com/k1LoW/runn, and it has 649 GitHub stars.

runn: Open Source API Development & Testing

Q: What does runn do?

runn is a package/tool for running operations following a scenario.

Coverage Code to Test Ratio Test Execution Time

runn ( means "Run N". is pronounced /rʌ́n én/. ) is a package/tool for running operations following a scenario.

Key features of runn are:

As a tool for scenario based testing.
As a test helper package for the Go language.
As a tool for workflow automation.
Support HTTP request, gRPC request, DB query, Chrome DevTools Protocol, and SSH/Local command execution
OpenAPI Document-like syntax for HTTP request testing.
Single binary = CI-Friendly.

Online book

Quickstart

You can use the runn new command to quickly start creating scenarios (runbooks).

:rocket: Create and run scenario using curl or grpcurl commands:

docs/runn.svg

Command details

$ curl https://httpbin.org/json -H "accept: application/json"
{
  "slideshow": {
    "author": "Yours Truly",
    "date": "date of publication",
    "slides": [
      {
        "title": "Wake up to WonderWidgets!",
        "type": "all"
      },
      {
        "items": [
          "Why <em>WonderWidgets</em> are great",
          "Who <em>buys</em> WonderWidgets"
        ],
        "title": "Overview",
        "type": "all"
      }
    ],
    "title": "Sample Slide Show"
  }
}
$ runn new --and-run --desc 'httpbin.org GET' --out http.yml -- curl https://httpbin.org/json -H "accept: application/json"
$ grpcurl -d '{"greeting": "alice"}' grpcb.in:9001 hello.HelloService/SayHello
{
  "reply": "hello alice"
}
$ runn new --and-run --desc 'grpcb.in Call' --out grpc.yml -- grpcurl -d '{"greeting": "alice"}' grpcb.in:9001 hello.HelloService/SayHello
$ runn list *.yml
  Desc             Path      If
---------------------------------
  grpcb.in Call    grpc.yml
  httpbin.org GET  http.yml
$ runn run *.yml
..

2 scenarios, 0 skipped, 0 failures

:rocket: Create scenario using access log:

docs/runn_axslog.svg

Command details

$ cat access_log
183.87.255.54 - - [18/May/2019:05:37:09 +0200] "GET /?post=%3script%3ealert(1); HTTP/1.0" 200 42433
62.109.16.162 - - [18/May/2019:05:37:12 +0200] "GET /core/files/js/editor.js/?form=\xeb\x2a\x5e\x89\x76\x08\xc6\x46\x07\x00\xc7\x46\x0c\x00\x00\x00\x80\xe8\xdc\xff\xff\xff/bin/sh HTTP/1.0" 200 81956
87.251.81.179 - - [18/May/2019:05:37:13 +0200] "GET /login.php/?user=admin&amount=100000 HTTP/1.0" 400 4797
103.36.79.144 - - [18/May/2019:05:37:14 +0200] "GET /authorize.php/.well-known/assetlinks.json HTTP/1.0" 200 9436
$ cat access_log| runn new --out axslog.yml
$ cat axslog.yml| yq
desc: Generated by `runn new`
runners:
  req: https://dummy.example.com
steps:
  - req:
      /?post=%3script%3ealert(1);:
        get:
          body: null
  - req:
      /core/files/js/editor.js/?form=xebx2ax5ex89x76x08xc6x46x07x00xc7x46x0cx00x00x00x80xe8xdcxffxffxff/bin/sh:
        get:
          body: null
  - req:
      /login.php/?user=admin&amount=100000:
        get:
          body: null
  - req:
      /authorize.php/.well-known/assetlinks.json:
        get:
          body: null
$

Usage

runn can run a multi-step scenario following a runbook written in YAML format.

As a tool for scenario based testing / As a tool for automation.

runn can run one or more runbooks as a CLI tool.

$ runn list path/to/**/*.yml
  id:      desc:             if:       steps:  path
-------------------------------------------------------------------------
  a1b7b02  Only if included  included       2  p/t/only_if_included.yml
  85ccd5f  List projects.                   4  p/t/p/list.yml
  47d7ef7  List users.                      3  p/t/u/list.yml
  97f9884  Login                            2  p/t/u/login.yml
  2249d1b  Logout                           3  p/t/u/logout.yml
$ runn run path/to/**/*.yml
S....

5 scenarios, 1 skipped, 0 failures

As a test helper package for the Go language.

runn can also behave as a test helper for the Go language.

Run N runbooks using httptest.Server and sql.DB

func TestRouter(t *testing.T) {
	ctx := context.Background()
	dsn := "username:password@tcp(localhost:3306)/testdb"
	db, err := sql.Open("mysql", dsn)
	if err != nil {
		log.Fatal(err)
	}
	dbr, err := sql.Open("mysql", dsn)
	if err != nil {
		log.Fatal(err)
	}
	ts := httptest.NewServer(NewRouter(db))
	t.Cleanup(func() {
		ts.Close()
		db.Close()
		dbr.Close()
	})
	opts := []runn.Option{
		runn.T(t),
		runn.Runner("req", ts.URL),
		runn.DBRunner("db", dbr),
	}
	o, err := runn.Load("testdata/books/**/*.yml", opts...)
	if err != nil {
		t.Fatal(err)
	}
	if err := o.RunN(ctx); err != nil {
		t.Fatal(err)
	}
}

Run single runbook using httptest.Server and sql.DB

func TestRouter(t *testing.T) {
	ctx := context.Background()
	dsn := "username:password@tcp(localhost:3306)/testdb"
	db, err := sql.Open("mysql", dsn)
	if err != nil {
		log.Fatal(err)
	}
	dbr, err := sql.Open("mysql", dsn)
	if err != nil {
		log.Fatal(err)
	}
	ts := httptest.NewServer(NewRouter(db))
	t.Cleanup(func() {
		ts.Close()
		db.Close()
		dbr.Close()
	})
	opts := []runn.Option{
		runn.T(t),
		runn.Book("testdata/books/login.yml"),
		runn.Runner("req", ts.URL),
		runn.DBRunner("db", dbr),
	}
	o, err := runn.New(opts...)
	if err != nil {
		t.Fatal(err)
	}
	if err := o.Run(ctx); err != nil {
		t.Fatal(err)
	}
}

Run N runbooks using grpc.Server

func TestServer(t *testing.T) {
	addr := "127.0.0.1:8080"
	l, err := net.Listen("tcp", addr)
	if err != nil {
		t.Fatal(err)
	}
	ts := grpc.NewServer()
	myapppb.RegisterMyappServiceServer(s, NewMyappServer())
	reflection.Register(s)
	go func() {
		ts.Serve(l)
	}()
	t.Cleanup(func() {
		ts.GracefulStop()
	})
	opts := []runn.Option{
		runn.T(t),
		runn.Runner("greq", fmt.Sprintf("grpc://%s", addr),
	}
	o, err := runn.Load("testdata/books/**/*.yml", opts...)
	if err != nil {
		t.Fatal(err)
	}
	if err := o.RunN(ctx); err != nil {
		t.Fatal(err)
	}
}

Run N runbooks with http.Handler and sql.DB

func TestRouter(t *testing.T) {
	ctx := context.Background()
	dsn := "username:password@tcp(localhost:3306)/testdb"
	db, err := sql.Open("mysql", dsn)
	if err != nil {
		log.Fatal(err)
	}
	dbr, err := sql.Open("mysql", dsn)
	if err != nil {
		log.Fatal(err)
	}
	t.Cleanup(func() {
		db.Close()
		dbr.Close()
	})
	opts := []runn.Option{
		runn.T(t),
		runn.HTTPRunnerWithHandler("req", NewRouter(db)),
		runn.DBRunner("db", dbr),
	}
	o, err := runn.Load("testdata/books/**/*.yml", opts...)
	if err != nil {
		t.Fatal(err)
	}
	if err := o.RunN(ctx); err != nil {
		t.Fatal(err)
	}
}

Examples

See the details

Runbook ( runn scenario file )

The runbook file has the following format.

step: section accepts list or ordered map.

List:

desc: Login and get projects.
runners:
  req: https://example.com/api/v1
  db: mysql://root:mypass@localhost:3306/testdb
vars:
  username: alice
  password: ${TEST_PASS}
steps:
  -
    db:
      query: SELECT * FROM users WHERE name = '{{ vars.username }}'
  -
    req:
      /login:
        post:
          body:
            application/json:
              email: "{{ steps[0].rows[0].email }}"
              password: "{{ vars.password }}"
    test: steps[1].res.status == 200
  -
    req:
      /projects:
        get:
          headers:
            Authorization: "token {{ steps[1].res.body.session_token }}"
          body: null
    test: steps[2].res.status == 200
  -
    test: len(steps[2].res.body.projects) > 0

Map:

desc: Login and get projects.
runners:
  req: https://example.com/api/v1
  db: mysql://root:mypass@localhost:3306/testdb
vars:
  username: alice
  password: ${TEST_PASS}
steps:
  find_user:
    db:
      query: SELECT * FROM users WHERE name = '{{ vars.username }}'
  login:
    req:
      /login:
        post:

What is runn?

What it is

Key capabilities

Who uses it and how

Getting started

How it compares

When to use it — and when not to