Prometheus metric library for Nginx

This is a Lua library that can be used with Nginx to keep track of metrics and expose them on a separate web page to be pulled by Prometheus.

Installation

To use this library, you will need the ngx_lua nginx module. You can either use a lua-enabled nginx-based server like OpenResty, or a regular nginx server with the module enabled: for example, on Debian 10 you can simply install libnginx-mod-http-lua (but please read the known issues if you use a later Debian version).

The library file - prometheus.lua - needs to be available in LUA_PATH. If this is the only Lua library you use, you can just point lua_package_path to the directory with this git repo checked out (see example below).

OpenResty users will find this library in opm. It is also available via luarocks.

Quick start guide

To track request latency broken down by server name and request count broken down by server name and status, add the following to the http section of nginx.conf:

lua_shared_dict prometheus_metrics 10M;
lua_package_path "/path/to/nginx-lua-prometheus/?.lua;;";

init_worker_by_lua_block {
  prometheus = require("prometheus").init("prometheus_metrics")

  metric_requests = prometheus:counter(
    "nginx_http_requests_total", "Number of HTTP requests", {"host", "status"})
  metric_latency = prometheus:histogram(
    "nginx_http_request_duration_seconds", "HTTP request latency", {"host"})
  metric_connections = prometheus:gauge(
    "nginx_http_connections", "Number of HTTP connections", {"state"})
}

log_by_lua_block {
  metric_requests:inc(1, {ngx.var.server_name, ngx.var.status})
  metric_latency:observe(tonumber(ngx.var.request_time), {ngx.var.server_name})
}

This:

• configures a shared dictionary for your metrics called prometheus_metrics with a 10MB size limit;
• registers a counter called nginx_http_requests_total with two labels: host and status;
• registers a histogram called nginx_http_request_duration_seconds with one label host;
• registers a gauge called nginx_http_connections with one label state;
• on each HTTP request measures its latency, recording it in the histogram and increments the counter, setting current server name as the host label and HTTP status code as the status label.

Last step is to configure a separate server that will expose the metrics. Please make sure to only make it reachable from your Prometheus server:

server {
  listen 9145;
  allow 192.168.0.0/16;
  deny all;
  location /metrics {
    content_by_lua_block {
      metric_connections:set(ngx.var.connections_reading, {"reading"})
      metric_connections:set(ngx.var.connections_waiting, {"waiting"})
      metric_connections:set(ngx.var.connections_writing, {"writing"})
      prometheus:collect()
    }
  }
}

Metrics will be available at http://your.nginx:9145/metrics. Note that the gauge metric in this example contains values obtained from nginx global state, so they get set immediately before metrics are returned to the client.

API reference

init()

syntax: require("prometheus").init(dict_name, [options]])

Initializes the module. This should be called once from the init_worker_by_lua_block section of nginx configuration.

• dict_name is the name of the nginx shared dictionary which will be used to store all metrics. Defaults to prometheus_metrics if not specified.
• options is a table of configuration options that can be provided. Accepted options are:
  • prefix (string): metric name prefix. This string will be prepended to metric names on output.
  • error_metric_name (string): Can be used to change the default name of error metric (see Built-in metrics for details).
  • sync_interval (number): sets the sync interval for per-worker counters and key index (in seconds). This sets the boundary on eventual consistency of counter metric increments, and metric resets/deletions. Defaults to 1.

Returns a prometheus object that should be used to register metrics.

Example:

init_worker_by_lua_block {
  prometheus = require("prometheus").init("prometheus_metrics", {sync_interval=3})
}

prometheus:counter()

syntax: prometheus:counter(name, description, label_names)

Registers a counter. Should be called once for each counter from the init_worker_by_lua_block section.

• name is the name of the metric.
• description is the text description that will be presented to Prometheus along with the metric. Optional (pass nil if you still need to define label names).
• label_names is an array of label names for the metric. Optional.

Naming section of Prometheus documentation provides good guidelines on choosing metric and label names.

Returns a counter object that can later be incremented.

Example:

init_worker_by_lua_block {
  prometheus = require("prometheus").init("prometheus_metrics")

  metric_bytes = prometheus:counter(
    "nginx_http_request_size_bytes", "Total size of incoming requests")
  metric_requests = prometheus:counter(
    "nginx_http_requests_total", "Number of HTTP requests", {"host", "status"})
}

prometheus:gauge()

syntax: prometheus:gauge(name, description, label_names)

Registers a gauge. Should be called once for each gauge from the init_worker_by_lua_block section.

• name is the name of the metric.
• description is the text description that will be presented to Prometheus along with the metric. Optional (pass nil if you still need to define label names).
• label_names is an array of label names for the metric. Optional.

Returns a gauge object that can later be set.

Example:

init_worker_by_lua_block {
  prometheus = require("prometheus").init("prometheus_metrics")

  metric_connections = prometheus:gauge(
    "nginx_http_connections", "Number of HTTP connections", {"state"})
}

prometheus:histogram()

syntax: prometheus:histogram(name, description, label_names, buckets)

Registers a histogram. Should be called once for each histogram from the init_worker_by_lua_block section.

• name is the name of the metric.
• description is the text description. Optional.
• label_names is an array of label names for the metric. Optional.
• buckets is an array of finite numbers defining strictly increasing bucket boundaries. Optional, defaults to 20 latency buckets covering a range from 5ms to 10s (in seconds).

Returns a histogram object that can later be used to record samples.

Example:

init_worker_by_lua_block {
  prometheus = require("prometheus").init("prometheus_metrics")

  metric_latency = prometheus:histogram(
    "nginx_http_request_duration_seconds", "HTTP request latency", {"host"})
  metric_response_sizes = prometheus:histogram(
    "nginx_http_response_size_bytes", "Size of HTTP responses", nil,
    {10,100,1000,10000,100000,1000000})
}

prometheus:collect()

syntax: prometheus:collect()

Presents all metrics in a text format compatible with Prometheus. This should be called in content_by_lua_block to expose the metrics on a separate HTTP page.

Example:

location /metrics {
  content_by_lua_block { prometheus:collect() }
  allow 192.168.0.0/16;
  deny all;
}

prometheus:metric_data()

syntax: prometheus:metric_data()

Returns metric data as an array of strings.

counter:inc()

syntax: counter:inc(value, label_values)

Increments a previously registered counter. This is usually called from log_by_lua_block globally or per server/location.

• value is a value that should be added to the counter. Defaults to 1.
• label_values is an array of label values.

The number of label values should match the number of label names defined when the counter was registered using prometheus:counter(). No label values should be provided for counters with no labels. Non-printable characters will be stripped from label values.

Example:

log_by_lua_block {
  metric_bytes:inc(tonumber(ngx.var.request_l