hl

High-performance log viewer and processor that transforms logs in JSON and logfmt formats into a human-readable output. Built with efficiency in mind, it enables quick parsing and analysis of large log files with minimal overhead.

Features overview

• Automatic Pager Integration: Automatically integrates with a pager for enhanced convenience, defaulting to less if available, but fully supporting any compatible pager.
• Log Streaming Mode: Enable log streaming with the -P flag, which disables the pager.
• Field-Based Filtering: Filter log records by key/value pairs using the -f option, with support for hierarchical keys.
• Level Filtering: Easily filter logs by level with the -l option.
• Timestamp Range Filtering: Filter logs by timestamp range using the --since and --until options with intuitive formats:
  • RFC-3339 timestamp format.
  • Current configured timestamp output format (via the -t option or environment variable).
  • User-friendly shortcuts like today, yesterday, friday, or relative offsets such as -3h or -14d.
• Field Visibility Control: Quickly hide or reveal specific fields using the -h option.
• Empty Field Hiding: Automatically hide empty fields with the -e flag.
• Field Expansion: Control multi-line field and message formatting with the -x option:
  • never: Compact single-line output with escaped newlines.
  • inline: Preserve newlines as-is in the output.
  • auto: Automatically expand fields containing multi-line content.
  • always: Expand all fields into structured multi-line format.
• High-Speed Message Sorting: Achieve lightning-fast message sorting with automatic indexing via the -s flag.
  • Performs the initial scan at approximately 2 GiB/s, enabling rapid filtering by timestamp and level without re-scanning.
  • Efficiently handles hundreds of local files totaling hundreds of gigabytes.
  • Reindexes large, growing files at speeds up to roughly 10 GiB/s by skipping unmodified blocks.
• Live Follow Mode: Use the -F flag for live, timestamp-sorted message updates across multiple sources, with a preview of recent messages via the --tail option.
• Complex Query Support: Construct custom queries with logical conditions (AND/OR) and additional advanced filtering options.
• Non-JSON Prefix Handling: Process logs with non-JSON prefixes using the --allow-prefix flag.
• Timezone Flexibility: Displays timestamps in UTC by default while allowing effortless timezone switching with the -Z option or local timezone adjustments using the -L flag.
• Customizability and Themes: Fully customizable through configuration files and environment variables, with support for easy theme switching and custom themes.

Performance comparison chart

Performance comparison with humanlog, hlogf and fblog on a 2.3 GiB log file

• See performance section for more details.

Installation options

macOS

• Install using homebrew on macOS

  brew install hl
  

Linux

• Download and extract using curl and tar on Linux (x86_64)

  curl -sSfL https://github.com/pamburus/hl/releases/latest/download/hl-linux-x86_64-musl.tar.gz | tar xz
  

• Install hl package from Arch Linux extra repository

  pacman -S hl
  

Windows

• Install from Scoop

  scoop bucket add pamburus https://github.com/pamburus/scoop-bucket.git
  scoop install hl
  

[!TIP] It is recommended to use Windows Terminal for better experience.

[!TIP] To make mouse scrolling work in the less pager, set the LESS environment variable to -R --mouse.

[!IMPORTANT] Currently, hl does not provide a built-in pager and relies on external pagers such as less. However, the build for Windows referenced on the original download page and distributed in the WinGet package manager does not work as expected. The authors state that they have not tested or verified this build and suggest that you use it at your own risk. Unfortunately, this build breaks some ANSI escape sequences and does not work properly with hl and many other programs that use ANSI escape sequences for colors and styles. It is recommended to install less from the Scoop or Chocolatey package manager. If you are using Scoop and install hl by running scoop install hl, it already installs less automatically as a dependency. Just make sure you do not have any other conflicting installations by running where less in cmd or Get-Command less in powershell.

• Download latest release from download page

NixOS

• Run using nix

  nix run github:pamburus/hl
  

  or binary package

  nix run github:pamburus/hl#bin
  

  or install with nix profile:

  nix profile add github:pamburus/hl
  

  or binary package

  nix profile add github:pamburus/hl#bin
  

• Install the package from source using nix-flakes

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-25.05";
    hl.url = "github:pamburus/hl";
  };
  outputs = {nixpkgs, hl, ...}:
  let
    system = "x86_64-linux";
  in
  {
    # this is just an example!
    nixosConfigurations.yourHost = nixpkgs.lib.nixosSystem {
      inherit system;
      modules = [
        ({...}: {
          environment.systemPackages = [
            hl.packages.${system}
          ];
        })
      ];
    };
  };
}

• Install the package with pre-built binaries using nix-flakes

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-25.05";
    hl.url = "github:pamburus/hl";
  };
  outputs = {nixpkgs, hl, ...}:
  let
    system = "x86_64-linux";
  in
  {
    # this is just an example!
    nixosConfigurations.yourHost = nixpkgs.lib.nixosSystem {
      inherit system;
      modules = [
        ({...}: {
          environment.systemPackages = [
            hl.packages.${system}.bin
          ];
        })
      ];
    };
  };
}

Examples

Screenshot

See other screenshots

Features and usage

Concatenation of multiple log files

• Concatenate all log files

  Command

  hl *.log
  

  Concatenates and displays all *.log files found in the current directory.

Support for compressed (bzip2, gzip, xz, zstd) log files

• Concatenate all log files including compressed log files

  Command

  hl $(ls -tr /var/log/example/*.{log,log.gz,log.zst,s})
  

  Concatenates and displays all *.log, *.log.gz, *.log.zst and *.s (will detect compression) files found in /var/log/example/.

Automatic usage of pager

• Use the default pager with the default parameters

  Command

  hl example.log
  

  Automatically opens less pager with the default parameters.

• Override options for default pager

  Command

  LESS=-SR hl example.log
  

  Opens less pager with disabled line wrapping.

• Use custom pager

  Command

  PAGER="most -w" hl example.log
  

  Opens most pager with -w option.

Quick filtering by log level

• Errors only

  Command

  hl -l e
  

  Displays only error log level messages.

• Errors and warnings

  Command

  hl -l w
  

  Displays only warning and error log level messages.

• Errors, warnings and informational

  Command

  hl -l i
  

  Displays all log messages except debug level messages.

Using live log streaming

• Command

  tail -f example.log | hl -P
  

  Tracks changes in the example.log file and displays them immediately. Flag -P disables automatic using of pager in this case.

Filtering by field values

• Command

  hl example.log --filter component=tsdb
  

  Displays only messages where the component field has the value tsdb.

• Command

  hl example.log -f component!=tsdb -f component!=uninteresting
  

  Displays only messages where the component field exists and has a value other than tsdb or uninteresting.

• Command

  hl example.log -f provider~=string
  

  Displays only messages where the provider field contains the string sub-string.

• Command

  hl example.log -f 'provider!~=string'
  

  Displays only messages where the provider field exists and does not contain the string sub-string.

• Command

  hl example.log -f 'request.method?!=GET'
  

  Displays only messages where the request.method field does not exist or exists and is not equal to GET. In JSON messages, this also matches composite fields, e.g., {"request":{"method":"POST"}}.

• Command

  hl example.log -f 'span.[].name=sp0001'
  

  Displays only messages where the span field is an array that contains an object with a name field equal to sp0001.

• Command

  hl example.log -f 'span.[1].name=sp0001'
  

  Displays only messages where the span field is an array with at least two elements, and the element at index 1 (i.e., the second element) is an object with a name field equal to sp0001.

Performing complex queries

• Command

  hl my-service.log --query 'level > info or status-code >= 400 or duration > 0.5'
  

  Displays messages that either have a level higher than info (i.e. warning or error) or have a status code field with a numeric value >= 400 or a duration field with a numeric value >= 0.5.

• Command

  hl my-service.log -q '(request in (95c72499d9ec, 9697f7aa134f, bc3451d0ad60)) or (method != GET)'
  

  Displays all messages that have the 'request' field with one of these values, or the 'method' field with a value other than 'GET'.

• Command

  hl my-service.log -q '.price?=3'
  

  Displays messages where the price field equals 3 OR the price field is absent. The ? modifier after a field name includes records that don't have that field. Without the ? modifier, filtering on a missing field will skip the record.

• Command

  hl my-service.log -q '.status?!=error'
  

  Displays messages where the status field is not equal to error OR the status field is absent. This is useful to include records that may not have a status field at all.

• Command

  hl my-service.log -q 'exists(.price)'
  

  Displays only messages that have the price field, regardless of its value. The exists() operator tests for field presence without any value comparison.

• Command

  hl my-service.log -q 'exists(msg) and msg~=warning'
  

  Displays messages that have a msg field that exists AND contains the substring warning. This combines field existence checking with value matching.

• Command

  hl my-service.log -q 'not exists(.internal)'
  

  Displays messages that do NOT have an internal field, effectively filtering out records with that field.

• Complete set of supported operators

  • Logical operators

    • Logical conjunction - and, &&
    • Logical disjunction - or, ||
    • Logical negation - not, !

  • Comparison operators

    • Equal - eq, =

    • Not equal - ne, !=

    • Greater than - gt, >

    • Greater or equal - ge, >=

    • Less than - lt, `/dev/null hlogf example.log --color always > /dev/null 6.79s user 2.05s system 100% cpu 8.787 total

    • humanlog v0.8.9 ~ 123 seconds

      $ time humanlog /dev/null
      humanlog --color always  /dev/null  136.18s user 7.17s system 116% cpu 2:03.25 total
      

    • fblog v4.17.0 ~ 34 seconds

      $ time fblog example.log >/dev/null
      fblog example.log > /dev/null  31.92s user 2.39s system 99% cpu 34.327 total
      

    • fblog with -d flag v4.17.0 ~ 164 seconds

      $ time fblog -d example.log >/dev/null
      fblog -d example.log > /dev/null  147.57s user 16.20s system 99% cpu 2:43.82 total
      

  • See #132 for how to repeat measurements