evo: Open Source Monitoring & Observability

What is evo?

evo is a Python package, released under the GPL-3.0 licence, that provides executables and a small library for handling, evaluating and comparing the trajectory output of odometry and SLAM algorithms, and it is aimed at robotics researchers and engineers who need to benchmark trajectories rather than maintain one evaluation script per dataset.

What it is

evo lives in the Python robotics ecosystem and treats ROS and ROS2 as first-class sources of data. It provides common tools for different trajectory formats: TUM trajectory files, KITTI pose files, and EuRoC MAV data in the form of a .csv groundtruth plus a TUM trajectory file. On the ROS side it reads bagfiles containing geometry_msgs/PoseStamped, geometry_msgs/TransformStamped, geometry_msgs/PoseWithCovarianceStamped, geometry_msgs/PointStamped or nav_msgs/Odometry topics, as well as TF messages. Around that core it offers a configurable command line interface and modular core and tools libraries intended for custom extensions. The project is widely adopted, with 4316 stars, 797 forks and 7 open issues at the time of writing.

The concrete problem it solves is the fragmentation of trajectory evaluation. Teams running odometry or SLAM systems accumulate separate scripts for each dataset and each format, and results become hard to compare across runs. evo replaces that patchwork with one set of utilities that ingest several formats and apply the same evaluation machinery to all of them. Its algorithmic options cover association, alignment and scale adjustment for monocular SLAM, which are exactly the steps that differ between experiments. The README is explicit that evo is not a 1-to-1 re-implementation of a particular evaluation protocol tailored to a specific dataset, so it supplies reusable tools rather than a fixed scoring recipe.

Key capabilities

Reads and compares trajectories from TUM trajectory files, KITTI pose files, EuRoC MAV .csv groundtruth plus TUM trajectory files, and ROS or ROS2 bagfiles.
Extracts poses from geometry_msgs/PoseStamped, geometry_msgs/TransformStamped, geometry_msgs/PoseWithCovarianceStamped, geometry_msgs/PointStamped, nav_msgs/Odometry topics and TF messages.
Applies algorithmic options for association, alignment and scale adjustment, including the scale handling needed for monocular SLAM.
Produces flexible output, including plotting, visualization, LaTeX plots and Excel tables.
Exposes a powerful, configurable CLI, with tab completion available through the argcomplete package via activate-global-python-argcomplete.
Ships modular core and tools libraries so custom extensions can be built on the same primitives.
Offers evo_config for persistent settings, for example evo_config set plot_backend qtagg to switch the matplotlib backend.
Reads ROS 1 and ROS 2 bags through the rosbags package even without a ROS installation.

Who uses it and how

SLAM and odometry researchers benchmarking algorithm output against KITTI, EuRoC MAV or TUM trajectory data.
Robotics teams on ROS or ROS2 evaluating trajectories recorded in bagfiles from real robot runs, with or without a full ROS install on the analysis machine.
Developers of monocular SLAM systems that need scale adjustment and alignment before comparing runs.
Authors preparing publications who export LaTeX plots or Excel tables from evaluation results.
Engineers extending the tooling through the core and tools libraries, or driving it from scripts rather than the CLI.

Getting started

Install the released executables from PyPI with pip install evo, using Python 3.10 or newer inside a proper environment such as uv, uvx, pipx, venv or Pixi. A source checkout installs with pip install --editable . from the repository base folder, and Pixi users can run pixi add evo in an existing environment or pixi shell in a clone.

How it compares

No paid products and no competing tools are named in the material available for this listing, so evo stands alone in this registry. The README only claims advantages over other public benchmarking tools in general terms, citing common tools for different formats, algorithmic options for association, alignment and scale adjustment, flexible output and plotting, a configurable CLI, modular libraries, and better speed than other established Python-based tools.

When to use it — and when not to

A self-hoster needs a working Python 3.10 or newer environment; pyproject.toml declares the dependencies that pip resolves automatically, and PyQt6 is optional and only adds the qtagg plot backend. A ROS installation is required only for some ROS-related features, since bag reading works through rosbags alone, so users without ROS are not locked out. Users who want a fixed, dataset-specific evaluation protocol reimplemented exactly should look elsewhere, and the README's installation warning is worth heeding: avoid pip install --user, anything with --break-system-packages, and sudo pip install.

project readme (upstream, from github) — read inline

evo

Python package for the evaluation of odometry and SLAM

Linux / macOS / Windows / ROS / ROS2

This package provides executables and a small library for handling, evaluating and comparing the trajectory output of odometry and SLAM algorithms.

Supported trajectory formats:

'TUM' trajectory files
'KITTI' pose files
'EuRoC MAV' (.csv groundtruth and TUM trajectory file)
ROS and ROS2 bagfile with geometry_msgs/PoseStamped, geometry_msgs/TransformStamped, geometry_msgs/PoseWithCovarianceStamped, geometry_msgs/PointStamped or nav_msgs/Odometry topics or TF messages

See here for more infos about the formats.

Why?

evo has several advantages over other public benchmarking tools:

common tools for different formats
algorithmic options for association, alignment, scale adjustment for monocular SLAM etc.
flexible options for output, plotting, visualization or export (e.g. LaTeX plots or Excel tables)
a powerful, configurable CLI that can cover many use cases
modular core and tools libraries for custom extensions
faster than other established Python-based tools (see here)

What it's not: a 1-to-1 re-implementation of a particular evaluation protocol tailored to a specific dataset.

Installation / Upgrade

Installation is easy-peasy if you're familiar with this: https://xkcd.com/1987/#

The latest version of evo supports Python 3.10+.

Please use pip only in a proper environment (e.g. uv/uvx, pipx, venv, Pixi or a classic virtual environment).

⚠️ ☠️☠️☠️ ⚠️

Whatever you do, don't use: pip install --user / anything with --break-system-packages / or worst: sudo pip install

From PyPi

If you just want to use the executables of the latest release version, the easiest way is to run:

pip install evo

This will download the package and its dependencies from PyPI and install or upgrade them. If you want, you can subscribe to new releases via https://libraries.io/pypi/evo.

To upgrade to a newer version: pip install --upgrade evo

From Source

Run this in the repository's base folder:

pip install --editable .

Pixi

Pixi is an easy-to-use alternative if you want to have an isolated development environment. Run pixi add evo if you have an existing environment, or clone this repository and run pixi shell to open a dev shell.

Tab completion

Tab completion is supported via the argcomplete package. Run activate-global-python-argcomplete after the installation to use it.

Dependencies

Python packages

evo has some required dependencies that are automatically resolved during installation with pip. See the pyproject.toml file for all details.

PyQt6 (optional)

PyQt6 will give you the enhanced GUI for plot figures from the "qtagg" matplotlib backend (otherwise: "TkAgg"). If PyQt6 is already installed when installing this package, it will be used as a default (see evo_config show). To change the plot backend afterwards, run evo_config set plot_backend qtagg.

If you run into issues with installing tkinter, trying PyQt6 is a good idea.

ROS (optional)

Some ROS-related features require a ROS installation, see here. We are testing this package with ROS Lyrical.

Reading ROS bag files works also without a ROS installation thanks to the great rosbags package that is installed together with evo. This allows you also to read ROS 1 & 2 bags even if you don't have one of those ROS distros installed. (except for reading /tf topics, because there we need the buffer implementation from ROS)

contextily (optional)

contextily is required for adding map tiles to plots of geo-referenced data.

Rerun (optional)

You can send data also to the rerun viewer. Just pip install rerun-sdk and add --rerun to your command. See the related Wiki page for more details.

Command Line Interface

After installation with pip, the following executables can be called globally from your command-line:

Metrics:

evo_ape - absolute pose error
evo_rpe - relative pose error

Tools:

evo_traj - tool for analyzing, plotting or exporting one or more trajectories
evo_res - tool for comparing one or multiple result files from evo_ape or evo_rpe
evo_config - tool for global settings and config file manipulation

Call the commands with --help to see the options, e.g. evo_ape --help. Tab-completion of command line parameters is available on UNIX-like systems.

More documentation Check out the Wiki on GitHub.

Example Workflow

There are some example trajectories in the source folder in test/data.

1.) Plot multiple trajectories

Here, we plot two KITTI pose files and the ground truth using evo_traj:

cd test/data
evo_traj kitti KITTI_00_ORB.txt KITTI_00_SPTAM.txt --ref=KITTI_00_gt.txt -p --plot_mode=xz

2.) Run a metric on trajectories

For example, here we calculate the absolute pose error for two trajectories from ORB-SLAM and S-PTAM using evo_ape (KITTI_00_gt.txt is the reference (ground truth)) and plot and save the individual results to .zip files for evo_res:

First trajectory (ORB Stereo):

mkdir results
evo_ape kitti KITTI_00_gt.txt KITTI_00_ORB.txt -va --plot --plot_mode xz --save_results results/ORB.zip

Second trajectory (S-PTAM):

evo_ape kitti KITTI_00_gt.txt KITTI_00_SPTAM.txt -va --plot --plot_mode xz --save_results results/SPTAM.zip

3.) Process multiple results from a metric

evo_res can be used to compare multiple result files from the metrics, i.e.:

print infos and statistics (default)
plot the results
save the statistics in a table

Here, we use the results from above to generate a plot and a table:

evo_res results/*.zip -p --save_table results/table.csv

IPython / Jupyter Resources

For an interactive source code documentation, open the Jupyter notebook metrics_tutorial.ipynb in the notebooks folder of the repository. More infos on Jupyter notebooks: see here

If you have IPython installed, you can launch an IPython shell with a custom evo profile with the command evo_ipython.

Contributing Utilities

A few "inoffical" scripts for special use-cases are collected in the contrib/ directory of the repository. They are inofficial in the sense that they don't ship with the package distribution and thus aren't regularly tested in continuous integration.

Trouble

"😱, this piece of 💩 software doesn't do what I want!!1!1!!"

First aid:

append -h/ --help to your command
check the Wiki
check the previous issues
open a new issue

Contributing

Patches are welcome, preferably as pull requests.

License

GPL-3.0 or later

If you use this package for your research, a footnote with the link to this repository is appreciated: github.com/MichaelGrupp/evo.

...or, for citation with BibTeX:

@misc{grupp2017evo,
  title={evo: Python package for the evaluation of odometry and SLAM.},
  author={Grupp, Michael},
  howpublished={\url{https://github.com/MichaelGrupp/evo}},
  year={2017}
}

Frequently asked questions

Is evo free to use?

evo is open source under the GPL-3.0 licence. There is no licence fee and no seat count — you can self-host it or, where the project offers one, pay a vendor for a managed version instead.

What does evo do?

Python package for the evaluation of odometry and SLAM

What is evo written in?

evo is primarily written in Python. Its source is publicly available at https://github.com/MichaelGrupp/evo, and it has 4,354 GitHub stars.