[opensourcedex]

popmon is a free, open source machine learning infrastructure project written in Python and released under MIT. It has 512 GitHub stars, 36 forks and 12 open issues, and was last pushed 10 months ago. On this registry it ranks #86 of 89 tracked projects in Machine Learning Infrastructure, with 5 head-to-head comparisons available. It gained 1 stars over the last 30 tracked days.

What is popmon?

popmon is an open-source Python library from ING Bank that monitors the stability of pandas and Apache Spark dataframes by binning features into time-sliced histograms and flagging distribution shifts with statistical tests, built for data scientists, analysts, and MLOps engineers who need to detect drift over time or against a reference dataset.

What it is

popmon stands for Population Shift Monitoring. It is a Python package, distributed under the MIT licence, that checks the stability of a dataset by creating histograms of features binned in time-slices and then comparing the profiles and distributions of those histograms using statistical tests, both over time and with respect to a reference. It handles numerical, ordinal, and categorical features, and the histograms can be higher-dimensional, which means it can also track correlations between any two features. The library lives in the AI and Machine Learning / Machine Learning Infrastructure category and carries topics such as covariate-shift, dataset-shifts, drift-detection, data-profiling, and mlops.

The concrete problem it solves is manual, ad-hoc stability checking. Instead of hand-writing comparisons between a reference cohort and each new time slice, popmon performs the binning, the statistical comparison, and the flagging in one pass, then reports the result. It replaces the repeated notebook scripts that teams otherwise maintain to inspect whether a feature's distribution has drifted. Monitoring business rules are applied automatically, so the output is a set of flags and alerts rather than raw numbers that someone must interpret.

Key capabilities

  • Works with both pandas and Apache Spark dataframes; importing popmon adds the pm_stability_report method directly to a dataframe.
  • Generates stability reports over a chosen time axis, for example df.pm_stability_report(time_axis="date", features=["date:age", "date:gender"]).
  • Bins features automatically by default, and accepts custom bin_specs with explicit bin_width and bin_offset per feature, including open-ended sparse histograms.
  • Supports time-axis control through time_width and time_offset, with time binning expressed in nanoseconds when specified explicitly.
  • Tracks higher-dimensional histograms such as date:isActive:age, which lets it monitor correlations between feature pairs.
  • Automatically flags changes observed over time, including trends, shifts, peaks, outliers, anomalies, and changing correlations, using monitoring business rules.
  • Renders reports inline in Jupyter notebooks and saves them to disk with report.to_file("monitoring_report.html").

Who uses it and how

  • Data scientists working in Jupyter or IPython notebooks, where the four supplied tutorials cover the basic workflow, advanced configuration with Apache Spark, incremental datasets for online analysis, and step-by-step report interpretation.
  • MLOps teams monitoring production model inputs for covariate shift and dataset shift, using the flagged changes as an early warning that a model's input distribution has moved.
  • Spark users running the library at scale, who configure the session with spark.jars.packages pointing at the matching histogrammar and histogrammar-sparksql artifacts.
  • Teams analysing incremental or streaming-style datasets, supported by the dedicated online analysis notebook.
  • Analysts validating the tool against known data, using the bundled synthetic dataset through popmon.resources.data("test.csv.gz") or the Flight Delays and Cancellations Kaggle dataset referenced in the documentation.

Getting started

Install with pip install popmon, which requires Python 3.6 or newer. Alternatively, clone https://github.com/ing-bank/popmon.git and install in editable mode with pip install -e popmon.

How it compares

The facts provided do not name any comparable or competing tools, so popmon stands alone in this registry. No paid products it replaces are listed either.

When to use it — and when not to

A self-hoster must provide a Python 3.6 or newer environment and, for Spark workloads, the correct histogrammar JAR files, matching Scala 2.13 for Spark 4.x or Scala 2.12 for Spark 3.x — a version mismatch here is an operational failure waiting to happen. Anyone who needs a hosted service, or who analyses data outside pandas and Spark, should not pick this tool. The documented interface is a Python library and an HTML report file, so teams expecting built-in alert delivery or a managed dashboard should look elsewhere.

project readme (upstream, from github) — read inline

=========================== Population Shift Monitoring

|build| |docs| |release| |release_date| |downloads| |ruff|

|logo|

popmon is a package that allows one to check the stability of a dataset. popmon works with both pandas and spark datasets.

popmon creates histograms of features binned in time-slices, and compares the stability of the profiles_ and distributions of those histograms using statistical tests _, both over time and with respect to a reference. It works with numerical, ordinal, categorical features, and the histograms can be higher-dimensional, e.g. it can also track correlations between any two features. popmon can automatically flag and alert on changes observed over time, such as trends, shifts, peaks, outliers, anomalies, changing correlations, etc, using monitoring business rules.

Latest update: Jan 2026.

|example|

|histograms|

Spark

For Spark make sure to pick up the correct histogrammar jar files. Spark 4.X is based on Scala 2.13; Spark 3.X is based on Scala 2.12 or 2.13.

.. code-block:: python

spark = SparkSession.builder.config( "spark.jars.packages", "io.github.histogrammar:histogrammar_2.13:1.0.30,io.github.histogrammar:histogrammar-sparksql_2.13:1.0.30", ).getOrCreate()

For Scala 2.12, in the string above simply replace "2.13" with "2.12" (twice).

Examples

  • Flight Delays and Cancellations Kaggle data _
  • Synthetic data (code example below) _

Documentation

The entire popmon documentation including tutorials can be found at read-the-docs _.

Notebooks

.. list-table:: :widths: 80 20 :header-rows: 1

    • Tutorial
    • Colab link
    • Basic tutorial _
    • |notebook_basic_colab|
    • Detailed example (featuring configuration, Apache Spark and more) _
    • |notebook_advanced_colab|
    • Incremental datasets (online analysis) _
    • |notebook_incremental_data_colab|
    • Report interpretation (step-by-step guide) _
    • |notebook_reports_colab|

Check it out

The popmon library requires Python 3.6+ and is pip friendly. To get started, simply do:

.. code-block:: bash

$ pip install popmon

or check out the code from our GitHub repository:

.. code-block:: bash

$ git clone https://github.com/ing-bank/popmon.git $ pip install -e popmon

where in this example the code is installed in edit mode (option -e).

You can now use the package in Python with:

.. code-block:: python

import popmon

Congratulations, you are now ready to use the popmon library!

Quick run

As a quick example, you can do:

.. code-block:: python

import pandas as pd import popmon from popmon import resources

open synthetic data

df = pd.read_csv(resources.data("test.csv.gz"), parse_dates=["date"]) df.head()

generate stability report using automatic binning of all encountered features

(importing popmon automatically adds this functionality to a dataframe)

report = df.pm_stability_report(time_axis="date", features=["date:age", "date:gender"])

to show the output of the report in a Jupyter notebook you can simply run:

report

or save the report to file

report.to_file("monitoring_report.html")

To specify your own binning specifications and features you want to report on, you do:

.. code-block:: python

time-axis specifications alone; all other features are auto-binned.

report = df.pm_stability_report( time_axis="date", time_, time_offset="2020-1-6" )

histogram selections. Here 'date' is the first axis of each histogram.

features = [ "date:isActive", "date:age", "date:eyeColor", "date:gender", "date:latitude", "date:longitude", "date:isActive:age", ]

Specify your own binning specifications for individual features or combinations thereof.

This bin specification uses open-ended ("sparse") histograms; unspecified features get

auto-binned. The time-axis binning, when specified here, needs to be in nanoseconds.

bin_specs = { "longitude": {"bin_width": 5.0, "bin_offset": 0.0}, "latitude": {"bin_width": 5.0, "bin_offset": 0.0}, "age": {"bin_width": 10.0, "bin_offset": 0.0}, "date": { "bin_width": pd.Timedelta("4w").value, "bin_offset": pd.Timestamp("2015-1-1").value, }, }

generate stability report

report = df.pm_stability_report(features=features, bin_specs=bin_specs, time_axis=True)

These examples also work with spark dataframes. You can see the output of such example notebook code here . For all available examples, please see the tutorials at read-the-docs.

Pipelines for monitoring dataset shift

Advanced users can leverage popmon's modular data pipeline to customize their workflow. Visualization of the pipeline can be useful when debugging, or for didactic purposes. There is a script _ included with the package that you can use. The plotting is configurable, and depending on the options you will obtain a result that can be used for understanding the data flow, the high-level components and the (re)use of datasets.

|pipeline|

Example pipeline visualization (click to enlarge)

Reports and integrations

The data shift computations that popmon performs, are by default displayed in a self-contained HTML report. This format is favourable in many real-world environments, where access may be restricted. Moreover, reports can be easily shared with others.

Access to the datastore means that its possible to integrate popmon in almost any workflow. To give an example, one could store the histogram data in a PostgreSQL database and load that from Grafana and benefit from their visualisation and alert handling features (e.g. send an email or slack message upon alert). This may be interesting to teams that are already invested in particular choice of dashboarding tool.

Possible integrations are:

+----------------+---------------+ | |grafana_logo| | |kibana_logo| | +----------------+---------------+ | Grafana | Kibana | +----------------+---------------+

Resources on how to integrate popmon are available in the examples directory _. Contributions of additional or improved integrations are welcome!

.. |grafana_logo| image:: https://upload.wikimedia.org/wikipedia/commons/a/a1/Grafana_logo.svg :alt: Grafana logo :height: 120 :target: https://github.com/grafana/grafana

.. |kibana_logo| image:: https://miro.medium.com/max/1400/1*HW_x9ZvIbUkyaqHstsB1ig.png :alt: Kibana logo :height: 120 :target: https://github.com/elastic/kibana

Comparison and profile extensions

External libraries or custom functionality can be easily added to Profiles_ and Comparisons_. If you developed an extension that could be generically used, then please consider contributing it to the package.

Popmon currently integrates:

  • Diptest _

A Python/C++ implementation of Hartigan & Hartigan's dip test for unimodality. The dip test tests for multimodality in a sample by taking the maximum difference, over all sample points, between the empirical distribution function, and the unimodal distribution function that minimizes that maximum difference. Other than unimodality, it makes no further assumptions about the form of the null distribution.

To enable this extension install diptest using pip install diptest or pip install popmon[diptest].

Resources

Presentations

+------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------+-------------------+-------------------------+ | Title | Host | Date | Speaker | +------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------+-------------------+-------------------------+ | popmon: Analysis Package for Dataset Shift Detection | SciPy Conference 2022 _ | July 13, 2022 | Simon Brugman | +------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------+-------------------+-------------------------+ | Popmon - population monitoring made easy | Big Data Technology Warsaw Summit 2021 _ | February 25, 2021 | Simon Brugman | +------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------+-------------------+-------------------------+ | Popmon - population monitoring made easy | Data Lunch @ Eneco _ | October 29, 2020 | Max Baak, Simon Brugman | +------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------+-------------------+-------------------------+ | Popmon - population monitoring made easy | Data Science Summit 2020 _ | October 16, 2020 | Max Baak | +------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------+-------------------+-------------------------+ | Population Shift Monitoring Made Easy: the popmon package _ | Online Data Science Meetup @ ING WBAA _ | July 8 2020 | Tomas Sostak | +------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------+-------------------+-------------------------+ | Popmon: Population Shift Monitoring Made Easy _ | PyData Fest Amsterdam 2020 _ | June 16, 2020 | Tomas Sostak | +------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------+-------------------+-------------------------+ | Popmon: Population Shift Monitoring Made Easy | Amundsen Community Meetup _ | June 4, 2020 | Max Baak | +------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------+-------------------+-------------------------+

Articles

+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+---------------------------------------------+ | Title | Date | Author | +---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+---------------------------------------------+ |POPMON v1.0.0: The Dataset-Shift Pokémon _ | Aug 3, 2022 | Pradyot Patil | +---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+---------------------------------------------+ |Monitoring Model Drift with Python _ | April 16, 2022 | Jeanine Schoonemann | +---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+---------------------------------------------+ |The Statistics Underlying the Popmon Hood _ | April 15, 2022 | Jurriaan Nagelkerke and Jeanine Schoonemann | +---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+---------------------------------------------+ |popmon: code breakfast session _ | November 9, 2022 | Simon Brugman |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+---------------------------------------------+ | Population Shift Analysis: Monitoring Data Quality with Popmon _ | May 21, 2021 | Vito Gentile | +---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+---------------------------------------------+ | Popmon Open Source Package — Population Shift Monitoring Made Easy _ | May 20, 2020 | Nicole Mpozika | +---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+---------------------------------------------+

Software

  • Kedro-popmon _ is a plugin to integrate popmon reporting with kedro. This plugin allows you to automate the process of popmon feature and output stability monitoring. Package created by Marian Dabrowski _ and Stephane Collot _.

Project contributors

This package was authored by ING Analytics Wholesale Banking (INGA WB). Special thanks to the following people who have contributed to the development of this package: Ahmet Erdem , Fabian Jansen , Nanne Aben _, Mathieu Grimal.

Citing popmon

If popmon has been relevant in your work, and you would like to acknowledge the project in your publication, we suggest citing the following paper:

  • Brugman, S., Sostak, T., Patil, P., Baak, M. popmon: Analysis Package for Dataset Shift Detection. Proceedings of the 21st Python in Science Conference. 161-168 (2022). (link _)

In BibTeX format:

.. code-block:: bibtex

@InProceedings{ popmon-proc-scipy-2022,
  author    = { {S}imon {B}rugman and {T}omas {S}ostak and {P}radyot {P}atil and {M}ax {B}aak },
  title     = { popmon: {A}nalysis {P}ackage for {D}ataset {S}hift {D}etection },
  booktitle = { {P}roceedings of the 21st {P}ython in {S}cience {C}onference },
  pages     = { 161 - 168 },
  year      = { 2022 },
  editor    = { {M}eghann {A}garwal and {C}hris {C}alloway and {D}illon {N}iederhut and {D}avid {S}hupe },
}

Contact and support

Please note that INGA WB provides support only on a best-effort basis.

License

Copyright INGA WB. popmon is completely free, open-source and licensed under the MIT license _.

.. |logo| image:: https://raw.githubusercontent.com/ing-bank/popmon/master/docs/source/assets/popmon-logo.png :alt: POPMON logo :target: https://github.com/ing-bank/popmon .. |example| image:: https://raw.githubusercontent.com/ing-bank/popmon/master/docs/source/assets/report_overview.png :alt: Traffic Light Overview .. |histograms| image:: https://raw.githubusercontent.com/ing-bank/popmon/master/docs/source/assets/histogram_inspector.png :alt: Histogram inspector .. |pipeline| image:: https://raw.githubusercontent.com/ing-bank/popmon/master/docs/source/assets/pipeline.png :alt: Pipeline Visualization :target: https://github.com/ing-bank/popmon/files/7417124/pipeline_amazingpipeline_subgraphs_unversioned.pdf .. |build| image:: https://github.com/ing-bank/popmon/workflows/build/badge.svg :alt: Build status .. |ruff| image:: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v1.json :alt: Ruff :target: https://github.com/charliermarsh/ruff .. |docs| image:: https://readthedocs.org/projects/popmon/badge/?version=latest :alt: Package docs status :target: https://popmon.readthedocs.io .. |release| image:: https://img.shields.io/github/v/release/ing-bank/popmon :alt: Latest GitHub release :target: https://github.com/ing-bank/popmon/releases .. |release_date| image:: https://img.shields.io/github/release-date/ing-bank/popmon :alt: GitHub Release Date :target: https://github.com/ing-bank/popmon/releases

.. |notebook_basic_colab| image:: https://colab.research.google.com/assets/colab-badge.svg :alt: Open in Colab :target: https://colab.research.google.com/github/ing-bank/popmon/blob/master/popmon/notebooks/popmon_tutorial_basic.ipynb .. |notebook_advanced_colab| image:: https://colab.research.google.com/assets/colab-badge.svg :alt: Open in Colab :target: https://colab.research.google.com/github/ing-bank/popmon/blob/master/popmon/notebooks/popmon_tutorial_advanced.ipynb .. |notebook_incremental_data_colab| image:: https://colab.research.google.com/assets/colab-badge.svg :alt: Open in Colab :target: https://colab.research.google.com/github/ing-bank/popmon/blob/master/popmon/notebooks/popmon_tutorial_incremental_data.ipynb .. |notebook_reports_colab| image:: https://colab.research.google.com/assets/colab-badge.svg :alt: Open in Colab :target: https://colab.research.google.com/github/ing-bank/popmon/blob/master/popmon/notebooks/popmon_tutorial_reports.ipynb .. |downloads| image:: https://pepy.tech/badge/popmon :alt: PyPi downloads :target: https://pepy.tech/project/popmon

.. _profiles: https://popmon.readthedocs.io/en/latest/profiles.html .. _comparisons: https://popmon.readthedocs.io/en/latest/comparisons.html

Frequently asked questions

Is popmon free to use?

popmon is open source under the MIT licence. There is no licence fee and no seat count — you can self-host it or, where the project offers one, pay a vendor for a managed version instead.

What does popmon do?

Monitor the stability of a Pandas or Spark dataframe ⚙︎

What is popmon written in?

popmon is primarily written in Python. Its source is publicly available at https://github.com/ing-bank/popmon, and it has 512 GitHub stars.