[opensourcedex]

pymatting is a free, open source photo & video editors project written in Python and released under MIT. It has 1,921 GitHub stars, 227 forks and 11 open issues, and was last pushed 1 months ago. On this registry it ranks #52 of 70 tracked projects in Photo & Video Editors, with 5 head-to-head comparisons available.

What is pymatting?

PyMatting is a Python library for alpha matting that lets developers and researchers estimate a foreground object's alpha channel from an image and a hand-drawn trimap.

What it is

PyMatting is an MIT-licensed Python package, published on PyPI as pymatting, that implements a range of algorithms for solving the alpha matting problem: given an input image and a trimap, it estimates the alpha channel of a foreground object so that object can be composed onto a different background. It is built on top of NumPy, SciPy and Numba, ships a pymatting CLI tool alongside its Python API (including a cutout() convenience function), and is documented at pymatting.github.io with published benchmarks.

The concrete problem it addresses is the extraction of a foreground object with correct soft edges, hair and fine detail — something a hard binary mask cannot express. All implemented methods rely on trimaps, which are expected to be numpy.ndarrays of type np.float64 with the same shape as the input image and a single color channel, where values of 0.0 denote pixels that are 100% background, 1.0 denote pixels that are 100% foreground, and all other values indicate unknown pixels the algorithm will estimate.

Key capabilities

  • Six alpha matting implementations: Closed Form Alpha Matting, Large Kernel Matting, KNN Matting, Learning Based Digital Matting, Random Walk Matting, and Shared Sampling Matting.
  • Two foreground estimation implementations: Closed Form Foreground Estimation and Fast Multi-Level Foreground Estimation, the latter available for CPU, CUDA and OpenCL.
  • Fast multithreaded KNN search.
  • Preconditioners to accelerate the convergence rate of conjugate gradient descent: the incomplete thresholded Cholesky decomposition and the V-Cycle Geometric Multigrid preconditioner.
  • GPU support for foreground estimation through CuPy and PyOpenCL, installed via pip3 install -r .[gpu].
  • A command-line interface, for example pymatting data/lemur/lemur.png data/lemur/lemur_trimap.png lemur_cutout.png.
  • Test suite runnable with pytest, currently covering 89% of the code.

Who uses it and how

  • Developers who need programmatic cutouts: calling pymatting.cutout() with an input image path, a trimap path, and an output cutout path produces a composited result.
  • Users working from the shell, passing image, trimap and output paths directly to the pymatting CLI.
  • Practitioners with NVIDIA or OpenCL hardware who install the [gpu] extra to run Fast Multi-Level Foreground Estimation on CUDA or OpenCL, provided they install the proper drivers separately.
  • Contributors and CI users running the test suite with pytest after pip3 install -r .[test], and upgrading with pip3 install --upgrade pymatting.
  • Researchers comparing methods against the published benchmarks at pymatting.github.io/benchmarks.html.

Getting started

Install from PyPI with pip3 install pymatting, or from source with git clone https://github.com/pymatting/pymatting, cd pymatting, pip3 install ..

How it compares

Among the topics and capabilities listed, PyMatting occupies the alpha-matting and image-processing niche in the Python ecosystem, pairing published academic methods with a package that installs from PyPI. Its position is reinforced by a JOSS paper badge and hosted benchmark tables, so it functions as both a library and a reference implementation for the trimap-based matting methods it contains.

When to use it — and when not to

A self-hoster must supply their own inputs and pipeline: a hand-drawn or otherwise produced trimap, and, for GPU acceleration, separately installed drivers plus CuPy or PyOpenCL — the README notes that CuPy may fail to find libnvrtc.so.12. The project is not a drop-in background-removal service; anyone who cannot produce a trimap, or who wants automatic segmentation without one, should look elsewhere. Worth noting as well are 11 open issues and a first import that takes about a minute due to compilation.

project readme (upstream, from github) — read inline

PyMatting: A Python Library for Alpha Matting

License: MIT CI PyPI JOSS Documentation

We introduce the PyMatting package for Python, which implements various methods to solve the alpha matting problem.

Lemur

Given an input image and a hand-drawn trimap (top row), alpha matting estimates the alpha channel of a foreground object which can then be composed onto a different background (bottom row).

PyMatting provides:

  • Alpha matting implementations for:
    • Closed Form Alpha Matting [1]
    • Large Kernel Matting [2]
    • KNN Matting [3]
    • Learning Based Digital Matting [4]
    • Random Walk Matting [5]
    • Shared Sampling Matting [6]
  • Foreground estimation implementations for:
    • Closed Form Foreground Estimation [1]
    • Fast Multi-Level Foreground Estimation (CPU, CUDA and OpenCL) [7]
  • Fast multithreaded KNN search
  • Preconditioners to accelerate the convergence rate of conjugate gradient descent:
    • The incomplete thresholded Cholesky decomposition (Incomplete is part of the name. The implementation is quite complete.)
    • The V-Cycle Geometric Multigrid preconditioner
  • Readable code leveraging NumPy, SciPy and Numba

Getting Started

Requirements

Minimal requirements

  • numpy>=1.16.0
  • pillow>=5.2.0
  • numba>=0.47.0
  • scipy>=1.1.0

Additional requirements for GPU support

  • cupy-cuda90>=6.5.0 or similar
  • pyopencl>=2019.1.2

Requirements to run the tests

  • pytest>=5.3.4

Installation with PyPI

pip3 install pymatting

Installation from Source

git clone https://github.com/pymatting/pymatting
cd pymatting
pip3 install .

Example

# First import will take a minute due to compilation
from pymatting import cutout

cutout(
    # input image path
    "data/lemur/lemur.png",
    # input trimap path
    "data/lemur/lemur_trimap.png",
    # output cutout path
    "lemur_cutout.png")

Alternatively, you can use pymatting CLI tool.

pymatting data/lemur/lemur.png data/lemur/lemur_trimap.png lemur_cutout.png

More advanced examples

Trimap Construction

All implemented methods rely on trimaps which roughly classify the image into foreground, background and unknown regions. Trimaps are expected to be numpy.ndarrays of type np.float64 having the same shape as the input image with only one color-channel. Trimap values of 0.0 denote pixels which are 100% background. Similarly, trimap values of 1.0 denote pixels which are 100% foreground. All other values indicate unknown pixels which will be estimated by the algorithm.

Testing

Run the tests from the main directory:

pip3 install -r .[test]
pytest

Currently 89% of the code is covered by tests.

Upgrade

pip3 install --upgrade pymatting
python3 -c "import pymatting"

GPU-Support

There is CuPy and PyOpenCL support for foreground estimation.

pip3 install -r .[gpu]
pytest tests/test_foreground.py

You still need to install proper drivers for GPU support separately.

If you have installed drivers, but CuPy can not find libnvrtc.so.12, you might have link to CUDA libraries manually:

sudo apt install plocate
# might resolve to something like /home/user/myenv/lib/python3.12/site-packages/nvidia/cuda_nvrtc/lib/
export LD_LIBRARY_PATH="$(dirname $(locate libnvrtc.so.12)):$LD_LIBRARY_PATH"

For PyOpenCL, see docs.
You can select computing device by setting PYOPENCL_CTX environment variable.

There currently is no GPU support for alpha estimation, only for foreground estimation.

Bug Reports, Questions and Pull-Requests

Please, see our community guidelines.

Authors

  • Thomas Germer
  • Tobias Uelwer
  • Stefan Conrad
  • Stefan Harmeling

See also the list of contributors who participated in this project.

Projects using PyMatting

License

This project is licensed under the MIT License - see the LICENSE.md file for details

Citing

If you found PyMatting to be useful for your work, please consider citing our paper:

@article{Germer2020,
  doi = {10.21105/joss.02481},
  url = {https://doi.org/10.21105/joss.02481},
  year = {2020},
  publisher = {The Open Journal},
  volume = {5},
  number = {54},
  pages = {2481},
  author = {Thomas Germer and Tobias Uelwer and Stefan Conrad and Stefan Harmeling},
  title = {PyMatting: A Python Library for Alpha Matting},
  journal = {Journal of Open Source Software}
}

References

[1] Anat Levin, Dani Lischinski, and Yair Weiss. A closed-form solution to natural image matting. IEEE transactions on pattern analysis and machine intelligence, 30(2):228–242, 2007.

[2] Kaiming He, Jian Sun, and Xiaoou Tang. Fast matting using large kernel matting laplacian matrices. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2165–2172. IEEE, 2010.

[3] Qifeng Chen, Dingzeyu Li, and Chi-Keung Tang. Knn matting. IEEE transactions on pattern analysis and machine intelligence, 35(9):2175–2188, 2013.

[4] Yuanjie Zheng and Chandra Kambhamettu. Learning based digital matting. In 2009 IEEE 12th international conference on computer vision, 889–896. IEEE, 2009.

[5] Leo Grady, Thomas Schiwietz, Shmuel Aharon, and Rüdiger Westermann. Random walks for interactive alpha-matting. In Proceedings of VIIP, volume 2005, 423–429. 2005.

[6] Eduardo S. L. Gastal and Manuel M. Oliveira. "Shared Sampling for Real-Time Alpha Matting". Computer Graphics Forum. Volume 29 (2010), Number 2, Proceedings of Eurographics 2010, pp. 575-584.

[7] Germer, T., Uelwer, T., Conrad, S., & Harmeling, S. (2020). Fast Multi-Level Foreground Estimation. arXiv preprint arXiv:2006.14970.

Lemur image by Mathias Appel from https://www.flickr.com/photos/mathiasappel/25419442300/ licensed under CC0 1.0 Universal (CC0 1.0) Public Domain License.

Frequently asked questions

Is pymatting free to use?

pymatting is open source under the MIT licence. There is no licence fee and no seat count — you can self-host it or, where the project offers one, pay a vendor for a managed version instead.

What does pymatting do?

A Python library for alpha matting

What is pymatting written in?

pymatting is primarily written in Python. Its source is publicly available at https://github.com/pymatting/pymatting, and it has 1,921 GitHub stars.