[opensourcedex]

hypertools is a free, open source business intelligence & reporting project written in Python and released under MIT. It has 1,887 GitHub stars, 164 forks and 8 open issues, and was last pushed 6 days ago. On this registry it ranks #51 of 77 tracked projects in Business Intelligence & Reporting, with 5 head-to-head comparisons available.

What is hypertools?

HyperTools is an MIT-licensed Python toolbox for gaining geometric insights into high-dimensional data, built for data scientists, researchers, and analysts who need to see the structure of datasets they cannot plot directly.

What it is

HyperTools packages a dimensionality-reduction pipeline into a single call. A user feeds in a high-dimensional dataset, or a series of high-dimensional datasets, and the library reduces the dimensionality and produces a plot in one function call. It is built atop matplotlib, scikit-learn, and seaborn, so it inherits familiar plotting and modeling behaviour rather than replacing it. Version 1.1 makes hierarchical MultiIndex DataFrames a first-class input and folds into the library code that the project's own examples previously wrote by hand.

The problem it solves is the gap between a high-dimensional matrix and any human-readable picture of it. Instead of manually chaining a scaler, a reducer such as PCA or UMAP, and a plotting layer, a user calls hyp.plot and gets the reduced figure back. The same pipeline covers comparison work: several datasets, or several reducers, can be drawn as panels with a shared or independent pipeline fit. It replaces the per-project glue code that otherwise accumulates in notebooks whenever high-dimensional data must be reduced and visualized.

Key capabilities

  • hyp.plot([a, b, c], panels=True, title=[...]) draws one panel per dataset, or one per reducer via reduce=['PCA', 'UMAP'], with a shared or independent pipeline fit.
  • hyp.load() generates seeded example data ('random_walk', 'helix', 'lorenz', 'blobs') and fetches live data through hyp.load('wikipedia: '), hyp.load('yahoo: ', start=, end=) and hyp.load('sec: '), with cache=True and offline=True keeping URL downloads on disk.
  • hyp.predict(x, model=['Kalman', 'ARIMA', 'GP'], holdout=30) scores each model against a naive baseline, and hyp.plot(x, predict=[...], truth=held_out) overlays the forecasts on the actual continuation.
  • Input helpers include hyp.text_windows for sliding text windows, hyp.damage for reproducible missing data, hyp.stack for hierarchical frames, Smooth(center=False) for trailing means, the Delay embedding, and hyp.align(..., return_score=True).
  • The animation API exposes ctx.progress, populated reveal counts in on_frame callbacks, and titles from a callable or a '{index:%B %Y}' date pattern, alongside dataset_fade=, companion=, loop=True, title_kwargs= and title_wrap=.
  • Optional interactive plotting runs through hyp.plot(..., backend='plotly'); the default backend='auto' selects plotly on Google Colab and Kaggle and matplotlib everywhere else, with matched colors, line and marker styles and sizes, format strings, and the signature cube and square framing.
  • Hierarchical MultiIndex DataFrames are accepted as a first-class input.

Who uses it and how

  • Kaggle kernel users: the package was featured in 2017 on Kaggle's now-retired "No Free Hunch" blog (archived copy), and backend='auto' still selects plotly there.
  • Google Colab notebook users get the same automatic plotly selection, since plotly is preinstalled in that environment.
  • Researchers and students: the project's general overview talk was given as part of the MIND Summer School at Dartmouth.
  • Time-series and market analysts: the Yahoo and SEC loaders feed hyp.predict with the Kalman, ARIMA and GP models, compared against a naive baseline.
  • Text and topic-modeling work: the Wikipedia loader, hyp.text_windows, and the project's topic-modeling and text-vectorization topics.

Getting started

Install the package from PyPI as hypertools and follow the documentation at hypertools.readthedocs.io; optional dependencies install themselves on demand, and the optional dependencies guide explains how.

How it compares

The facts name no paid products that HyperTools replaces, and no directly comparable tool, so it stands alone in this registry on that axis. It sits atop matplotlib, scikit-learn and seaborn rather than competing with them, adding the reduction-and-plot pipeline those libraries leave to the user to assemble.

When to use it — and when not to

There is no server to operate here: no database, storage layer or SMTP service is required, and the library runs wherever Python, matplotlib and scikit-learn already run. It is the wrong choice for interactive dashboards or scheduled reporting, because it produces figures rather than a hosted business-intelligence surface, and the interactive plotly backend is an optional extra rather than a default. The main caveat is scope: optional features carry their own setup, the animation API brings callbacks and title patterns that a casual user will not need, and the README is dense with version-specific detail rather than a short quickstart.

project readme (upstream, from github) — read inline

Hypertools logo

Tests Documentation Status PyPI version

"To deal with hyper-planes in a 14 dimensional space, visualize a 3D space and say 'fourteen' very loudly. Everyone does it." - Geoff Hinton

Hypertools example

Overview

HyperTools is designed to facilitate dimensionality reduction-based visual explorations of high-dimensional data. The basic pipeline is to feed in a high-dimensional dataset (or a series of high-dimensional datasets) and, in a single function call, reduce the dimensionality of the dataset(s) and create a plot. The package is built atop many familiar friends, including matplotlib, scikit-learn and seaborn. Our package was featured in 2017 on Kaggle's now-retired "No Free Hunch" blog (archived copy). For a general overview, you may find this talk useful (given as part of the MIND Summer School at Dartmouth).

What's new in 1.1

HyperTools 1.1 makes hierarchical (MultiIndex) DataFrames a first-class input, grows the animation API, installs optional extras on demand, and folds into the library the code its own examples used to write by hand:

  • One call, many panels: hyp.plot([a, b, c], panels=True, title=[...]) draws one panel per dataset (or reduce=['PCA', 'UMAP'] one per reducer) with a shared or independent pipeline fit.
  • Data in one line: hyp.load('random_walk' | 'helix' | 'lorenz' | 'blobs' | ...) generates seeded example data; hyp.load('wikipedia:'), hyp.load('yahoo:', start=, end=) and hyp.load('sec:') fetch live text and market data; cache=True / offline=True keep URL downloads on disk.
  • Forecast comparisons: hyp.predict(x, model=['Kalman', 'ARIMA', 'GP'], holdout=30) scores every model against a naive baseline; hyp.plot(x, predict=[...], truth=held_out) overlays them with the actual continuation.
  • Helpers for the data you feed in: hyp.text_windows (sliding text windows), hyp.damage (reproducible missing data), hyp.stack (hierarchical frames), Smooth(center=False) (trailing means), the Delay embedding, and hyp.align(..., return_score=True).
  • Animations that know where they are: ctx.progress and populated reveal counts in on_frame callbacks, titles from a callable or a '{index:%B %Y}' date pattern, dataset_fade=, companion= panels, loop=True morphs, and title_kwargs= / title_wrap= styling.

See the changelog for the complete list, and the optional dependencies guide for how extras install themselves.

What's new in 1.0

HyperTools 1.0 modernizes the toolbox while keeping the familiar API:

  • Interactive plotting (optional): hyp.plot(..., backend='plotly') renders interactive figures. With the default backend='auto', HyperTools automatically uses plotly on Google Colab and Kaggle (where interactive figures work best and plotly is preinstalled) and matplotlib everywhere else — existing workflows are unchanged. The two backends produce visually matched output: identical colors, line/marker styles and sizes, format strings, and the signature cube/square framing.
  • Multicolored lines: passing continuous values (or a per-observation matrix) as hue together with a line format colors each trajectory continuously along its length, on both backends.
  • hyp.apply_model: a unified stack → fit-once → unstack core for applying any scikit-learn style model (or pipeline of models) across one or more datasets, with return_model=True for reuse on held-out data.
  • Mixture-model ("soft") clustering: cluster and plot support GaussianMixture, BayesianGaussianMixture, LatentDirichletAllocation, and NMF. hyp.cluster returns per-observation membership proportions, and hyp.plot colors each observation by blending component colors according to its mixture weights.
  • Richer coloring: the hue argument now accepts categorical labels, continuous values, or entire matrices (e.g. mixture proportions or model weights), which are mapped to colors via the new mat2colors helper (from hypertools.plot.colors import mat2colors).
  • Nested-list input: hyp.plot([[a, b], [c]]) colors datasets by their outermost grouping and renders more deeply nested datasets with thinner, fainter lines.
  • Hull surfaces (optional): hyp.plot(..., surface=True) overlays a smooth, lit surface over each dataset's convex hull — a filled outline in 2D, or a shaded, Taubin-smoothed "blob" in 3D. A dict of scalar options (e.g. surface={'alpha': 0.6}) customizes all surfaces at once; a list of bools/dicts (e.g. surface=[{'alpha': 0.2}, {'alpha': 0.8}]) controls alpha/color/lighting/smoothing per dataset.
  • Morph animation: hyp.plot(datasets, animate='morph') treats each dataset as a point cloud and morphs smoothly between them (Hungarian- matched, smoothstep-eased), holding on each one along the way; rotations accepts a per-segment list for independent camera control over each hold and transition, and surface=True composes with it to morph a lit hull instead of raw points. See examples/animate_morph_zoo.py and examples/animate_surface_morph.py.
  • More animation styles, and 2-D animation: in addition to True/ 'parallel', animate='spin' keeps all the data drawn and spins the camera, 'serial' reveals each dataset one at a time in list order, and 'window' (hyp.plot(traj, animate='window', focused=2)) slides a fixed- length, fully-opaque window along each trajectory. Every style except 'spin' (which is inherently a 3-D camera move) now also works for ndims=2 data, using a fixed (non-rotating) viewport.
  • hyp.Pipeline: a scikit-learn-style Pipeline chains manip/normalize/reduce/align/cluster stages, fit once and reused on new data without refitting -- hyp.plot(A, ..., return_model=True) returns a bundle whose 'pipeline' entry can be passed back in as hyp.plot(B, pipeline=bundle['pipeline']) to apply the exact same fitted transformation to a structurally-identical dataset B.
  • hyp.manip and manip chaining: hyp.manip(data, model='ZScore') applies a manipulation (Normalize, ZScore, Smooth, Resample) to each dataset. Smooth and Resample run independently per dataset (kernels never cross dataset boundaries); ZScore and Normalize also transform each dataset separately but fit one shared set of statistics across all datasets in a list (like normalize='across'). A list of specs chains several manipulations as a Pipeline, and hyp.plot(data, manip=[...]) runs the chain at the canonical manip stage -- first, before normalize/reduce/align/cluster -- e.g. hyp.plot(data, manip=[{'model': 'Smooth', 'kwargs': {'kernel_width': 5}}, 'ZScore'], reduce='PCA').
  • Autoencoder reducers (optional): hyp.reduce/hyp.plot(..., reduce= 'Autoencoder') supports six torch-backed autoencoder reducers (Autoencoder, DeepAutoencoder, SparseAutoencoder, ConvolutionalAutoencoder, SequenceAutoencoder, VariationalAutoencoder), from the [torch] extra (installed on demand the first time one is fit).
  • gensim text vectorizers/semantic models (optional): hyp.plot(texts, vectorizer='Word2Vec', semantic=None, reduce='PCA') (and hyp.tools.text2mat) add Word2Vec/Doc2Vec/FastText vectorizers and LdaModel/LsiModel/HdpModel semantic models, from the [gensim] extra (installed on demand) -- with an embedding vectorizer (gensim, or a Hugging Face model id like 'all-MiniLM-L6-v2') the default LDA semantic stage is skipped automatically (gensim warns; pass semantic=None to silence it) and corpus is unused. Note that non-default vectorizers train on the default corpus='wiki' corpus the first time, which can take a couple of minutes even for tiny inputs; pas

readme truncated — read the full docs on github

Frequently asked questions

Is hypertools free to use?

hypertools is open source under the MIT licence. There is no licence fee and no seat count — you can self-host it or, where the project offers one, pay a vendor for a managed version instead.

What does hypertools do?

A Python toolbox for gaining geometric insights into high-dimensional data

What is hypertools written in?

hypertools is primarily written in Python. Its source is publicly available at https://github.com/ContextLab/hypertools, and it has 1,887 GitHub stars.