miasma is a free, open source data extraction & web scraping project written in Rust and released under GPL-3.0. It has 1,198 GitHub stars, 40 forks and 12 open issues, and was last pushed 8 hours ago. On this registry it ranks #99 of 138 tracked projects in Data Extraction & Web Scraping, with 5 head-to-head comparisons available.

What is miasma?

Miasma is a free-software anti-scraping tool for website operators who want to feed AI crawlers poisoned training data instead of merely blocking them.

What it is

Miasma is a Rust project distributed under the GPL-3.0 licence that sits in front of a public website and responds to AI web scrapers with poisoned training data and self-referential links. Rather than refusing a request or returning an error, it serves content designed to lead the crawler back into a loop of generated slop, so the machine consuming it keeps following its own tail. It occupies the Data & Analytics / Data Extraction & Web Scraping category, and its stated design goals are speed and a minimal memory footprint, on the principle that defending your own content should not impose a compute burden on you.

The concrete problem it addresses is the large-scale scraping of the web by AI companies to gather training data for their models. The project's premise is that anyone with a public website is already having that content taken, and that existing approaches are defensive in the wrong way: many tools attempt to block scrapers, but Miasma instead punishes them by returning degraded data alongside links that keep them trapped. The output is not a barrier for the site owner to maintain but a trap for the crawler to fall into.

Key capabilities

  • Returns poisoned training data to scrapers so that the models consuming it are degraded in quality.
  • Injects self-referential links that catch crawlers in a loop of generated content.
  • Ships as a Rust library, published as the miasma crate on crates.io and documented on docs.rs.
  • Ships as a Docker image published as austinweeks/miasma on Docker Hub.
  • Provides pre-built binaries through GitHub releases, plus packaging through a range of package managers.
  • Is implemented in Rust and advertises a minimal memory footprint and fast response handling.
  • Carries a No AI badge and is published as free software under GPL-3.0.

Who uses it and how

  • Operators of public websites who believe AI companies are already scraping their content and want an active response rather than a blocklist.
  • Site owners who prefer a low-overhead defence, given the project's explicit claim that users should not bear a compute burden to protect their own material.
  • Self-hosters and free-software adherents who deploy the Docker image or a pre-built binary on their own infrastructure rather than paying for a hosted service.
  • Rust developers integrating the trap directly into an existing application through the published miasma library.

Getting started

Miasma can be pulled as the Docker image austinweeks/miasma, added as the miasma crate through your preferred package manager or Cargo, or downloaded as a pre-built binary from the GitHub releases page; the project documentation at austin-weeks.github.io/miasma covers setup and configuration.

How it compares

No comparable tools or paid products are named in the project's own materials, so there is no factual basis here for a feature-by-feature comparison. Within this registry it stands alone, and its closest framing is the project's own contrast with tools that merely attempt to block scrapers rather than feed them degraded data.

When to use it — and when not

The README attaches an explicit caution that there is some risk in deploying this software, and instructs users to read the documentation and the project's disclaimer in full before use, so it should not be adopted casually or without understanding what it returns to visitors. You should skip it if you need a well-established tool with a large track record or if you are unwilling to accept the deployment risk the maintainers flag; the repository also currently carries 12 open issues, which is worth weighing alongside the documentation before committing to it.

project readme (upstream, from github) — read inline

Miasma

No AI crates.io downloads Docker Pulls GitHub commits since latest release

Web crawlers getting stuck in a cloud of poison miasma

AI companies continually scrape the internet at an enormous scale, swallowing up all of its contents to use as training data for their next models. If you have a public website, they are already stealing your work.

Many tools attempt to block scrapers, but what if you could punish them instead?

Miasma does just that! It sends poisoned training data to scrapers alongside self-referential links. Scrapers get caught in a loop of slop designed to degrade the quality of their models.

Miasma is lightning fast with a minimal memory footprint. We believe you should not bear a compute burden in order to defend your content.

Check out our documentation to get started!

Installation

Miasma is available as a Docker image, a Rust library, a pre-built binary, or via your preferred package manager.

Package Managers


[!CAUTION] There is some risk in deploying this software. Please fully read our documentation and see our disclaimer before use.

Banner art by @cerberussaturn07

Frequently asked questions

Is miasma free to use?

miasma is open source under the GPL-3.0 licence. There is no licence fee and no seat count — you can self-host it or, where the project offers one, pay a vendor for a managed version instead.

What does miasma do?

Trap AI web scrapers in an endless poison pit.

What is miasma written in?

miasma is primarily written in Rust. Its source is publicly available at https://github.com/austin-weeks/miasma, and it has 1,198 GitHub stars.