Open source ai-scraping projects
Every project in the registry tagged ai-scraping, ranked by real GitHub adoption.
🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDce
Python scraper based on AI
Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI, REST API, and MCP server.
Lightweight library for scraping web-sites with LLMs
Related tags
Frequently asked questions
How many open source ai-scraping projects are there?
This registry tracks 5 projects tagged ai-scraping, with 326,801 GitHub stars between them. The most-adopted is firecrawl at 197,942 stars.
Are these ai-scraping projects free to use?
Yes — 5 of the 5 carry an explicit open-source licence across 4 distinct licences, so there is no licence fee. Where a project also sells a hosted or enterprise version, the self-hosted path remains free.
Which ai-scraping project should I choose?
The list above is ranked by GitHub stars, but stars measure attention rather than fit. Check three things on each card: the licence (permissive versus copyleft), the language it is written in, and the last-push date — a high-star project that has not been pushed in a year is a liability.
Are these ai-scraping projects still maintained?
4 of the 5 were pushed in the last 90 days, and every card shows its exact last-push date so you can see the rest. Sort your shortlist by that date before committing to a migration.