Open source scraper projects
Every project in the registry tagged scraper, ranked by real GitHub adoption.
LLM-ready web crawler built for AI data pipelines
Create agents that monitor and act on your behalf. Your agents are standing by!
👾 Fast and simple video download library and CLI tool written in Go
Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or G
Elegant Scraper and Crawler Framework for Golang
🚀 Self-hosted TikTok & Douyin scraper and no-watermark video downloader — async REST API, MCP server, CLI and web console for posts, profiles, comments and pla
Automated CAPTCHA solver for your browser. Works with Selenium, Puppeteer, Playwright, and more.
Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, P
🔮 A Node.js scraper for humans.
Open-source web scraping API. Turn any website into clean markdown or structured JSON. Anti-detect browser, proxy auto-selection, self-hosted. One command: make
Node.js scraper to get data from Google Play
Python library and CLI for X/Twitter scraping with multi-account rotation and built-in rate-limit handling.
Geziyor, blazing fast web crawling & scraping framework for Go. Supports JS rendering.
NewPipe's core library for extracting data from streaming sites
Python binding to Modest and Lexbor engines. Fast HTML5 parser with CSS selectors for Python.
Scrape tweets, profiles, followers and following from Twitter/X, no API key needed. Python library with smart multi-account pooling, proxy support and async.
📰 Newspaper4k a fork of the beloved Newspaper3k. Extraction of articles, titles, and metadata from news websites.
A Kotlin-based testing/scraping/parsing library providing the ability to analyze and extract data from HTML (server & client-side rendered). It places particula
A universal web-util for PHP.
Related tags
Frequently asked questions
How many open source scraper projects are there?
This registry tracks 20 projects tagged scraper, with 484,905 GitHub stars between them. The most-adopted is firecrawl at 197,725 stars.
Are these scraper projects free to use?
Yes — 20 of the 20 carry an explicit open-source licence across 5 distinct licences, so there is no licence fee. Where a project also sells a hosted or enterprise version, the self-hosted path remains free.
Which scraper project should I choose?
The list above is ranked by GitHub stars, but stars measure attention rather than fit. Check three things on each card: the licence (permissive versus copyleft), the language it is written in, and the last-push date — a high-star project that has not been pushed in a year is a liability.
Are these scraper projects still maintained?
18 of the 20 were pushed in the last 90 days, and every card shows its exact last-push date so you can see the rest. Sort your shortlist by that date before committing to a migration.