Open source web-scraping projects
Every project in the registry tagged web-scraping, ranked by real GitHub adoption.
Scrapy, a fast high-level web crawling & scraping framework for Python.
Best and simplest tool for website change detection, web page monitoring, and website change alerts. Perfect for tracking content changes, price drops, restock
Privacy-first knowledge base with any AI model you choose
jsoup: the Java HTML parser, built for HTML editing, cleaning, scraping, and XSS safety.
High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.
scrape data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,emai
An agentic extractor for messaging platforms. Noria receives messages & URLs via chat; scrapes web data to evaluate the content against custom scoring matrices
🔮 A Node.js scraper for humans.
Open-source web scraping API. Turn any website into clean markdown or structured JSON. Anti-detect browser, proxy auto-selection, self-hosted. One command: make
Free anti fingerprint browser, 指纹浏览器, 隐私浏览器, 防识别浏览器, 反识别浏览器, 防关联浏览器, 免费的web3空投专用指纹浏览器 https://virtualbrowser.cc/?src=github
Foundational low latency web data collecting in Rust
Python binding to Modest and Lexbor engines. Fast HTML5 parser with CSS selectors for Python.
AgentQL is a suite of tools for connecting your AI to the web. Featuring a query language and Playwright integrations for interacting with elements and extracti
Trap AI web scrapers in an endless poison pit.
WebAI2API: 基于 Camoufox 的网页 AI 转 API 工具，支持 LMArena/Gemini等，多窗口并发与账号隔离。 | Web AI to OpenAI API via Camoufox. Supports LMArena/Gemini and more, multi-window concur
Universal content reader MCP Server for 10+ platforms
Stack blocks visually to build complex browser workflows and execute them via API
Fast, stateless CLI for web search and scrape. Built for AI agents.
🤖 AI-powered web scraping editor with visual workflow builder. Build, test & deploy web scrapers using natural language. Powered by ScrapeGraphAI & LangGraph.
Undetected web-scraping & seamless HTML parsing in Python!
NBA Stats API via Basketball Reference
Turn any website into a compact CLI tailored for AI agents. Browse the web in hundreds of tokens, not tens of thousands.
Run any site JavaScript from Python with the real Firefox engine. Headless SpiderMonkey + Necko for automation and scraping.
Related tags
Frequently asked questions
How many open source web-scraping projects are there?
This registry tracks 23 projects tagged web-scraping, with 176,648 GitHub stars between them. The most-adopted is scrapy at 65,180 stars.
Are these web-scraping projects free to use?
Yes — 23 of the 23 carry an explicit open-source licence across 5 distinct licences, so there is no licence fee. Where a project also sells a hosted or enterprise version, the self-hosted path remains free.
Which web-scraping project should I choose?
The list above is ranked by GitHub stars, but stars measure attention rather than fit. Check three things on each card: the licence (permissive versus copyleft), the language it is written in, and the last-push date — a high-star project that has not been pushed in a year is a liability.
Are these web-scraping projects still maintained?
20 of the 23 were pushed in the last 90 days, and every card shows its exact last-push date so you can see the rest. Sort your shortlist by that date before committing to a migration.