head to head · open source
scrapy vs page-agent
scrapy has 65,231 GitHub stars, 12,094 forks, 394 open issues and last shipped 2 days ago. page-agent has 29,634 stars, 2,642 forks, 94 open issues and last shipped 2 days ago. scrapy leads on adoption by 120% (65,231 vs 29,634 stars). scrapy is written in Python under BSD-3-Clause; page-agent is written in TypeScript under MIT. scrapy has attracted 19% as many forks as stars, page-agent 9%. scrapy was the more recently maintained of the two, and both are self-hostable with no licence fee.
Two open source projects, one decision. Both are free and self-hostable — the differences are community size, license terms, language stack and release pace.
← all 20902 open source comparisons
Side by side
| scrapy | page-agent | |
|---|---|---|
| GitHub stars | ★ 65K | ★ 30K |
| License | BSD-3-Clause | MIT |
| Written in | Python | TypeScript |
| Last push | 2026-11-04 | 2026-11-04 |
| Forks | ⑂ 12K | ⑂ 2.6K |
| Self-hosting | Yes | Yes |
| Data ownership | Your server | Your server |
pick scrapy if
- You weight community size — 65K stars and counting
- You want the BSD-3-Clause license terms
- Your stack matches Python
- You value the larger contributor base for long-term maintenance
pick page-agent if
- You want the page-agent feature set and don't need the biggest community
- You prefer the MIT license terms
- Your stack matches TypeScript
- You evaluated both and page-agent fits your workflow better
About scrapy
.. logo image:: :target: :alt: Scrapy :width: 480px .. version image:: :target: :alt: PyPI Version .. python version image:: :target: :alt: Supported Python Versions .. tests image:: :target: :alt: Tests .. coverage image:: :target: :alt: Coverage report .. conda image:: :target: :alt: Conda Version .. deepwiki image:: :target: :alt: Ask DeepWiki Scrapy is a web scraping framework to extract structured data from websites. It is cross platform, and requires Python 3.10+. It is maintained by Zyte (formerly Scrapinghub) and many other contributors . .. many other contributors: .. Scrapy: .. Zyte: Install with…
read the full scrapy overview →
About page-agent
Page Agent The GUI Agent Living in Your Webpage. One script gives any web page its own AI agent. 🌐 English 中文 🚀 Demo 📖 Docs 📢 HN Discussion 𝕏 Follow on X ✨ Features 🎯 Easy integration No need for browser extension / python / headless browser . Just in page javascript. Everything happens in your web page. 📖 Text based DOM manipulation No screenshots. No multi modal LLMs or special permissions needed. 🧠 Bring your own LLMs Works with most mainstream models, including locally deployed ones. See supported models. 🐙 Optional chrome extension for multi page tasks. And an MCP Server (Beta) to control it from…
read the full page-agent overview →
More in Data & Analytics
Related comparisons
More Data Extraction & Web Scraping projects
Compare either of these against the rest of the Data Extraction & Web Scraping field.
Frequently asked questions
Is scrapy or page-agent more popular?
scrapy has 65,231 GitHub stars and page-agent has 29,634. scrapy has the larger community by that measure.
Are scrapy and page-agent free?
Both are open source. scrapy is licensed under BSD-3-Clause and page-agent under MIT. Neither carries a licence fee.
What is the difference between scrapy and page-agent?
scrapy is written in Python and page-agent in TypeScript. The practical differences are community size, licence terms, language stack and release cadence — all compared in the table above.
Which should I choose, scrapy or page-agent?
Choose scrapy if you want the larger community (65,231 stars) or its BSD-3-Clause licence terms. Choose page-agent if its feature set, stack or MIT licence fits better. Both are self-hostable.