head to head · open source
proxy_pool vs crawly
proxy_pool has 23,786 GitHub stars, 5,411 forks, 298 open issues and last shipped 5 months ago. crawly has 1,129 stars, 125 forks, 8 open issues and last shipped 2 days ago. proxy_pool leads on adoption by 2,007% (23,786 vs 1,129 stars). proxy_pool is written in Python under MIT; crawly is written in Elixir under Apache-2.0. proxy_pool has attracted 23% as many forks as stars, crawly 11%. crawly was the more recently maintained of the two, and both are self-hostable with no licence fee. The two share 2 topic tags (crawler, spider), so they are genuine substitutes rather than adjacent tools.
Two open source projects, one decision. Both are free and self-hostable — the differences are community size, license terms, language stack and release pace.
← all 20902 open source comparisons
Side by side
| proxy_pool | crawly | |
|---|---|---|
| GitHub stars | ★ 24K | ★ 1.1K |
| License | MIT | Apache-2.0 |
| Written in | Python | Elixir |
| Last push | 2026-06-15 | 2026-11-09 |
| Forks | ⑂ 5.4K | ⑂ 125 |
| Self-hosting | Yes | Yes |
| Data ownership | Your server | Your server |
pick proxy_pool if
- You weight community size — 24K stars and counting
- You want the MIT license terms
- Your stack matches Python
- You value the larger contributor base for long-term maintenance
pick crawly if
- You want the crawly feature set and don't need the biggest community
- You prefer the Apache-2.0 license terms
- Your stack matches Elixir
- You evaluated both and crawly fits your workflow better
About proxy_pool
ProxyPool 爬虫代理IP池 ======= ( ) 0: try: html = requests.get(' proxies={"http": " 使用代理访问 return html except Exception: retry count = 1 删除代理池中代理 delete proxy(proxy) return None python from fetcher.baseFetcher import BaseFetcher from util.webRequest import WebRequest class MyProxyFetcher(BaseFetcher): """我的代理源""" name = "myproxy" url = " enabled = True def fetch(self): r = WebRequest().get(" for item in r.json: yield item["ip"] + ":" + item["port"] 2、添加好后， schedule 进程下次抓取时会自动扫描 fetcher/sources/ 目录并启用新代理源，无需修改配置。 可用 python proxyPool.py fetcher 命令查看当前启用的代理源列表。 如需临时禁用某个代理源，在 setting.py 的 PROXY FETCHER EXCLUDE 黑名单中添加其…
read the full proxy_pool overview →
About crawly
Crawly Overview Crawly is an application framework for crawling web sites and extracting structured data which can be used for a wide range of useful applications, like data mining, information processing or historical archival. Requirements 1. Elixir 1.14 2. Works on GNU/Linux, Windows, macOS X, and BSD. Quickstart 0. Create a new project: mix new quickstart sup 1. Add Crawly as a dependencies: 2. Fetch dependencies: $ mix deps.get 3. Create a spider New in 0.15.0 : It's possible to use the command to speed up the spider creation, so you will have a generated file with all needed callbacks: mix…
read the full crawly overview →
More in Data & Analytics
Related comparisons
More Data Extraction & Web Scraping projects
Compare either of these against the rest of the Data Extraction & Web Scraping field.
Frequently asked questions
Is proxy_pool or crawly more popular?
proxy_pool has 23,786 GitHub stars and crawly has 1,129. proxy_pool has the larger community by that measure.
Are proxy_pool and crawly free?
Both are open source. proxy_pool is licensed under MIT and crawly under Apache-2.0. Neither carries a licence fee.
What is the difference between proxy_pool and crawly?
proxy_pool is written in Python and crawly in Elixir. The practical differences are community size, licence terms, language stack and release cadence — all compared in the table above.
Which should I choose, proxy_pool or crawly?
Choose proxy_pool if you want the larger community (23,786 stars) or its MIT licence terms. Choose crawly if its feature set, stack or Apache-2.0 licence fits better. Both are self-hostable.