faster-than-requests is an MIT-licensed Python 3 HTTP client written in Nim that keeps a requests-style call surface while adding a built-in multi-threaded web scraper, shipping as a single file with no direct dependencies, and it is aimed at developers doing high-volume scraping or downloads who find python-requests, urllib3, or PyCURL too slow.

What it is

faster-than-requests is a Python 3 library imported as "faster_than_requests" and conventionally aliased: import faster_than_requests as requests. The public surface deliberately mirrors python-requests, so familiar verbs such as get(), post(), put(), head(), patch(), and delete() carry over, alongside response helpers get2str(), get2dict(), get2json(), post2str(), post2dict(), and post2json(). The implementation is a Nim library, distributed as one file of 999 lines of code with zero direct dependencies, and it is released under the MIT licence. Beyond plain HTTP, it exposes file download helpers download() and download2(), a WebSocket client through websocket_send(), and seven scraper entry points, scraper() through scraper7().

The concrete problem it solves is throughput in Python data extraction and web scraping, where the request library itself becomes the bottleneck. The README's own comparison table measures the real time needed to complete 10,000 local HTTP requests, and reports faster_than_requests at 0.40 against Requests at 15.58, a cached Requests object at 5.50, urllib at 4.00, urllib3 at 3.55, PyCURL at 0.75, and PyWGET at 152.39. It therefore replaces python-requests and urllib3 as the transport layer, and it also absorbs work that would otherwise be handed to Scrapy or a hand-rolled thread pool, since multi-threaded crawling and extraction are built in.

Key capabilities

• Standard HTTP verbs get(), post(), put(), head(), patch(), and delete() for direct call-for-call replacement of python-requests code.
• File download through download() and download2(), for example requests.download("http://example.com/foo.jpg", "out.jpg").
• Multi-threaded web scraping via scraper(), invoked as requests.scraper(["http://foo.io", "http://bar.io"], threads=True), with a one-liner example at examples/multithread_web_scraper.py.
• URL-to-SQLite scraping through scraper5(), which takes sqlite_file_path="database.db" and writes crawled URLs into a database.
• Regex-powered scraping through scraper6(), for example requests.scraper6(["http://python.org"], ["(www|http:|https:)+[^\s]+[\w]"]).
• CSS-selector scraping through scraper7(), for example requests.scraper7("http://python.org", "body > div.someclass a#someid").
• WebSockets for binary and text frames over websocket_send(), for example requests.websocket_send("ws://echo.websocket.org", "data here").

Who uses it and how

• Scraping pipelines that fan out across many URLs use scraper() with threads=True instead of maintaining their own worker pool.
• Teams that need crawled results persisted rather than printed point scraper5() at a sqlite_file_path and land results directly in a SQLite database.
• Scrapers targeting specific DOM nodes use scraper7() with a CSS selector instead of parsing HTML downstream.
• Harvesting links or patterns from open-data pages uses scraper6() with a regular expression such as (www|http:|https:)+[^\s]+[\w].
• Asset-heavy jobs, such as bulk image or file retrieval, use download() and download2() rather than looping over get() calls.

Getting started

The README's "Use" section shows the whole entry point: import faster_than_requests as requests, then call requests.get("http://httpbin.org/get") or any of the other functions directly. No package manager command, Docker image, or compose file appears in the provided material, so the import line is the documented starting point; the repository homepage is a GitHub gist.

How it compares

Within the readme's own benchmark table, faster_than_requests is positioned against python-requests, a cached Requests object, urllib, urllib3, PyCURL, and PyWGET on raw elapsed time for 10,000 local requests, and it is the only entry in that table marked as having WebSockets and a built-in multi-threaded scraper. Its topics also place it alongside requests-toolbelt, requests3, and Scrapy in the same problem space, though the facts describe no paid product that it replaces.

When to use it — and when not to

The appeal for a self-hoster is operational: one file, zero direct dependencies, and no database, storage, or SMTP service to run, although scraper5() does require a writable SQLite database path. It is a poor fit for teams that depend on the surrounding python-requests ecosystem, such as session adapters, authentication helpers, or requests-toolbelt extensions, since those are not documented here. The honest caveat is that the benchmark figures carry the README's own footnote, "Stats as of year 2020," measured on one x86_64 64-bit AMD machine with an SSD running Arch Linux via Docker, and the contributor count for the package is one, so the speed advantage should be re-measured on the target workload.

import faster_than_requests as requests

requests.get("http://httpbin.org/get")                                      # GET
requests.post("http://httpbin.org/post", "Some Data Here")                  # POST
requests.download("http://example.com/foo.jpg", "out.jpg")                  # Download a file
requests.scraper(["http://foo.io", "http://bar.io"], threads=True)          # Multi-Threaded Web Scraper
requests.scraper5(["http://foo.io"], sqlite_file_path="database.db")        # URL-to-SQLite Web Scraper
requests.scraper6(["http://python.org"], ["(www|http:|https:)+[^\s]+[\w]"]) # Regex-powered Web Scraper
requests.scraper7("http://python.org", "body > div.someclass a#someid"])    # CSS Selector Web Scraper
requests.websocket_send("ws://echo.websocket.org", "data here")             # WebSockets Binary/Text

import faster_than_requests as requests
requests.get("http://example.com")