wreq-python: Open Source Data Extraction & Web Scraping

What is wreq-python?

wreq-python is an ergonomic, privacy-aware Python HTTP client built on the Rust wreq library, aimed at developers who need high-fidelity TLS, JA3/JA4 and HTTP/2 browser fingerprint emulation from Python.

What it is

wreq-python is a Python HTTP client library whose networking core is written in Rust and exposed through a Python API. It runs in the Data & Analytics stack under Data Extraction & Web Scraping, is distributed on PyPI as the wreq package under the Apache-2.0 licence, and is documented at python.wreq.org. Both asynchronous and blocking Client objects are provided, and the client deliberately mirrors the API shape that Python developers already use with other HTTP libraries, so Client(emulation=Emulation.Chrome153) reads like ordinary Python while the underlying work — TLS, HTTP/2 frames, connection handling — happens in Rust.

The concrete problem it solves is fingerprint mismatch. Anti-bot systems such as Akamai classify clients by their TLS and HTTP/2 signatures, and string-based fingerprints like JA3 and JA4 cannot be reliably reproduced by simply pasting a fingerprint string, because those protocols change more slowly than browser release cycles and the same fingerprint appears across many browser models. wreq-python replaces the string-pasting workaround with fine-grained control over TLS and HTTP/2 extensions and settings, plus more than one hundred maintained browser device emulation profiles, so a request from a script can present the protocol behaviour of a real browser.

Key capabilities

Async and blocking Client classes, so the same library serves asyncio code and synchronous scripts.
Customizable TLS and HTTP/2 signatures, with JA3, JA4 and Akamai fingerprint topics listed for the project.
Browser emulation through Emulation profiles such as Emulation.Chrome153, backed by 100+ maintained device profiles.
Body handling for plain bodies, JSON, urlencoded and multipart payloads, plus an HTTP Trailer capability.
Session behaviour including a cookie store, redirect policy, original header preservation and a certificate store covering CAs and mTLS.
Transfer features covering connection pooling, streaming transfers, zero-copy transfers, automatic decompression and WebSocket upgrade.
Rotating proxies and HTTPS through BoringSSL, with support for free-threaded Python builds.

Who uses it and how

Scraping and crawler teams that hit anti-bot defences and need their client to look like a browser at the protocol level rather than at the header level.
Workflows that rotate proxies across many requests while keeping a connection pool and cookie store consistent.
Projects that need JA3, JA4 or Akamai fingerprint control without leaving Python, for example when integrating into an existing asyncio pipeline.
Teams operating in censorship-resistant contexts, where the ability to shape TLS and HTTP/2 behaviour is the point of the tool.
Developers who want the API they already know from other Python clients but need the throughput of a Rust networking core.

Getting started

Install from PyPI with pip install wreq, then construct a client and call it, as in the README's Client(emulation=Emulation.Chrome153) example using the asyncio runtime. Building from source requires the BoringSSL build environment and maturin, with maturin develop --uv for development and maturin build --release for wheels.

How it compares

The README's own benchmark suite, driven by pyperf, reports that wreq-python outperforms requests, httpx, aiohttp and curl_cffi, though the README states those results are for reference only and vary by environment. Compared with those clients, wreq-python is a Rust core with a Python surface rather than a pure-Python or libcurl-based implementation, and fingerprint emulation is a first-class feature rather than an add-on.

When to use it — and when not to

Choose it when protocol-level fingerprint fidelity matters and a Rust-backed client is acceptable. Building from source means installing a BoringSSL toolchain with build-essential, cmake, perl, pkg-config, libclang-dev and musl-tools, which is real work for teams that only ever install wheels. Avoid it if a plain HTTP client without emulation already passes the endpoints in question, or if the benchmark claims cannot be verified in your own environment.

project readme (upstream, from github) — read inline

wreq

PyPI - License

🚀 Help me work seamlessly with open source sharing by sponsoring me on GitHub

An ergonomic and modular Python HTTP Client for high-fidelity protocol matching, featuring customizable TLS, JA3/JA4, and HTTP/2 signature capabilities, powered by wreq.

Features

Async and Blocking Clients
Plain bodies, JSON, urlencoded, multipart
HTTP Trailer
Cookie Store
Redirect Policy
Original Header
Rotating Proxies
Connection Pooling
Streaming Transfers
Zero-Copy Transfers
WebSocket Upgrade
HTTPS via BoringSSL
Free-Threaded Safety
Automatic Decompression
Certificate Store (CAs & mTLS)

Example

The following example uses the asyncio runtime with wreq installed via pip:

pip install wreq

And then the code:

import asyncio
from wreq import Client, Emulation


async def main():
    # Build a client
    client = Client(emulation=Emulation.Chrome153)

    # Use the API you're already familiar with
    resp = await client.get("https://pingly.us.kg/api/all")
    print(await resp.text())


if __name__ == "__main__":
    asyncio.run(main())

Additional learning resources include:

Behavior

HTTP/2 over TLS

Due to the complexity of TLS encryption and the widespread adoption of HTTP/2, browser fingerprints such as JA3, JA4, and Akamai cannot be reliably emulated using simple fingerprint strings. Instead of parsing and emulating these string-based fingerprints, wreq provides fine-grained control over TLS and HTTP/2 extensions and settings for precise browser behavior emulation.

Device Emulation

TLS and HTTP/2 fingerprints are often identical across various browser models because these underlying protocols evolve slower than browser release cycles. 100+ browser device emulation profiles are maintained in wreq.

Building

Platforms

Linux(glibc/musl): x86_64, aarch64, armv7, i686
macOS: x86_64,aarch64
Windows: x86_64,i686,aarch64
Android: aarch64, x86_64

Development

Install the BoringSSL build environment by referring to boringssl.

# on ubuntu or debian
sudo apt install -y build-essential cmake perl pkg-config libclang-dev musl-tools git
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
pip install uv maturin

uv venv
source .venv/bin/activate

# development
maturin develop --uv

# build wheels
maturin build --release

Benchmark

Outperforms requests, httpx, aiohttp, and curl_cffi according to our benchmark suite driven by pyperf, though results are for reference only as they vary by environment.

Services

Help sustain the ongoing development of this open-source project by reaching out for commercial support. Receive private guidance, expert reviews, or direct access to the maintainer, with personalized technical assistance tailored to your needs.

License

Licensed under either of Apache License, Version 2.0 (LICENSE or http://www.apache.org/licenses/LICENSE-2.0).

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be licensed as above, without any additional terms or conditions.

Is wreq-python free to use?

wreq-python is open source under the Apache-2.0 licence. There is no licence fee and no seat count — you can self-host it or, where the project offers one, pay a vendor for a managed version instead.

What does wreq-python do?

An ergonomic, privacy-aware Python HTTP Client

What is wreq-python written in?

wreq-python is primarily written in Python. Its source is publicly available at https://github.com/0x676e67/wreq-python, and it has 1,505 GitHub stars.