URS (Universal Reddit Scraper) is a free, MIT-licensed Python command-line tool for scraping, archiving, and livestreaming Reddit, aimed at researchers, data scientists, archivists, and OSINT analysts who need structured local copies of subreddit and redditor activity.

What it is

URS is a comprehensive Reddit scraping and archival command-line tool written in Python. It brings several capabilities into one interface: it scrapes subreddits, scrapes redditors, and scrapes submission comments, and it livestreams Reddit activity, including comments submitted within subreddits or by redditors and submissions made within subreddits or by redditors. It lives in the Python ecosystem and is built around PRAW, the official Python Reddit API Wrapper, which URS relies on for its Reddit API access. Alongside collection, it ships analytical tools: it generates word frequencies for words found in submission titles, bodies, and comments, and it generates wordclouds from scrape results.

The concrete problem URS solves is fragmentation. Anyone who wants Reddit data at volume otherwise writes bespoke PRAW scripts, handles pagination, tracks rate limits, and separately converts raw API objects into analysable files. URS replaces that ad hoc scripting with a single flag-driven command surface: -r for subreddits, -u for redditors, -c for submission comments, -lr and -lu for livestreaming, -f for word frequencies, and -wc for wordclouds, with --csv and --raw controlling output format. It also removes guesswork about API throttling, because --check reports PRAW rate limits directly.

Key capabilities

• Scrapes subreddits, redditors, and submission comments through the -r, -u, and -c flags.
• Livestreams Reddit via PRAW, streaming comments and submissions within subreddits or by redditors through -lr, -lu, and --stream-submissions.
• Generates word frequencies for words found in submission titles, bodies, and comments.
• Generates wordclouds from scrape results with the -wc flag.
• Exports scraped data to CSV and JSON, with --csv selecting comma-separated output and --raw returning unprocessed data.
• Checks PRAW rate limits through the --check flag before large jobs run.
• Displays the directory tree of scraped output, and supports --nosave to skip writing results to disk.

Who uses it and how

• Researchers and archivists capture subreddit and redditor histories into local CSV or JSON files for later analysis and preservation.
• Data scientists and analysts build text corpora from submissions and comments, then run word frequencies and wordclouds over the results without writing PRAW code.
• OSINT analysts track subreddits or individual redditors over time using the livestream modes.
• Data collection teams running large jobs pair URS with residential proxy providers, since the project's sponsors offer rotating IPs for scraping at scale.

Getting started

URS runs as a Python command-line tool, and full install and usage instructions live in the URS Manual linked from the project homepage at https://josephlai241.github.io/URS/. The README states it has grown too long to hold all usage information, so the manual is the authoritative reference.

How it compares

No list of paid products this project replaces is provided in the facts; the sponsors named are proxy services rather than competing products. Among similar tools, URS is the only comprehensive Reddit scraping and archival command-line tool named in this registry, and it sits above PRAW, the library many Reddit scripts call directly, by adding a flag-driven CLI, livestreaming, and analytics on top. Where PRAW gives developers a programming interface, URS gives them a finished command-line tool.

When to use it — and when not to

Anyone self-hosting URS must supply the Reddit API credentials that PRAW requires and accept the rate limits that --check reports; there is no hosted option or database to manage, but there is also no server-side convenience. It is a poor fit for users who want a graphical interface, a managed service, or a non-Python runtime, since URS is a command-line tool written in Python. The honest weakness is documentation sprawl: the README has outgrown its usable length and now defers to an external manual, so newcomers must leave the repository to find complete usage information.