q is a Python command-line tool, licensed under GPL-3.0, that runs SQL statements directly against delimited text files and multi-file SQLite databases without importing the data into a database server first.

What it is

q lives in the Linux command line and in the broader "text as data" ecosystem, where tabular files such as CSV and TSV are treated as queryable database tables. It accepts ordinary delimited files as table sources and supports standard SQL constructs including WHERE, GROUP BY and JOIN, with automatic column name and type detection and full support for multiple character encodings. It also opens sqlite3 databases directly, and can address several database files at once using the ::: syntax.

The concrete problem it solves is the gap between having data in a file or a set of SQLite files and being able to ask questions of it with SQL. Normally that means writing an import step, loading the file into a database, and only then querying, which costs time and disk and leaves a copy behind. q replaces that import path: the file is the table, and the query runs against it. It repeats this for multi-file SQLite, joining across separate database files without merging them or loading them into memory. An internal cache is available to accelerate repeated queries on the same file.

Key capabilities

• Runs SQL directly on tabular text files: q "SELECT COUNT(*) FROM ./clicks_file.csv WHERE c3 > 32.3".
• Reads standard input as a table named -, so pipelines work: ps -ef | q -H "SELECT UID, COUNT(*) cnt FROM - GROUP BY UID ORDER BY cnt DESC LIMIT 3".
• Queries several SQLite database files in one statement with ::: syntax, joining across files such as some_db.sqlite3:::albums and another_db.sqlite3:::tracks.
• Auto-caches files to accelerate repeat queries, enabled per query with -C readwrite or -C read, or set persistently through caching_mode in .qrc.
• The caching gain is documented in the README: a 5,000,000-row, 100-column, 4.8GB file drops from 4 minutes 47 seconds to 1.92 seconds, and a 100,000-row, 50-column, 48MB file drops from 2.7 seconds to 0.105 seconds.
• Performs automatic column name and type detection and supports multiple character encodings.
• Ships as a CLI for all operating systems; current major version is 3.1.6, with the previous 2.0.19 still downloadable from GitHub releases.

Who uses it and how

• Shell users who want to filter and aggregate live process or log output in a pipeline rather than writing it to a temporary file first.
• Analysts working on large delimited exports, in the multi-gigabyte range shown in the README's caching table, who want answers without standing up a database.
• Engineers holding several SQLite files for the same domain who need a join across them and do not want to merge or load them into memory.
• Ad-hoc exploration and one-off reporting in command-line environments, where the deployment shape is a single user running a single command.
• Repeat-query workflows on the same file, where enabling the cache trades disk space for a large speed improvement.

Getting started

Installation instructions for every operating system are published at https://harelba.github.io/q/#installation, and that page also links to the download for immediate use. The README states that the new major version 3.1.6 is out, while 2.0.19 remains available from the GitHub releases page.

How it compares

The facts provided do not name any paid products that q replaces, and they do not name any comparable tools either. It stands alone in this registry under Infrastructure & Operations / Databases.

When to use it — and when not to

Caching is not enabled by default, because the caches consume disk space, so a self-hoster who wants the documented speed gains must choose the caching mode and manage the resulting cache files. It is a command-line tool, so it provides no server, no user accounts and no scheduler; anyone who needs concurrent multi-user access, or wants querying to be an automated, shared service rather than a command a person runs, should look elsewhere. The project also carries 132 open issues, and the README itself is short and directs most detail to the external documentation site.

$ q "SELECT COUNT(*) FROM ./clicks_file.csv WHERE c3 > 32.3"

$ ps -ef | q -H "SELECT UID, COUNT(*) cnt FROM - GROUP BY UID ORDER BY cnt DESC LIMIT 3"

$ q "select count(*) from some_db.sqlite3:::albums a left join another_db.sqlite3:::tracks t on (a.album_id = t.album_id)"