CLP (Compressed Log Processor) is YScope's free, Apache-2.0 licensed log management tool, written in C++, that compresses JSON and unstructured logs and allows those compressed logs to be searched without decompression, aimed at engineers and operators who store and query large volumes of logs.

What it is

YScope's Compressed Log Processor is a log management system built around a single idea: logs should be compressed as early as possible and searched in their compressed form. It provides an end-to-end pipeline covering compression, search, analytics, and viewing, and it accepts two kinds of input — raw logs, or CLP's compressed intermediate representation (IR) produced by CLP's own logging libraries. Logs are compressed into archives, which can then be searched and analyzed through a purpose-built web UI.

The concrete problem it solves is the cost of index-based log storage. Searching logs conventionally means building and maintaining an index before any query can run, which consumes storage and compute whether or not anyone queries the data. CLP uses an index-less design, so logs are compressed and searched directly. In the project's benchmarks, this design is measured against Elasticsearch and Splunk, which are fundamentally index-based and cannot be searched without indexes, and against MongoDB and PostgreSQL, whose indexes were disabled for a fair comparison because enabling them would have worsened their compression ratio. It also replaces general-purpose compression for log data: CLP's IR format achieves a higher compression ratio than general-purpose compressors such as Zstandard.

Key capabilities

• Compresses logs into archives and searches those archives without decompressing them first.
• Handles both JSON logs and unstructured, free-text logs.
• Provides real-time compression through logging libraries for Python and Java (Log4j1, Log4j2, and Logback appenders), so only compressed logs are written to disk or transmitted over the network.
• Writes compressed logs in CLP's intermediate representation (IR) format; re-compressing IR into archives can roughly double the compression ratio and enable global search, at the cost of buffering enough logs in memory.
• Ships a web UI (components/webui) for searching and viewing compressed logs, plus a log viewer that filters by log level verbosity, for example showing only logs at ERROR or above.
• Provides IR analytics libraries for Python (clp-ffi-py) and Go (clp-ffi-go).
• Includes log-surgeon, a pushdown-automata-based log parser described as 3x faster.

Who uses it and how

• Teams running JSON-emitting services, such as the MongoDB logs used for CLP's JSON benchmarks, who need compressed storage plus fast queries over that data.
• Teams with unstructured or free-text logs, such as the Hadoop logs used for the unstructured benchmarks, where no fixed schema exists to index against.
• Applications written in Python or Java that adopt CLP's logging libraries so that compressed logs alone are written to disk or sent across the network, rather than full-text logs that are compressed later.
• Operators who prefer a web UI for searching and viewing logs over grepping files by hand, and who want log-level filtering to cut noise during debugging.
• Environments where index maintenance is the bottleneck, since CLP's index-less design removes the need to build indexes before searching.

Getting started

The facts supplied do not include an install command, package name, Docker image, or compose file, so the entry points to work from are CLP's logging libraries for Python and Java, the web UI in components/webui, and the project's 2021 and 2024 papers. Installation and deployment specifics should be taken from the repository README rather than assumed.

How it compares

Among the tools named in CLP's own benchmarks, Elasticsearch and Splunk are index-based systems that cannot search logs without indexes, while MongoDB and PostgreSQL were compared with indexing disabled. CLP positions itself as the index-less alternative, accepting raw logs or its IR format and searching compressed archives directly. Against general-purpose compressors such as Zstandard, CLP's IR format is documented as achieving a higher compression ratio for logs.

When to use it — and when not to

A self-hoster takes on operating a compression and search pipeline, managing the trade-off between IR and archives, and provisioning enough memory to buffer logs when archive-level global search is required. CLP is a poor fit for anyone who wants a managed service with no operational surface, or who depends on a rich ecosystem of pre-built index-based plugins and integrations. Two weaknesses are visible in the facts: the repository carries 493 open issues against roughly 1,088 stars, and the README excerpt does not document installation, leaving setup discovery to the full repository.