Compression Lab

The problem

Most explanations of data compression teach three algorithms and imply there is one "entropy floor" they all approach. That picture is wrong in an interesting way. How cheaply a text can be stored depends on how much the compressor remembers. If it looks at the previous letter, the cost drops. Look at two letters and it drops again. But each step makes the model bigger, and the model has to be stored with the data.

So the true size is the compressed text plus the model, and there is a sweet spot that moves with the length of the text. Existing explainers never count the model, so they never show that trade-off. I built Compression Lab for students, self-taught engineers and anyone who half-remembers Huffman coding from a course.

What I built

One chart carries the whole argument

The central chart plots six levels of model memory as descending steps. Against them it plots the rate each compressor achieves, the model's own size as a rising curve, and the total as a third curve with its lowest point marked. Paste a longer text and that lowest point moves right. That one moment makes the argument without a paragraph of explanation.

The model's size is measured byte by byte

I wrote code that saves each model as real bytes and code that reads it back. Tests check that a decoder given only the compressed text and those model bytes rebuilds the original exactly, at every level. The model cost on screen is the length of those bytes. If it were an estimate, the lowest point on the chart would be made up.

Every compressor can undo its work

Nobody uses the app to decompress. But a compressor without a matching decompressor is unproven, and the app's claim is that its numbers are correct. So each of the three compressors has a decoder, and the tests round-trip awkward inputs: an empty string, a single character, characters outside the basic Unicode range and every bundled sample.

The pictures replay the real calculation

Each compressor records what it did as it runs: the order Huffman merged symbols, where the sliding window found matches, and the interval at every step of arithmetic coding. The views draw that record and never run the algorithm a second time. Your own text is also tinted letter by letter by how many bits it costs, so structure appears as you raise the model's memory.

A four-language test that reports both outcomes

The app compares the same UN human rights declaration in English, Indonesian, German and Finnish. The idea being tested: languages that build words from stacked affixes hide extra repetition that pattern matching can find. Indonesian gained the most. Finnish, the most affix-heavy of the four, did not. The page reports both results and says four texts can't settle the question.

Result

Compression Lab is live, with a separate page that explains the method. The whole engine is my own code: the frequency models, Huffman coding, arithmetic coding and the sliding-window compressor. It uses no compression, statistics or chart library. Recomputing all six levels takes about 14 milliseconds on my machine, inside a 16-millisecond budget, so the charts update as you type. Nothing you paste leaves the browser. I built it over two days, 2 and 3 September 2026.