# blockchain-compression

> blockchain-compression is a Rust library that compresses blockchain data losslessly using Zstandard plus custom dictionaries trained on Solana-specific byte patterns. The README reports up to 60:1 ratios with the `MaxCompression` preset (Zstd level 19) on archival workloads; ratios fall to 5–15:1 for real-time compression with `FastCompression`.

The library targets archive-node operators, indexer teams, and anyone storing historical blockchain data who is tired of generic compressors leaving bytes on the table because they do not know what an Ed25519 public key or an SPL program ID looks like.

## Start Here

- [Home](https://blockchain-compression.cryptuon.com/): What it compresses, the preset table, architecture, install + usage
- [Features](https://blockchain-compression.cryptuon.com/features/): Lossless round-trip, the Solana dictionary, six presets, feature-gated backends
- [Presets](https://blockchain-compression.cryptuon.com/presets/): All six presets — level, README ratio range, and how to choose per workload
- [Quickstart](https://blockchain-compression.cryptuon.com/quickstart/): cargo add to a verified lossless round-trip in five steps
- [How it works](https://blockchain-compression.cryptuon.com/how-it-works/): The trait layer, the algorithms, and the Solana dictionary, with a pipeline diagram
- [FAQ](https://blockchain-compression.cryptuon.com/faq/): Is it really 60:1? Is it lossless? Which chains, which backend, does it need a validator?
- [About](https://blockchain-compression.cryptuon.com/about/): Design choices — Zstandard, custom dictionaries, the trait-based architecture

## Use Cases

- [Use cases overview](https://blockchain-compression.cryptuon.com/use-cases/): Archive nodes, account snapshots, block explorers, indexer warehousing — the preset for each workload
- [Archive nodes](https://blockchain-compression.cryptuon.com/use-cases/archive-nodes/): Full write-once history with MaxCompression (level 19) at up to 60:1 — CPU you pay once
- [Account state snapshots](https://blockchain-compression.cryptuon.com/use-cases/account-state-snapshots/): Dense repeated owner and program fields — the Accounts preset at an illustrative 15–40:1
- [Block explorers](https://blockchain-compression.cryptuon.com/use-cases/block-explorers/): Keep more of history on warm disk with the Mixed preset at 12–35:1
- [Indexer & DePIN warehousing](https://blockchain-compression.cryptuon.com/use-cases/indexer-warehousing/): Warehouse and replicate decoded transactions with the Transactions preset (level 3) at 10–30:1

## Comparisons

- [vs zstd (default)](https://blockchain-compression.cryptuon.com/compare/zstd/): Why a tuned dictionary beats vanilla Zstandard on chain data
- [vs DEFLATE / gzip](https://blockchain-compression.cryptuon.com/compare/deflate/): Where this library wins, and where DEFLATE is still the right answer

## Blog

- [Blog index](https://blockchain-compression.cryptuon.com/blog/): Engineering writing on dictionary compression, ratio honesty, and chain-specific structure
- [Reading the algorithm: how we exploit chain-specific structure](https://blockchain-compression.cryptuon.com/blog/reading-the-algorithm/): A source-level tour of the Solana preset — the dictionary patterns, the Zstd construction, the preset-to-level mapping
- [Measuring compression: ratios that matter for archive node operators](https://blockchain-compression.cryptuon.com/blog/measuring-ratios-that-matter/): How to measure ratios honestly, why the lower bound matters more than the 60:1 headline, and how to benchmark your own workload
- [Why generic compressors leave bytes on the table for chain data](https://blockchain-compression.cryptuon.com/blog/generic-compressors-and-chain-data/): What vanilla zstd actually does on Solana data, and what a hardcoded dictionary buys you

## Reference

- [Contact](https://blockchain-compression.cryptuon.com/contact/): Get in touch with the blockchain-compression team
- [RSS](https://blockchain-compression.cryptuon.com/rss.xml): Blog feed

## Source

- [GitHub](https://github.com/cryptuon/blockchain-compression)
- [crates.io](https://crates.io/crates/blockchain-compression)
- [docs.rs](https://docs.rs/blockchain-compression)