Commit 97a1e20
committed
Add CRAM 3.1 (PR #433)
- Added CRAM_codecs document to describe the proposed CRAM3.1 codecs.
This also has the CRAM3.0 rANS codec in it which has been removed from
CRAMv3.tex.
- Update CRAMv3.tex to include both CRAM 3.0 and 3.1. The differences
are so minor it's essentiall just expanding the list of legal values
in Block compression methods and updating a few 3.0s to be more general.
The new EXTERNAL compression codecs added for CRAM 3.1 are:
- rANSNx16. 16-bit renormalisation instead of the old 8-bit
renormalisation, for extra speed. This now permits either 4-way of
32-way unrolling, with the latter being a good target for SIMD
optimisations.
The newer rANS also includes PACK, RLE and STRIPE (interleaving)
transformations to be applied prior to entropy encoding.
- Adaptive arithmetic coder. This has all the same transformations of
rANSNx16, but with slightly better ratio and better performing on
data sets with non-stationary probabilities. It's slower however.
- FQZComp quality codec. A generalised version of the old fqzcomp-4.6
tool.
- Name tokeniser. A compression tool for any data series that has
repeated structured text elements. It breaks the line down into a
series of tokens and optionally deltas each against a previous
line. Each token them forms a new series of compression streams,
which are in turn compressed and serialised into a single block.
With thanks to Michael Macias and Yash Puligundla for review and
corrections.1 parent c044c04 commit 97a1e20
4 files changed
Lines changed: 3351 additions & 556 deletions
0 commit comments