Skip to content

Commit 7c472d7

Browse files
committed
update readme
1 parent 18b633f commit 7c472d7

1 file changed

Lines changed: 29 additions & 5 deletions

File tree

README.md

Lines changed: 29 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,18 @@ streamvbyte
22
===========
33
[![Ubuntu 22.04 CI (GCC 9, 10, 11 and 12, LLVM 12, 13, 14)](https://github.com/lemire/streamvbyte/actions/workflows/ubuntu22.yml/badge.svg)](https://github.com/lemire/streamvbyte/actions/workflows/ubuntu22.yml)
44
[![Ubuntu 24.04 CI (GCC 13, 14, LLVM 17, 18)](https://github.com/lemire/streamvbyte/actions/workflows/ubuntu24.yml/badge.svg)](https://github.com/lemire/streamvbyte/actions/workflows/ubuntu24.yml)
5-
[![macOS 11 CI (LLVM 13, GCC 10, 11, 12)](https://github.com/lemire/streamvbyte/actions/workflows/macos.yml/badge.svg)](https://github.com/lemire/streamvbyte/actions/workflows/macos.yml)
5+
[![macOS CI](https://github.com/lemire/streamvbyte/actions/workflows/macos.yml/badge.svg)](https://github.com/lemire/streamvbyte/actions/workflows/macos.yml)
66
[![VS-Latest-CI](https://github.com/lemire/streamvbyte/actions/workflows/vs16.yml/badge.svg)](https://github.com/lemire/streamvbyte/actions/workflows/vs16.yml)
77
[![VS17-CI](https://github.com/lemire/streamvbyte/actions/workflows/vs.yml/badge.svg)](https://github.com/lemire/streamvbyte/actions/workflows/vs.yml)
88

9-
StreamVByte is a new integer compression technique that applies SIMD instructions (vectorization) to
9+
StreamVByte is an integer compression technique that applies SIMD instructions (vectorization) to
1010
Google's Group Varint approach. The net result is faster than other byte-oriented compression
1111
techniques.
1212

13+
Both the encoder and the decoder are vectorized: there is an optimized path using SSE 4.1 on
14+
x64 processors and one using NEON on 64-bit ARM (AArch64) processors. A portable scalar
15+
fallback is used otherwise, and the appropriate path is selected at runtime.
16+
1317
The approach is patent-free, the code is available under the Apache License.
1418

1519

@@ -129,13 +133,15 @@ cmake --install build
129133
#### Benchmarking with CMake
130134

131135

132-
After building, you may run our benchmark as follows:
136+
After building (with `-DSTREAMVBYTE_ENABLE_TESTS=ON`), you may run our benchmark as follows:
133137

134138
```
135-
./build/test/perf
139+
./build/perf
136140
```
137141

138-
The benchmarks are not currently built under Windows.
142+
On Apple Silicon, run it with `sudo` to enable the hardware performance counters
143+
(instructions, cycles, branch and cache misses); on Linux you may instead relax
144+
`kernel.perf_event_paranoid`. The benchmarks are not currently built under Windows.
139145

140146

141147
### 2. Building with Makefile:
@@ -180,6 +186,24 @@ zigzag_encode(mysignedints, myunsignedints, number); // mysignedints => myunsign
180186
zigzag_decode(myunsignedints, mysignedints, number); // myunsignedints => mysignedints
181187
```
182188
189+
Performance
190+
-----------
191+
192+
StreamVByte sustains multi-gigabyte-per-second throughput for both encoding and
193+
decoding. The exact speed depends on the data distribution and on your hardware.
194+
As an illustration, the `perf` benchmark (500,000 random 32-bit integers per run,
195+
throughput measured against the 2 MB uncompressed input) reports the following on
196+
an Apple M4 Max:
197+
198+
| data distribution | encode | decode |
199+
|------------------------------|-----------|-----------|
200+
| mixed 1–4 byte (log-uniform) | ~27 GB/s | ~41 GB/s |
201+
| full-range (mostly 4 byte) | ~26 GB/s | ~33 GB/s |
202+
| small values (1 byte) | ~27 GB/s | ~36 GB/s |
203+
204+
You can reproduce these numbers with `./build/perf` after building the tests
205+
(see the Benchmarking instructions above).
206+
183207
Technical posts
184208
---------------
185209

0 commit comments

Comments
 (0)