Skip to content

Commit a920ff4

Browse files
dfa1claude
andcommitted
docs: terser CHANGELOG entries, add changelog skill
The 5 entries I added for the FSST/dispatch work were verbose paragraphs (full rationale + before/after numbers inline) — too verbose per feedback. Rewrote them as one-sentence titles ending in a commit link; the numbers and rationale live in the commit message, which the link points to. Also records a standalone `changelog` skill (.claude/skills/) codifying this convention, split out of the `release` skill's CHANGELOG-sync step so it can run without cutting a release. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
1 parent eec25c1 commit a920ff4

2 files changed

Lines changed: 69 additions & 5 deletions

File tree

.claude/skills/changelog.md

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
---
2+
name: changelog
3+
description: Write CHANGELOG.md entries for unreleased commits — terse, one-line bullets that link to the introducing commit(s), not paragraphs. Triggers on "prepare the changelog", "update CHANGELOG", "write changelog entries", or invokes `/changelog`. Standalone version of the `release` skill's "CHANGELOG sync" step — use this when you want entries written without cutting a release yet.
4+
---
5+
6+
## Overview
7+
8+
Write CHANGELOG.md entries for commits that don't have one yet, under `## [Unreleased]`.
9+
Entries are **terse** — a title, not a paragraph — ending in a link to the introducing
10+
commit(s). The link is the receipt; anyone who wants the "why" and the numbers reads the
11+
commit message (`git show <sha>` or the GitHub commit page), not the changelog.
12+
13+
## Steps
14+
15+
1. Find the range of commits missing a changelog entry: compare `## [Unreleased]` in
16+
`CHANGELOG.md` against `git log --oneline <last-documented-commit>..HEAD`. If unclear
17+
where the last-documented commit is, ask.
18+
2. For each notable commit, write **one bullet**:
19+
- A single short sentence — what changed, fewest words that stay precise. No numbers,
20+
no "why", no before/after prose — those live in the commit message.
21+
- End with the commit SHA(s), GitHub-auto-linked:
22+
`([abc1234](https://github.com/dfa1/vortex-java/commit/abc1234))`
23+
- One logical change spanning multiple commits: comma-separate SHAs, newest first.
24+
3. Categorize under the right `###` heading (create if missing): `Added` / `Changed` /
25+
`Fixed` / `Performance` / `Removed` / `Security` — Keep-a-Changelog order (Added,
26+
Changed, Deprecated, Removed, Fixed, Security), plus this project's `Performance`.
27+
4. Skip pure-internal churn: CI/Sonar fixes, source-tree moves, dedup refactors,
28+
test/tooling-only commits (see [[feedback_changelog_terse]]) — omit rather than write a
29+
bullet nobody user-facing cares about.
30+
5. Commit as `docs: CHANGELOG entries for <short description>`.
31+
32+
## Example
33+
34+
Good (terse — what to write):
35+
```
36+
- Utf8/Binary now use cost-based encoding selection instead of first-match dispatch. ([1bbc3549](https://github.com/dfa1/vortex-java/commit/1bbc3549))
37+
- FsstEncodingDecoder no longer throws on valid files with empty FSST metadata. ([1bbc3549](https://github.com/dfa1/vortex-java/commit/1bbc3549))
38+
```
39+
40+
Bad (too verbose — do NOT write this):
41+
```
42+
- `CascadingCompressor` now runs `Utf8`/`Binary` columns through the same sample-and-measure
43+
encoding competition `Primitive` columns already get, instead of first-match dispatch.
44+
`DictEncodingEncoder` is registered ahead of `FsstEncodingEncoder`, so Dict always won
45+
regardless of actual output size — confirmed by inspecting a 50k-row, high-cardinality
46+
string file and finding `vortex.dict` where `vortex.fsst` is ~40% smaller. On
47+
`highCardinalityUtf8_javaVsJni` (50k distinct 6-byte strings) this drops the Java/JNI
48+
file-size ratio from 2.26× to 1.36× (801,908 → 483,336 bytes). ([1bbc3549](...))
49+
```
50+
The numbers and rationale belong in the commit message, not duplicated in the changelog.
51+
52+
## Rules
53+
54+
- Never invent entries for commits that don't exist — the link is the receipt.
55+
- Never leave a bullet without a commit link.
56+
- One sentence per bullet. If it needs two sentences, split it into two bullets, or the
57+
extra detail belongs in the commit message instead.
58+
- Skip internal-only commits (matches the `release` skill's CHANGELOG sync policy).
59+
60+
## When NOT to use
61+
62+
- Finalizing a release's version header/date, or the compare link at the bottom of the
63+
file — that's the `release` skill's job (its CHANGELOG sync step supersedes this one).
64+
- Writing release notes for work that isn't committed yet.

CHANGELOG.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -14,14 +14,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1414
- `MaskedEncodingEncoder` also tries `vortex.sparse` for a mixed-validity bitmap (fill = the majority value, patches = the minority positions) and keeps it over the raw bitmap when smaller — the patch-index array is compressed through a dedicated `vortex.sequence`/`fastlanes.delta`/FoR/bitpack cascade, so a clustered or regular null pattern (e.g. one null every 10 rows) collapses to a handful of bytes instead of a full bitmap. On the 200k-row, 10-chunk nullable low-cardinality Utf8 benchmark this drops the file a further 20% (107 KB → 86 KB) and the gap to the Rust reference from ~1.45× to ~1.17×. Also: `SequenceEncodingEncoder.encodeCascade` no longer lets `encode`'s "not an arithmetic sequence" exception escape when a stratified sample looked arithmetic but the full data wasn't — it now reports the cascade step as not applicable, like every other encoder's contract requires. (internal)
1515
- `ConstantEncodingEncoder` now accepts `DType.Bool` (previously `DType.Primitive` only), so any all-true or all-false dense Bool column — not just a masked/nullable one — can win `vortex.constant` through the normal per-column cascade, not only through `MaskedEncodingEncoder`'s dedicated check. `MaskedEncodingEncoder`'s validity encoding now delegates to it instead of duplicating the scalar-building logic. `RunEndEncodingEncoder` also gains a `vortex.runend` path for Bool: `MaskedEncodingEncoder` tries it alongside `vortex.sparse` and keeps whichever is smallest — `vortex.runend` wins on long clustered runs of valid/invalid rows (regardless of which value dominates), a shape `vortex.sparse`'s per-flip patch cost handles poorly. (internal)
1616
- `RleEncodingEncoder`/`RleEncodingDecoder` (`fastlanes.rle`) round out Bool coverage: the same FastLanes 1024-row chunked layout the numeric path uses, with a `vortex.bool`-encoded values pool (at most 2 distinct values per chunk) instead of a ptype-width one. `MaskedEncodingEncoder` tries it alongside `vortex.sparse`/`vortex.runend` and keeps the smallest — `DType.Bool` now has every general-purpose encoding family (`vortex.constant`, `vortex.sparse`, `vortex.runend`, `fastlanes.rle`) that `DType.Primitive` has. (internal)
17-
- `CascadingCompressor` now runs `Utf8`/`Binary` columns through the same sample-and-measure encoding competition `Primitive` columns already get, instead of first-match dispatch. `DictEncodingEncoder` is registered ahead of `FsstEncodingEncoder`, so Dict always won regardless of actual output size — confirmed by inspecting a 50k-row, high-cardinality string file and finding `vortex.dict` where `vortex.fsst` is ~40% smaller. On `highCardinalityUtf8_javaVsJni` (50k distinct 6-byte strings) this drops the Java/JNI file-size ratio from 2.26× to 1.36× (801,908 → 483,336 bytes). ([1bbc3549](https://github.com/dfa1/vortex-java/commit/1bbc3549))
18-
- `FsstEncodingDecoder` no longer throws on a valid file whose `vortex.fsst` metadata segment is absent. Proto3 omits fields at their default (zero/`U8`) value, so an all-`U8` metadata message legitimately encodes to zero bytes and the writer omits the segment entirely; the decoder previously treated any absent metadata as corruption. Also adds explicit buffer-count (3) and child-count (≥2) bounds checks, which the removed null-metadata check had been incidentally — and incorrectly — covering for a separate malformed-input case. ([1bbc3549](https://github.com/dfa1/vortex-java/commit/1bbc3549))
19-
- `FsstEncodingEncoder` now writes its symbol table in the order the Rust reference requires (`FSSTData::validate_symbol_lengths`: multi-byte symbols non-decreasing by length, then all length-1 symbols) — a Java-written file with an out-of-order table previously failed to open in the JNI/Rust reader. ([95e0dfb4](https://github.com/dfa1/vortex-java/commit/95e0dfb4))
17+
- `Utf8`/`Binary` columns now use cost-based encoding selection instead of first-match dispatch (Dict no longer always beats FSST regardless of size). ([1bbc3549](https://github.com/dfa1/vortex-java/commit/1bbc3549))
18+
- `FsstEncodingDecoder` no longer throws on valid files with an all-default (empty) `vortex.fsst` metadata segment. ([1bbc3549](https://github.com/dfa1/vortex-java/commit/1bbc3549))
19+
- `FsstEncodingEncoder` writes symbol tables in the order the Rust reference requires; previously an out-of-order table could fail to open in the JNI/Rust reader. ([95e0dfb4](https://github.com/dfa1/vortex-java/commit/95e0dfb4))
2020

2121
### Performance
2222

23-
- `FsstEncodingEncoder` replaces its fixed 2-byte bigram symbol table with iterative FSST-paper-style training (longest-match-first parsing, gain-ranked candidate refinement over several passes), producing variable-length 1–8 byte symbols instead of a fixed 2-byte cap. Also narrows the `uncompressed_lengths`/`codes_offsets` metadata buffers to the smallest ptype that fits the actual data (previously always 4-byte `I32`) using wire-format fields that already existed for this. ([95e0dfb4](https://github.com/dfa1/vortex-java/commit/95e0dfb4), [1bbc3549](https://github.com/dfa1/vortex-java/commit/1bbc3549))
24-
- `FsstEncodingEncoder`'s symbol-match lookup replaces a `HashMap<SymbolCandidate, Integer>` (boxing a record key on every probe) with a flat open-addressing table over primitive arrays; training sampling switches from "first N rows" to the same stratified stride sampling `CascadingCompressor` uses elsewhere, avoiding bias on sorted/clustered input. ~18–23% faster encode on incompressible data, ~10% on repetitive text. ([4a3fa0a9](https://github.com/dfa1/vortex-java/commit/4a3fa0a9))
23+
- `FsstEncodingEncoder`: iterative symbol-table training (variable-length 1–8 byte symbols, was a fixed 2-byte bigram scheme) and narrower metadata buffer ptypes. ([95e0dfb4](https://github.com/dfa1/vortex-java/commit/95e0dfb4), [1bbc3549](https://github.com/dfa1/vortex-java/commit/1bbc3549))
24+
- `FsstEncodingEncoder`: non-boxing symbol-match lookup and stratified training-sample selection. ([4a3fa0a9](https://github.com/dfa1/vortex-java/commit/4a3fa0a9))
2525

2626
## [0.12.2] — 2026-07-12
2727

0 commit comments

Comments
 (0)