Skip to content

Commit 6dec8f6

Browse files
dfa1claude
andcommitted
docs(todo): docs-gating plan; duplicate-column-name bug findings
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 8012c13 commit 6dec8f6

1 file changed

Lines changed: 25 additions & 0 deletions

File tree

TODO.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,19 @@ Per-encoding gotchas:
6767
## Build
6868

6969
- [ ] use JPMS, watch out for "dfa1" in package name
70+
- [ ] **Docs need a compiler** — living prose (CLAUDE.md, `docs/*.md`) drifts from code with no
71+
gate (2026-07-04 found: phantom `register(ExtensionDecoder)`, stale "not yet implemented" on
72+
MASKED/PATCHED, dead service-file path, stale FQNs after package moves). Tiered plan:
73+
1. Markdown link checker in CI (relative links; the `docs/adr``adr/` move rewrote ~40 by hand).
74+
2. `DocsConsistencyTest` (integration module): extract backticked FQNs / `Class#member` refs /
75+
`META-INF/services` paths from living docs, assert existence via reflection. Historical files
76+
(`adr/`, released CHANGELOG sections) exempt by policy.
77+
3. Golden-test or generate enumerable tables (encodings/extensions/layout ids in
78+
`docs/reference.md`, `docs/compatibility.md`) from `EncodingId.WellKnown`/`LayoutId.WellKnown`/
79+
service manifests — a declared capability the code lacks is a bug, in both directions.
80+
4. Compile ` ```java ` blocks from living docs (defer until 1–3 pay rent).
81+
Plus: standing stale-docs dimension in the vortex-reviewer agent (grep living docs for
82+
identifiers a diff renames/moves/deletes).
7083

7184
## Tooling
7285

@@ -82,6 +95,18 @@ Per-encoding gotchas:
8295
- See [ADR-0008](adr/0008-domain-primitives-unsigned-integers.md) and https://dfa1.github.io/articles/rethink-domain-primitives-with-valhalla
8396
- Candidates: `PType` integer kinds, buffer offsets, row indices, byte lengths
8497
- Goal: type-safety at zero cost (value class = no heap alloc, no boxing)
98+
- [ ] **Column identity: duplicate/empty field names (BUG + ADR candidate)** — measured 2026-07-04:
99+
`""` and duplicate field names round-trip through our writer/reader (the `DType.Struct` record
100+
constructor validates nothing; only `StructBuilder` rejects duplicates, and the proto decode path
101+
bypasses it — it can also desync `fieldNames`/`fieldTypes` sizes). The Rust reference *documents*
102+
duplicates as legal with **first**-match name resolution (`vortex-array/src/dtype/struct_.rs`),
103+
but our `Chunk`'s `Map<String, Array>` silently drops columns (two `dup` fields → one entry) and
104+
resolves **last**-wins — a legal Rust file loses data in our reader. Also: duplicate-name write
105+
produced two chunks from one `writeChunk` (unexplained, investigate).
106+
Fix direction (ADR): `Chunk` columns become ordered (list/parallel arrays) with name lookup =
107+
first match, mirroring Rust; `columns()` map API rethought. Revisit `ColumnName` as a domain
108+
primitive here — note the honest limit: `""` is wire-legal, so the per-value invariant is thin;
109+
the real invariants are structural (ordered fields, first-match resolution, names/types arity).
85110

86111
## Compute
87112

0 commit comments

Comments
 (0)