Skip to content

fix(gguf): restore 32-byte scaling on data_offset (chatgpt-codex catch on PR #367)#368

Merged
AdaWorldAPI merged 1 commit into
mainfrom
claude/gguf-data-offset-fix
May 13, 2026
Merged

fix(gguf): restore 32-byte scaling on data_offset (chatgpt-codex catch on PR #367)#368
AdaWorldAPI merged 1 commit into
mainfrom
claude/gguf-data-offset-fix

Conversation

@AdaWorldAPI

Copy link
Copy Markdown
Owner

Summary

Fixes a regression introduced by PR #367's sprint-log-8 agent W4: the manual_div_ceil rewrite on gguf_thinking_styles.rs:360 dropped the * 32 byte-scaling, turning a byte offset into a block COUNT (off by ×32).

chatgpt-codex P2 catch on PR #367 review thread:

When this example parses any real GGUF file, data_offset is later used as the absolute base for SeekFrom::Start(h.data_offset + t.offset), but this replacement now returns the number of 32-byte blocks rather than the byte offset rounded up to a 32-byte boundary. For example a header ending near byte 3200 now produces about 101, so tensor reads seek back into the header/metadata and decode garbage or fail; the expression needs to multiply the div_ceil(32) result by 32 as the other GGUF parsers in this commit still do.

The bug

// PR #367 W4 commit (BUG — drops * 32):
data_offset: (pos + 31).div_ceil(32),

// PR #367 W2/W3/W7 commits (CORRECT — keeps * 32):
data_offset: pos.div_ceil(32) * 32,

Both look superficially like manual_div_ceil rewrites, but the original (pos + 31) / 32 * 32 had TWO operations:

  1. Round-up to next 32-byte boundary (div_ceil(32))
  2. Multiply by 32 to get the byte offset

W4's "fix" preserved only the first.

Audit across the workspace

Verified all four GGUF parsers in bgz-tensor/examples/ after this PR:

File Line Code State
gguf_euler_fold.rs 373 pos.div_ceil(32) * 32
gguf_families.rs 337 pos.div_ceil(align) * align
gguf_thinking_styles.rs 360 pos.div_ceil(32) * 32 ✓ this PR
gamma_phi_gguf.rs 356 pos.div_ceil(32) * 32

Verification

  • rustup run 1.95.0 cargo clippy --workspace --all-targets -- -D warnings exits 0
  • rustup run 1.95.0 cargo fmt --all --check exits 0
  • No other files affected — the diff is 1 line of code + 8 lines of comment explaining the trap

Process note

The 12-agent fleet's per-file isolation enabled parallel work but missed the cross-file invariant that "all GGUF parsers in the workspace must produce the same data_offset shape". W4's per-file clippy verification passed; the bug only surfaces when comparing W4's output against the parallel parsers in W2/W3/W7's files.

Future linter-sweep fleets that touch a pattern present in multiple sibling files should either:

  • Add a cross-file diff to the meta-orchestrator's verification step, OR
  • Be reviewed against a single reviewer (or another LLM) who can spot the divergence

Codex review was the safety net here. Worth keeping that loop in place.


Generated by Claude Code

The 1.95 clippy sweep in PR #367 (sprint-log-8 agent W4) flagged
`(pos + 31) / 32 * 32` as `manual_div_ceil` and rewrote it to
`(pos + 31).div_ceil(32)` — losing the `* 32` byte-scaling and
turning a byte-rounded offset into a block COUNT (off by ×32).

chatgpt-codex P2 catch on PR #367 review thread.

Symptom: every GGUF header parsed by `gguf_thinking_styles.rs` returned
data_offset ≈ N/32 where it should have returned N rounded up to the
next 32-byte boundary. `SeekFrom::Start(h.data_offset + t.offset)`
then landed in the header/metadata area and decoded garbage.

Fix: `pos.div_ceil(32) * 32` (matches gguf_euler_fold.rs:373 +
gguf_families.rs:337 + gamma_phi_gguf.rs:356 which all kept the
`* 32` scaling correctly).

Audit summary across the workspace's GGUF parsers post-PR #367:
- gguf_euler_fold.rs:373        `pos.div_ceil(32) * 32`        ✓
- gguf_families.rs:337          `pos.div_ceil(align) * align`  ✓
- gguf_thinking_styles.rs:360   was buggy (this PR)            → ✓ fix
- gamma_phi_gguf.rs:356         `pos.div_ceil(32) * 32`        ✓

Workspace clippy on 1.95.0 still clean; fmt still clean. No CI behavior
change for non-GGUF code paths.

Process note: the 12-agent fleet's per-file isolation was good for
parallelism but missed the cross-file invariant that "all GGUF parsers
in the workspace must produce the same data_offset". A future linter
sweep that touches a pattern present in multiple sibling files should
either cross-check the result or be reviewed against a single reviewer
who can spot the divergence. The codex review was the safety net here.
@AdaWorldAPI AdaWorldAPI merged commit 91b0ebd into main May 13, 2026
5 checks passed
AdaWorldAPI pushed a commit that referenced this pull request May 14, 2026
Channel was bumped to 1.95.0 in PR #367 but the explanatory comment
above it still said "Pinned to 1.94.1 ... 1.95 turned several
previously-safe patterns into denied lints ... without sufficient
value to justify the churn." Pure documentation drift — the bump
already happened, the lint debt was closed by #367 + #368, and the
comment now describes a state that has not been true for ~a day.

Update the comment to reflect the actual 1.95.0 pin, cross-ref
PR #367, name the clippy lints that came with 1.95, and keep the
"never auto-track stable" rule.
AdaWorldAPI added a commit that referenced this pull request May 28, 2026
fix(gguf): restore 32-byte scaling on data_offset (chatgpt-codex catch on PR #367)
AdaWorldAPI pushed a commit that referenced this pull request May 28, 2026
Channel was bumped to 1.95.0 in PR #367 but the explanatory comment
above it still said "Pinned to 1.94.1 ... 1.95 turned several
previously-safe patterns into denied lints ... without sufficient
value to justify the churn." Pure documentation drift — the bump
already happened, the lint debt was closed by #367 + #368, and the
comment now describes a state that has not been true for ~a day.

Update the comment to reflect the actual 1.95.0 pin, cross-ref
PR #367, name the clippy lints that came with 1.95, and keep the
"never auto-track stable" rule.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants