fix(fpss): bound per-session delta-decode state against non-conformant peers#889
Merged
Conversation
…t peers The delta-decode maps (prev, ohlcvc, field_counts) are keyed by raw wire contract id and only reset at START/STOP/RESTART boundaries and on reconnect. A non-conformant peer that streams ever-incrementing contract ids within a single no-STOP session would grow all three maps without limit. Add a generous per-session distinct-contract cap: on overflow the maps are cleared once with a single warn and re-seeded from the current tick, which decodes as a fresh absolute baseline, so no value is corrupted. A conformant session never approaches the cap and retains every live contract. While here, document why the recorded field width is fixed at the first absolute row: the trade-format width is a server-wide session constant, not a per-row property, and later FIT delta rows carry only the changed-field count, so they cannot re-derive the width. The decode buffer is always full width with unused slots zero-filled, so a stale width can only under-report which trailing fields a caller reads, never read out of bounds. Also stop decoding flat-file INDEX entries with the option contract layout. The flat-file service publishes no INDEX dataset, so the request layer rejects every Index pair before any blob is fetched and the walker is never reached with an INDEX section. No vendor-confirmed INDEX entry layout exists, so the walker now returns a typed unsupported error rather than borrowing the option layout and risking a misparse if a future format differs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two correctness-hardening fixes in the core crate from an adversarial decoder audit.
Bound per-session delta-decode state
The FIT delta-decode maps (
prev,ohlcvc,field_counts) are keyed by raw wire contract id and only reset at START/STOP/RESTART boundaries and on reconnect. There is no per-session entry cap, so a non-conformant peer that streams ever-incrementing contract ids within a single no-STOP session would grow all three maps without limit. The connection is SPKI-pinned to the vendor host, so this is not reachable in practice, but an institutional-grade decoder should still bound it.A generous per-session distinct-contract cap now guards the insert path. On overflow the maps are cleared once with a single warn and re-seeded from the current tick, which decodes as a fresh absolute baseline, so no value is corrupted. A conformant session resets at every market boundary and never approaches the cap, so it retains every live contract.
The recorded field width is left fixed at the first absolute row, now with a comment explaining why this is correct: the trade-format width is a server-wide session constant, not a per-row property, and later FIT delta rows carry only the changed-field count, so they cannot re-derive the width. The decode buffer is always full width with unused slots zero-filled, so a stale width can only under-report which trailing fields a caller reads, never read out of bounds.
Stop decoding flat-file INDEX entries with the option layout
The INDEX walker decoded
SecType::Indexwith the option contract layout, which expects arightbyte. The flat-file service publishes no INDEX dataset, so the request layer rejects every Index pair before any blob is fetched and the walker is never reached with an INDEX section. This was dead defensive code that would have misparsed a real index layout if it differed from options. No vendor-confirmed INDEX entry layout exists, so the walker now returns a typed unsupported error rather than borrowing the option layout.Tests
delta_state_bounds_unbounded_distinct_contract_idsfeeds well past the cap and asserts every map stays bounded.delta_state_normal_session_retains_all_contractsasserts a normal session keeps every distinct contract.index_sec_type_is_rejected_as_unsupportedasserts the typed error on an INDEX section.cargo test -p thetadatadx --lib fpssand--lib flatfilesboth pass;cargo fmt --all -- --checkclean;cargo clippy -p thetadatadx --libclean.🤖 Generated with Claude Code