fix: silence find_header warn on optional Greeks columns (#472)#473
Merged
Conversation
Closes #472. The v3 server splits the Greeks column set across `option_*_greeks_*_order` endpoints — `_first_order` ships seven Greeks, `_second_order` ships five, and `_third_order` ships four — but the shared `GreeksTick` schema in `tick_schema.toml` carries the full 23-Greek union. Every absent column on a subset response triggered `tracing::warn!("expected column header not found in DataTable", header=…)` from `find_header` in `crates/thetadatadx/src/decode.rs`, so a single `option_snapshot_greeks_third_order` call printed eight warn lines per row before any user-visible decoding finished. The reporter on Issue #472 saw the spam on every Python `option_snapshot_greeks_first_order` call. Demote the diagnostic to `tracing::trace!` so it stays reachable via `RUST_LOG=thetadatadx=trace` for genuine schema-drift investigations but stops competing with stderr on routine subset calls. Required-column drift continues to surface as a typed `Error::MissingRequiredHeader` from the generated parser, so the silenced path is strictly the documented optional-column case. Pin the per-endpoint vendor schema in the `GreeksTick` doc-comment against the upstream OpenAPI capture (`scripts/upstream_openapi.yaml`), so the column-list / endpoint mapping is visible from the schema file itself rather than scattered across endpoint metadata. Add three regression tests in `decode.rs::tests` that drive `parse_greeks_ticks` against `_first_order`, `_second_order`, and `_third_order` wire shapes. Each test asserts bit-exact decoded values for every wire-present column and `0.0` defaults for the documented gaps. A future regression of `find_header` back to `tracing::warn!`, or any column-list drift in either direction, surfaces here as a behavioural test failure rather than as live log spam. Workspace 8.0.25 → 8.0.26, tdbe 0.12.6 → 0.12.7. CHANGELOG + docs-site/docs/changelog.md kept byte-identical. Local CI gate: cargo fmt --all -- --check # clean cargo clippy --workspace --all-targets -- -D warnings # clean cargo test --workspace # 0 failures cargo deny check # advisories+bans+licenses+sources ok generate_sdk_surfaces --check # no drift Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #472.
crates/thetadatadx/src/decode.rs::find_headeremitted atracing::warn!("expected column header not found in DataTable", header=…)on every optional column missing from a wire response. The Greeks family
splits the column set across endpoints —
_first_orderships seven Greeks,_second_orderships five,_third_orderships four — but the sharedGreeksTickschema carries the full 23-Greek union. The reporter saweight warn lines per row on a single
option_snapshot_greeks_first_ordercall from Python. Demote the diagnostic to
tracing::trace!so genuineschema-drift investigations are still reachable via
RUST_LOG=thetadatadx=tracewhile routine subset calls stay clean.Required-column drift continues to surface as a typed
Error::MissingRequiredHeaderfrom the generated parser, so thesilenced path is strictly the documented optional-column case.
Pin the per-endpoint vendor column lists for the four Greeks families
in the
GreeksTickschema doc-comment, against the upstream OpenAPIcapture in
scripts/upstream_openapi.yaml. The struct layout itselfis unchanged — every Greeks endpoint still returns
Vec<GreeksTick>—so the SSOT pickup is doc-only and there is zero generated-SDK drift.
Add three regression tests in
decode.rs::testsdrivingparse_greeks_ticksagainst_first_order,_second_order, and_third_orderwire shapes. Each asserts bit-exact decoded values forthe wire-present columns and
0.0defaults for the documented gaps,so a future regression of
find_headerback totracing::warn!—or any column-list drift in either direction — surfaces as a
behavioural test failure rather than as live log spam.
Per-endpoint vendor schemas pinned to:
_first_order: items_option_snapshot_greeks_first_order — delta, theta, vega, rho, epsilon, lambda + IV pair_second_order: items_option_snapshot_greeks_second_order — gamma, vanna, charm, vomma, veta + IV pair_third_order: items_option_snapshot_greeks_third_order — speed, zomma, color, ultima + IV pair (vera not in third-order)_all: items_option_snapshot_greeks_all — full 23-Greek union_implied_volatility: items_option_snapshot_greeks_implied_volatility — handled by IvTickWorkspace 8.0.25 → 8.0.26, tdbe 0.12.6 → 0.12.7. CHANGELOG +
docs-site/docs/changelog.mdkept byte-identical.Follow-up: per-endpoint typed structs
The bug-report comment thread also asked for per-endpoint typed
return structs (
GreeksFirstOrderTick,GreeksSecondOrderTick,GreeksThirdOrderTick) so the type system reflects the wire schema.Surveying the surface, that change ripples through ~30 codegen
tables (build_support helpers, FFI macros with hand-tuned padding,
hand-written C++ headers, Go FFI mirrors, three SDK regenerations,
the
EndpointOutputenum + every consumer in CLI / MCP / server /ffi). It deserves its own issue + PR rather than being bolted onto a
warn-silence fix; the schema doc-comment in this PR is the
prerequisite that nails down the column lists per endpoint, so the
follow-up can map each subset to a typed struct without re-deriving
the wire shape from scratch.
Test plan
cargo fmt --all -- --checkcleancargo clippy --workspace --all-targets -- -D warningscleancargo test --workspace0 failures across all cratescargo deny checkclean (advisories + bans + licenses + sources)cargo run -p thetadatadx --bin generate_sdk_surfaces --features config-file -- --checkno driftdecode::testsexercising thethree Greeks subset wire shapes
v8.0.26