Skip to content

refactor(decode): split god-file into mdds/decode/{error,headers,transport,extract,cell,v3} + lift tdbe::time (refs #500)#505

Closed
userFRM wants to merge 1 commit into
mainfrom
refactor/wave2-decode-split
Closed

refactor(decode): split god-file into mdds/decode/{error,headers,transport,extract,cell,v3} + lift tdbe::time (refs #500)#505
userFRM wants to merge 1 commit into
mainfrom
refactor/wave2-decode-split

Conversation

@userFRM
Copy link
Copy Markdown
Owner

@userFRM userFRM commented May 7, 2026

Summary

Wave 2 structural refactor of crates/thetadatadx/src/decode.rs (2177 LoC).
No public-API change; thetadatadx::decode::* paths are preserved via
pub use mdds::decode at the crate root. Eastern-time / DST primitives
become a canonical tdbe::time module reused by mdds, fpss latency, and
flatfiles.

Module surface

Old decode.rs lines New module Concern
12-73 mdds/decode/error.rs DecodeError enum + observed_name diagnostic helper
75-142 mdds/decode/headers.rs HEADER_ALIASES v3 <-> schema map + find_header lookup
330-409 mdds/decode/transport.rs decompress_response / decode_data_table (zstd thread-local pool)
411-481 mdds/decode/extract.rs extract_{number,text,price}_column projections
296-328, 501-694, 704-713, generated mod mdds/decode/cell.rs per-cell strict decoders (row_*) + macro-generated parser surface
715-1037 mdds/decode/v3.rs hand-written v3 parsers + parse_iso_date/parse_time_text/calendar consts
144-294 (and tests) crates/tdbe/src/time.rs Eastern-time + DST primitives (lifted to canonical tdbe::time)

Lifted to tdbe::time

  • eastern_offset_ms
  • march_second_sunday_utc
  • november_first_sunday_utc
  • april_first_sunday_utc
  • october_last_sunday_utc
  • civil_to_epoch_days
  • timestamp_to_ms_of_day
  • timestamp_to_date

tdbe::latency now reuses these helpers, removing the duplicate civil-date
arithmetic that previously lived alongside the latency wrapper. Generator
templates (build_support/ticks/templates/parser/eod_{date,num,num64}.rs.tmpl)
updated to call tdbe::time::* directly.

Public API confirmation

Zero public surface change. thetadatadx::decode::{DecodeError, decompress_response, decode_data_table, extract_*_column, parse_*_ticks, parse_option_contracts_v3, parse_calendar_days_v3, CALENDAR_STATUS_*}
all resolve through pub use mdds::decode at the crate root. External
consumers (sdks/python/src/decode_bench.rs, crates/thetadatadx/benches/bench_decode.rs,
crates/thetadatadx/tests/test_decode_captures.rs) compile unchanged.

Versioning

  • thetadatadx: 8.0.33 -> 8.0.34 (patch; structural refactor only)
  • tdbe: 0.12.9 -> 0.12.10 (additive time module)

Test plan

  • cargo fmt --all -- --check
  • cargo clippy --workspace --all-targets -- -D warnings
  • cargo test --workspace (453 lib tests + 9 integration tests pass)
  • cargo deny check
  • cargo run -p thetadatadx --bin generate_sdk_surfaces --features config-file -- --check
  • cargo check --manifest-path tools/mcp/Cargo.toml --locked
  • cargo clippy --manifest-path tools/mcp/Cargo.toml --all-targets -- -D warnings
  • cargo test --manifest-path tools/mcp/Cargo.toml --no-run
  • cargo check --manifest-path tools/server/Cargo.toml --locked
  • cargo check --manifest-path sdks/python/Cargo.toml --locked
  • cargo check --manifest-path sdks/typescript/Cargo.toml --locked
  • python3 scripts/check_version_sync.py

Refs #500.

…sport,extract,cell,v3} + lift tdbe::time (refs #500)

Wave 2 structural refactor of `crates/thetadatadx/src/decode.rs`. No public
API change; existing `thetadatadx::decode::*` paths preserved via
`pub use mdds::decode` at the crate root.

Module split:

* `mdds/decode/error.rs`     -- `DecodeError` enum + `observed_name` helper
* `mdds/decode/headers.rs`   -- `HEADER_ALIASES` v3 <-> schema map + `find_header`
* `mdds/decode/transport.rs` -- zstd `decompress_response` / `decode_data_table`
* `mdds/decode/extract.rs`   -- `extract_{number,text,price}_column` projections
* `mdds/decode/cell.rs`      -- per-cell strict decoders + generated parser surface
* `mdds/decode/v3.rs`        -- `parse_option_contracts_v3`, `parse_calendar_days_v3`,
                                 `parse_iso_date`, `parse_time_text`, calendar consts

Eastern-time + DST primitives (`eastern_offset_ms`, `march_second_sunday_utc`,
`november_first_sunday_utc`, `april_first_sunday_utc`, `october_last_sunday_utc`,
`civil_to_epoch_days`, `timestamp_to_ms_of_day`, `timestamp_to_date`) lifted
from `decode.rs` into the new `tdbe::time` module. `tdbe::latency` now reuses
the canonical implementation, eliminating the duplicate civil-date arithmetic
that previously lived alongside the latency wrapper.

Generator templates updated to call `tdbe::time::timestamp_to_*` directly
(`build_support/ticks/templates/parser/eod_{date,num,num64}.rs.tmpl`).

Versioning:
* thetadatadx: 8.0.33 -> 8.0.34 (patch; structural refactor only)
* tdbe:        0.12.9 -> 0.12.10 (additive `time` module)
@userFRM
Copy link
Copy Markdown
Owner Author

userFRM commented May 7, 2026

Closing — refolding all 4 Wave 2 god-file splits into a single PR per workflow discipline (one wave = one PR). Will reopen as a single coordinated commit covering decode + protocol + config + conditions.

@userFRM userFRM closed this May 7, 2026
@userFRM userFRM deleted the refactor/wave2-decode-split branch May 7, 2026 03:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant