refactor: Wave 3 — typed SubscriptionTier + ArcSwap streaming state + layout moves + LOW batch (refs #500)#507
Merged
Merged
Conversation
… layout moves + LOW batch (refs #500) E. Typed `SubscriptionTier` enum ================================ `crates/thetadatadx/src/mdds/tier.rs` introduces a four-variant `SubscriptionTier` enum (`Free`, `Value`, `Standard`, `Pro`) replacing the prior `Option<i32>` representation of the `stock_tier` and `options_tier` fields on `MddsClient`. The discriminant mirrors the wire byte; `from_wire(i32)` decodes (returning `None` for unknown values rather than silently coercing); `max_concurrent_requests` codifies the `2^tier` semaphore semantics. Re-exported as `thetadatadx::SubscriptionTier`. The wire-side `auth::nexus::AuthUser` keeps its raw `Option<i32>` fields so deserialization stays infallible for unknown future tiers; the typed enum is the post- decode in-memory shape callers see. Unit tests cover the round-trip, the unknown-value rejection, and the powers-of-two semaphore math (1, 2, 4, 8). F. `ArcSwap<StreamingSlot>` state machine ========================================= The streaming side of `ThetaDataDx` collapses to a single `ArcSwap<StreamingSlot>` cell walking `Idle -> Live -> Stopped -> Live -> Stopped`. The previous trio of coordinated fields — `Mutex<Option<FpssClient>>`, `Mutex<Option<StreamingDispatcher>>`, `AtomicBool was_streaming` — went away. Read paths (`is_streaming`, `connection_status`, `with_streaming`, every per-subscription forwarder) collapse to one atomic load on the slot and a borrow of the contained `Arc<FpssClient>`. Lifecycle paths retain serial semantics through an rcu-CAS install and an atomic swap-to-`Stopped`, with the previous `Live` payload (if any) running the FPSS-then-dispatcher shutdown sequence. The `dispatcher` field sits in a `Mutex<Option<...>>` inside `Live` because `StreamingDispatcher::shutdown` consumes by value; the lock is hit only by `stop_streaming` and `dropped_event_count`, never by hot-path readers. Adds `arc-swap` 1.7 to the runtime dependencies. A state-machine walk through `Idle -> Live -> Stopped -> Live -> Stopped` and a CAS-race test covering the rejected double-install live alongside the existing `reconnect_streaming` regression tests. The `Drop` contract is documented in place: stop_streaming is idempotent because only a `Live` slot triggers the shutdown sequence. G. Layout moves =============== Top-level mdds-specific modules move under `mdds/`: endpoint.rs -> mdds/endpoint_args.rs (rename clarifies args-only) macros.rs -> mdds/macros.rs registry.rs -> mdds/registry.rs validate.rs merged into mdds/validate.rs wire_semantics.rs -> mdds/wire_semantics.rs `mdds/validate.rs` now hosts the canonical two-arg validators (date / expiration / strike / symbol / interval / right / year) plus a `validate_date_required` single-arg adapter the `parsed_endpoint!` macro expansion needs. The macro switches to the fully-qualified `$crate::mdds::validate::validate_date_required` path so it expands cleanly from generated endpoint bodies; the generator emit (`build_support/endpoints/render/mdds.rs`) follows. `unified.rs` is renamed to `client.rs` so the filename matches the primary type (`ThetaDataDx`). The frames module folds into a directory: `frames.rs` + `frames_generated.rs` -> `frames/mod.rs` + `frames/generated.rs`. The build-time generator paths (`build_support/ticks/mod.rs`, `build_support/ticks/rust_frames.rs`) follow. The `tdbe::types::*_generated.rs` trio segregates under `tdbe::types::generated/`: `enums_endpoint.rs`, `tick.rs`, `tick_layout_asserts.rs`, with a `mod.rs` documenting the directory. The hand-written siblings keep their `include!` sites (now `include!("generated/...rs")`); `mod generated;` is declared privately in `types/mod.rs`. The build-time generator paths (`build_support/endpoints/render/sdk_files.rs`, `build_support/ticks/tdbe_structs.rs`, `build_support/ticks/mod.rs`) follow. Re-exports in `lib.rs` keep the public surface stable: pub use mdds::endpoint_args as endpoint; pub use mdds::registry::{by_category, find, ..., EndpointMeta, ENDPOINTS, ...}; pub use client::{ConnectionStatus, SubscriptionInfo, ThetaDataDx}; `thetadatadx::endpoint::*` and `thetadatadx::ENDPOINTS` / `EndpointMeta` continue to resolve. Build-script `[path = ...]` mounts (`build_support/mod.rs`, `crates/thetadatadx/src/bin/generate_sdk_surfaces.rs`) follow the file moves. `git mv` was used everywhere file blame is meaningful so history follows the rename. H. LOW batch ============ * 3.1 (`mdds.connect_timeout_secs`) — already shipped in Wave 2. * 3.2 (`extract_*_column` return type) — kept as `Vec<Option<T>>`. The three helpers are public surface, exercised by benches, integration tests, the macro-driven list endpoints, and the Polars / Arrow column projections. Switching to `impl Iterator<Item = Option<T>>` would force every caller (including generated FFI bindings) to deal with the iterator shape and would lose the "missing-header -> empty result" early return the warn-log path relies on. The Vec allocation is one per column per call, never on a per-tick path. * 3.5 (`FpssConnectArgs` struct) — deferred to Wave 4 B4 as a breaking signature change. Not landed here. * 3.9 (`Drop::drop` idempotency on `ThetaDataDx`) — addressed in F. The state machine guarantees `stop_streaming` runs the FPSS / dispatcher shutdown at most once; `Drop::drop`'s doc comment now records the contract in-place. * 3.10 (`#[allow(dead_code)]` on `flatfiles::framing::msg`) — removed. Every one of the ten `u16` wire-code constants is genuinely used by `flatfiles::request` and `flatfiles::session`. The attribute was a false positive; clippy stays green. Version bump: 8.0.36 -> 8.0.37 across every crate manifest, `sdks/typescript/package.json` (+ optionalDependencies, + the three per-platform `npm/<platform>/package.json` files), Cargo.lock, and the matching CHANGELOG entry mirrored to `docs-site/docs/changelog.md`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Wave 3 of the #500 refactor sweep. Four items, one squashable commit, version
bump 8.0.36 → 8.0.37, CHANGELOG entry mirrored to
docs-site/. No behaviorchange at the wire / FPSS / MDDS layer — every modification is internal
shape, public re-exports preserve the existing surface.
E. Typed
SubscriptionTierenumcrates/thetadatadx/src/mdds/tier.rsintroduces a four-variant enum(
Free,Value,Standard,Pro) replacing the priorOption<i32>representation of
stock_tierandoptions_tieronMddsClient.Option<i32>Option<SubscriptionTier>SubscriptionTier::from_wire(i32) -> Option<Self>2^tierliteral in two placesSubscriptionTier::max_concurrent_requests(self)thetadatadx::SubscriptionTierThe wire-side
auth::nexus::AuthUserkeeps its rawOption<i32>fields sodeserialization stays infallible for unknown future tiers; the typed enum is
the post-decode in-memory shape callers see. Unit tests cover the round-trip,
the unknown-value rejection (
from_wire(99) == None), and the powers-of-twosemaphore math (1, 2, 4, 8).
F.
ArcSwap<StreamingSlot>state machineStreaming side of
ThetaDataDxcollapses to a singleArcSwap<StreamingSlot>cell walking
Idle → Live → Stopped → Live → Stopped.Mutex<Option<FpssClient>>+Mutex<Option<StreamingDispatcher>>+AtomicBool was_streamingArcSwap<StreamingSlot>is_streamingmatches!connection_statusAtomicBoolloadwith_streamingstop_streamingLivetriggers shutdownAdds
arc-swap1.7 as a runtime dep. A state-machine walk throughIdle → Live → Stopped → Live → Stoppedand a CAS-race test covering therejected double-install live alongside the existing
reconnect_streamingregression tests.
G. Layout moves
Top-level mdds-specific modules move under
mdds/:crates/thetadatadx/src/endpoint.rscrates/thetadatadx/src/mdds/endpoint_args.rscrates/thetadatadx/src/macros.rscrates/thetadatadx/src/mdds/macros.rscrates/thetadatadx/src/registry.rscrates/thetadatadx/src/mdds/registry.rscrates/thetadatadx/src/validate.rscrates/thetadatadx/src/mdds/validate.rscrates/thetadatadx/src/wire_semantics.rscrates/thetadatadx/src/mdds/wire_semantics.rscrates/thetadatadx/src/unified.rscrates/thetadatadx/src/client.rscrates/thetadatadx/src/frames.rs+frames_generated.rscrates/thetadatadx/src/frames/{mod,generated}.rscrates/tdbe/src/types/{enums_endpoint,tick,tick_layout_asserts}_generated.rscrates/tdbe/src/types/generated/{enums_endpoint,tick,tick_layout_asserts}.rsmdds/validate.rsnow hosts the canonical two-arg validators(date / expiration / strike / symbol / interval / right / year) plus a
validate_date_requiredsingle-arg adapter theparsed_endpoint!macroexpansion needs. The macro switches to the fully-qualified
$crate::mdds::validate::validate_date_requiredpath so it expands cleanlyfrom generated endpoint bodies; the generator emit
(
build_support/endpoints/render/mdds.rs) follows.tdbe::types::generated/ships a documentingmod.rs; the hand-writtensiblings keep their
include!sites (nowinclude!("generated/...rs")).mod generated;is declared privately intypes/mod.rsso the directory ispart of the module tree without re-exposing the generator output.
Re-exports in
lib.rspreserve the public surface:git mvwas used everywhere file blame is meaningful. Build-script[path = ...]mounts (build_support/mod.rs,crates/thetadatadx/src/bin/generate_sdk_surfaces.rs) follow the file moves;the sdk-surfaces generator (
build_support/ticks/,build_support/endpoints/render/sdk_files.rs) emits to the new paths.H. LOW batch
mdds.connect_timeout_secsextract_*_columniterator returnVec<Option<T>>FpssConnectArgsstructDrop::dropidempotency onThetaDataDxstop_streamingruns the FPSS / dispatcher shutdown at most once;Drop::drop's doc comment records the contract#[allow(dead_code)]onflatfiles::framing::msgu16wire-code constants are genuinely used byflatfiles::requestandflatfiles::session; the attribute was a false positiveTest plan
cargo fmt --all -- --checkcargo clippy --workspace --all-targets -- -D warningscargo test --workspace(every test suite green; 6 new unit tests added — 4 forSubscriptionTier, 2 for theArcSwapstate machine)cargo deny checkcargo run -p thetadatadx --bin generate_sdk_surfaces --features config-file -- --check(no surface drift)cargo check --manifest-path tools/{cli,mcp,server}/Cargo.toml --lockedcargo check --manifest-path sdks/{python,typescript}/Cargo.toml --lockedcargo clippy --manifest-path tools/mcp/Cargo.toml --all-targets -- -D warningscargo test --manifest-path tools/mcp/Cargo.toml --no-runpython3 scripts/check_version_sync.py→version sync: ok