Skip to content

Commit 1f4d2f0

Browse files
userFRMclaude
andcommitted
docs: comprehensive documentation audit -- 14 fixes across 13 files
Findings and fixes: - SECURITY.md: version table said 4.5.x current, updated to 5.2.x with accurate support matrix including Go price formula bug note - CHANGELOG.md: missing [5.1.1] release entry (link ref existed but no section) -- added entry documenting tdbe 0.2.0 crates.io publish fix - Go streaming docs: 4 files still showed PriceToF64() on pre-decoded float64 fields (would produce wrong results since v5.2). Updated streaming.md, streaming/index.md, streaming/reconnection.md examples to use pre-decoded fields directly - options.md: wildcard docs said right must be "C" or "P" -- added "both" as valid value (from normalize_right) - Go/C++ SDK READMEs: Config section listed mixed-language syntax (Config.stage()/StageConfig()/Config::stage()) -- each now shows only its own language - Go/C++ SDK READMEs: GreeksTick type missing Vera field (present in code since v4.5.0) -- added - C++ README: Greeks standalone struct missing vera field -- added - docs/architecture.md, jvm-deviations.md, reverse-engineering.md, api-reference.md: macro name define_endpoint! renamed to parsed_endpoint! (actual current macro name) - streaming/latency.md: replaced ASCII diagram with Mermaid sequence diagram and latency breakdown flowchart. Added network physics section with fiber optic speed-of-light calculations for 8 locations. Added dev server warning (historical replay timestamps are from the past, not valid for latency measurement) - streaming/index.md: replaced ASCII architecture diagram with Mermaid - option/index.md: "At-Time OHLC" link text corrected to "At-Time Quote" (file contains option_at_time_quote, not OHLC) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent e68c94a commit 1f4d2f0

13 files changed

Lines changed: 99 additions & 38 deletions

File tree

CHANGELOG.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,12 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
3333

3434
- **Go `PriceToF64` formula** was `value / 10^pt` instead of `value * 10^(pt-10)`. All FPSS streaming prices would have been wrong. (#95)
3535

36+
## [5.1.1] - 2026-04-03
37+
38+
### Fixed
39+
40+
- `tdbe` dependency bumped to 0.2.0 for crates.io publish (0.1.x was yanked). No code changes.
41+
3642
## [5.1.0] - 2026-04-03
3743

3844
### Breaking Changes

SECURITY.md

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -19,13 +19,17 @@ for critical issues.
1919

2020
| Version | Supported | Notes |
2121
| ------- | ------------------ | ----- |
22-
| 4.5.x | :white_check_mark: | Current release (`tdbe` + `#[repr(C)]` FFI) |
23-
| 4.0-4.4 | :x: | Upgrade to 4.5.x (timezone bug in 4.0-4.4, FFI improvements in 4.5) |
22+
| 5.2.x | :white_check_mark: | Current release (f64 prices, `#[repr(C)]` FPSS events, contract ID fields) |
23+
| 5.0-5.1 | :x: | Upgrade to 5.2.x (Go price formula bug fixed in 5.2) |
24+
| 4.5.x | :x: | Pre-builder-pattern API, missing f64 convenience methods |
25+
| 4.0-4.4 | :x: | Timezone bug (ms_of_day shifted +1 hour Nov-Mar) |
2426
| 3.x | :x: | Pre-`tdbe` extraction, stale API |
2527
| < 3.0 | :x: | Contract wire format bug, missing endpoints |
2628

2729
> **Important:** Versions prior to 4.5.0 contain a timezone bug that shifts ms_of_day by +1 hour
28-
> for all historical data from November through March (EST period). All users should upgrade to 4.5.x.
30+
> for all historical data from November through March (EST period). Versions prior to 5.2.0
31+
> contain a Go SDK price formula bug where `PriceToF64` used the wrong formula. All users
32+
> should upgrade to 5.2.x.
2933
3034
## Security Design
3135

docs-site/docs/historical/option/index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -73,7 +73,7 @@ Historical time series data for a contract.
7373
Data at a specific time of day across a date range.
7474

7575
- [At-Time Trade](./at-time/trade)
76-
- [At-Time OHLC](./at-time/ohlc)
76+
- [At-Time Quote](./at-time/ohlc)
7777

7878
## Streaming (Rust only)
7979

docs-site/docs/options.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -142,7 +142,7 @@ print(df[["strike", "close", "volume"]].head(20))
142142
When you pass `"0"` for `expiration` or `strike`, the server returns data across all matching contracts. Each tick includes contract identification fields (`expiration`, `strike`, `right`, `strike_price_type`) so you can distinguish which contract each tick belongs to.
143143

144144
::: warning
145-
The `right` parameter does **not** support wildcards. You must specify `"C"` (call) or `"P"` (put). Only `expiration` and `strike` accept `"0"` as a wildcard.
145+
The `right` parameter does **not** accept `"0"` as a wildcard. Use `"C"` (call), `"P"` (put), or `"both"` (calls and puts). Only `expiration` and `strike` accept `"0"` as a wildcard.
146146
:::
147147

148148
::: code-group

docs-site/docs/streaming.md

Lines changed: 6 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ Each SDK exposes FPSS differently:
1111

1212
- **Rust** -- Fully synchronous callback model. Events dispatched through an LMAX Disruptor ring buffer. No Tokio on the streaming hot path.
1313
- **Python** -- Polling model with `next_event()`. Events returned as Python dicts with all fields.
14-
- **Go** -- Polling model with `NextEvent()`. Events returned as typed `*FpssEvent` structs. Use `PriceToF64()` for price decoding.
14+
- **Go** -- Polling model with `NextEvent()`. Events returned as typed `*FpssEvent` structs. Price fields are pre-decoded to `float64`; raw integers available as `*Raw` fields.
1515
- **C++** -- Polling model with `next_event()`. Events returned as `FpssEventPtr` (`unique_ptr<TdxFpssEvent>`, RAII). `#[repr(C)]` layout-compatible structs.
1616

1717
::: warning No JSON in FFI
@@ -237,16 +237,15 @@ for {
237237
switch event.Kind {
238238
case thetadatadx.FpssQuoteEvent:
239239
q := event.Quote
240-
bid := thetadatadx.PriceToF64(q.Bid, q.PriceType)
241-
ask := thetadatadx.PriceToF64(q.Ask, q.PriceType)
240+
// Bid and Ask are pre-decoded to float64
242241
fmt.Printf("Quote: contract=%d bid=%.4f ask=%.4f rx=%dns\n",
243-
q.ContractID, bid, ask, q.ReceivedAtNs)
242+
q.ContractID, q.Bid, q.Ask, q.ReceivedAtNs)
244243

245244
case thetadatadx.FpssTradeEvent:
246245
t := event.Trade
247-
price := thetadatadx.PriceToF64(t.Price, t.PriceType)
246+
// Price is pre-decoded to float64
248247
fmt.Printf("Trade: contract=%d price=%.4f size=%d\n",
249-
t.ContractID, price, t.Size)
248+
t.ContractID, t.Price, t.Size)
250249

251250
case thetadatadx.FpssOpenInterestEvent:
252251
oi := event.OpenInterest
@@ -553,7 +552,7 @@ Undecoded fallback for corrupt or unrecognized frames. Fields: `code` (u8), `pay
553552
| `Shutdown` | `()` | Graceful shutdown |
554553
| `Close` | `()` | Free the FPSS handle |
555554

556-
Helper: `PriceToF64(value int32, priceType int32) float64`
555+
Helper: `PriceToF64(value int32, priceType int32) float64` -- decode raw integer prices. Note: FPSS event price fields are pre-decoded to `float64` as of v5.2; this helper is for custom use cases or raw field decoding.
557556

558557
### C++ (`tdx::FpssClient`)
559558

docs-site/docs/streaming/index.md

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -9,9 +9,11 @@ Real-time market data is delivered via ThetaData's FPSS (Feed Processing Streami
99

1010
## Architecture
1111

12-
```
13-
Exchange (NJ) --> ThetaData FPSS servers --> TLS/TCP --> Your application
14-
(4 production hosts) (Disruptor ring buffer)
12+
```mermaid
13+
graph LR
14+
A["Exchange<br/>(NYSE/NASDAQ)"] --> B["ThetaData FPSS<br/>(4 NJ hosts)"]
15+
B -->|"TLS/TCP"| C["SDK I/O Thread<br/>(FIT decode)"]
16+
C -->|"Disruptor<br/>ring buffer"| D["Your Application<br/>(callback / poll)"]
1517
```
1618

1719
Events are decoded from the FIT wire format and delta-decompressed on an I/O thread, then dispatched through an LMAX Disruptor ring buffer to your callback (Rust) or polling queue (Python/Go/C++). Every data event carries a `received_at_ns` nanosecond timestamp captured at frame decode time.
@@ -22,7 +24,7 @@ Events are decoded from the FIT wire format and delta-decompressed on an I/O thr
2224
|-----|-------|------------|---------|
2325
| **Rust** | Synchronous callback | `&FpssEvent` enum | Disruptor ring buffer dispatch. No Tokio on the hot path. |
2426
| **Python** | Polling | `dict` | `next_event()` returns events as Python dicts with all fields. |
25-
| **Go** | Polling | `*FpssEvent` struct | `NextEvent()` returns typed Go structs. `PriceToF64()` for price decoding. |
27+
| **Go** | Polling | `*FpssEvent` struct | `NextEvent()` returns typed Go structs. Price fields pre-decoded to `float64`. |
2628
| **C++** | Polling | `FpssEventPtr` | `next_event()` returns `unique_ptr<TdxFpssEvent>` (RAII). `#[repr(C)]` layout. |
2729

2830
::: warning No JSON in FFI

docs-site/docs/streaming/latency.md

Lines changed: 58 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -11,17 +11,41 @@ Combined with the exchange's `ms_of_day` timestamp on each tick, this gives you
1111

1212
## How it works
1313

14-
```
15-
Exchange (NJ) ──── ThetaData FPSS server ──── TLS/TCP ──── Your application
16-
| |
17-
ms_of_day received_at_ns
18-
(exchange clock) (your clock)
19-
20-
latency = received_at_ns - exchange_timestamp_ns
14+
```mermaid
15+
sequenceDiagram
16+
participant Exchange as Exchange (NYSE/NASDAQ)
17+
participant FPSS as ThetaData FPSS (NJ)
18+
participant SDK as ThetaDataDx SDK
19+
participant App as User Application
20+
21+
Exchange->>FPSS: Market data feed
22+
Note over FPSS: FIT encode + delta compress
23+
FPSS->>SDK: TLS/TCP frame
24+
Note over SDK: received_at_ns captured
25+
SDK->>SDK: FIT decode + delta decompress
26+
SDK->>App: Callback (FpssEvent)
27+
Note over App: latency = received_at_ns - exchange_ns
2128
```
2229

2330
The exchange stamps each quote/trade with `ms_of_day` (milliseconds since midnight ET). Your application stamps `received_at_ns` (nanoseconds since UNIX epoch). The difference is your total latency: exchange -> ThetaData -> network -> TLS -> decode -> your callback.
2431

32+
```mermaid
33+
graph LR
34+
A["Exchange --> FPSS<br/>(~0ms)"] --> B["FPSS Processing<br/>(~0ms)"]
35+
B --> C["Network Transit<br/>(physics: distance/c)"]
36+
C --> D["SDK Decode<br/>(&lt; 1 us)"]
37+
D --> E["User Callback"]
38+
39+
style C fill:#ff9999,color:#000
40+
style D fill:#99ff99,color:#000
41+
```
42+
43+
The network transit segment (red) dominates total latency. The SDK decode time (green) is sub-microsecond and negligible.
44+
45+
::: danger Production only
46+
Latency can only be measured meaningfully on the **production** FPSS server (`DirectConfig::production()`, port 20000) **during live market hours** (9:30 AM - 4:00 PM ET). The dev server (port 20200) replays historical data from a past trading day at maximum speed -- the exchange timestamps are from the past, so `received_at_ns` minus the event's original timestamp produces values that are months or years, not real latency. The dev server is for functional testing only, not latency benchmarking.
47+
:::
48+
2549
## `tdbe::latency::latency_ns()`
2650

2751
The `tdbe` crate provides a DST-aware helper that converts the exchange `ms_of_day` + `date` into epoch nanoseconds and computes the delta:
@@ -135,6 +159,33 @@ For the absolute lowest latency:
135159

136160
3. **Use the Rust SDK directly** -- Python, Go, and C++ add an mpsc channel hop between the Disruptor and `next_event()`.
137161

162+
## Network Physics: Minimum Achievable Latency
163+
164+
ThetaData's FPSS servers are located in New Jersey (NJ datacenter). The speed of light in fiber optic cable is approximately 200,000 km/s (about 2/3 of the vacuum speed of light, due to the refractive index of glass). This sets an absolute physical floor on latency that no software optimization can overcome.
165+
166+
The formula: `minimum_round_trip = distance_km / (300,000 * 0.67) * 2 * 1000` (in milliseconds).
167+
168+
| Your Location | Distance to NJ | Minimum Round-Trip | Typical Observed |
169+
|---------------|---------------|-------------------|-----------------|
170+
| AWS us-east-1 (Virginia) | ~350 km | ~3.5 ms | 2-5 ms |
171+
| NJ/NYC datacenter | <50 km | <0.5 ms | <1 ms |
172+
| Chicago | ~1,200 km | ~12 ms | 10-15 ms |
173+
| Los Angeles | ~3,900 km | ~39 ms | 35-50 ms |
174+
| London | ~5,600 km | ~56 ms | 55-70 ms |
175+
| Frankfurt | ~6,200 km | ~62 ms | 60-80 ms |
176+
| Tokyo | ~10,800 km | ~108 ms | 105-130 ms |
177+
| Sydney | ~16,000 km | ~160 ms | 155-180 ms |
178+
179+
If you are seeing 60-80ms latency from Europe, that is not a bug -- it is the speed of light in fiber. No SDK, no protocol change, no configuration tweak can make photons travel faster.
180+
181+
The SDK's own overhead (`received_at_ns` capture, FIT decode, Disruptor dispatch, callback invocation) is sub-microsecond and entirely negligible compared to network physics.
182+
183+
For latency-sensitive applications:
184+
185+
1. **Colocate near NJ** -- AWS us-east-1 (N. Virginia) or any NJ/NYC-area datacenter gets sub-5ms
186+
2. **`FpssFlushMode::Immediate`** reduces software batching latency by up to 100ms, but cannot beat physics
187+
3. **Use the Rust SDK directly** -- eliminates the FFI channel hop present in Python/Go/C++ (adds <1ms)
188+
138189
## Latency Histogram Example (Rust)
139190

140191
```rust

docs-site/docs/streaming/reconnection.md

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -255,16 +255,15 @@ func main() {
255255
switch event.Kind {
256256
case thetadatadx.FpssQuoteEvent:
257257
q := event.Quote
258-
bid := thetadatadx.PriceToF64(q.Bid, q.PriceType)
259-
ask := thetadatadx.PriceToF64(q.Ask, q.PriceType)
258+
// Bid and Ask are pre-decoded to float64
260259
fmt.Printf("[QUOTE] contract=%d bid=%.4f ask=%.4f rx=%dns\n",
261-
q.ContractID, bid, ask, q.ReceivedAtNs)
260+
q.ContractID, q.Bid, q.Ask, q.ReceivedAtNs)
262261

263262
case thetadatadx.FpssTradeEvent:
264263
t := event.Trade
265-
price := thetadatadx.PriceToF64(t.Price, t.PriceType)
264+
// Price is pre-decoded to float64
266265
fmt.Printf("[TRADE] contract=%d price=%.4f size=%d\n",
267-
t.ContractID, price, t.Size)
266+
t.ContractID, t.Price, t.Size)
268267

269268
case thetadatadx.FpssControlEvent:
270269
ctrl := event.Control

docs/api-reference.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -657,7 +657,7 @@ Nexus HTTP responses with status 401 (Unauthorized) or 404 (Not Found) are treat
657657

658658
### Endpoint Count
659659

660-
ThetaDataDx exposes **61 typed methods** (plus 4 `_stream` variants) covering all 60 gRPC RPCs in `BetaThetaTerminal` plus 1 convenience range-query variant (`stock_history_ohlc_range`). Historical methods are provided via `Deref<Target = DirectClient>` (an internal implementation detail) and generated by the `define_endpoint!` macro in `direct.rs`.
660+
ThetaDataDx exposes **61 typed methods** (plus 4 `_stream` variants) covering all 60 gRPC RPCs in `BetaThetaTerminal` plus 1 convenience range-query variant (`stock_history_ohlc_range`). Historical methods are provided via `Deref<Target = DirectClient>` (an internal implementation detail) and generated by the `parsed_endpoint!` macro in `direct.rs`.
661661

662662
### FFI Coverage
663663

docs/architecture.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,7 @@ MDDS is a standard gRPC service over TLS, operating on port 443.
5757

5858
- **Package**: `BetaEndpoints`
5959
- **Service**: `BetaThetaTerminal`
60-
- **Methods**: 60 RPCs, all server-streaming (returning `stream ResponseData`). thetadatadx wraps all 60 gRPC RPCs plus 1 convenience range-query variant = **61 methods** on `ThetaDataDx`, generated via a declarative `define_endpoint!` macro (internal implementation uses `DirectClient` via `Deref`).
60+
- **Methods**: 60 RPCs, all server-streaming (returning `stream ResponseData`). thetadatadx wraps all 60 gRPC RPCs plus 1 convenience range-query variant = **61 methods** on `ThetaDataDx`, generated via a declarative `parsed_endpoint!` macro (internal implementation uses `DirectClient` via `Deref`).
6161
- **Categories**: Stock, Option, Index, Interest Rate, Calendar - each with List, History, Snapshot, AtTime, and Greeks sub-categories
6262

6363
### Request Structure
@@ -513,7 +513,7 @@ graph TD
513513
end
514514
515515
UNIFIED["unified.rs<br/><i>ThetaDataDx — unified entry point<br/>Deref to DirectClient</i>"]
516-
DIRECT["direct.rs<br/><i>DirectClient (internal) — 61 endpoints<br/>via define_endpoint! macro</i>"]
516+
DIRECT["direct.rs<br/><i>DirectClient (internal) — 61 endpoints<br/>via parsed_endpoint! macro</i>"]
517517
CONFIG["config.rs<br/><i>DirectConfig</i>"]
518518
DECODE["decode.rs<br/><i>zstd + DataTable parsing<br/>(includes generated parsers)</i>"]
519519
REGISTRY["registry.rs<br/><i>EndpointMeta, ENDPOINTS static</i>"]

0 commit comments

Comments
 (0)