1- # Contributing to thetadatadx
1+ # Contributing to ThetaDataDx
22
3- Thank you for your interest in contributing to thetadatadx . This guide covers everything
3+ Thank you for your interest in contributing. This guide covers everything
44you need to get started.
55
66## Prerequisites
77
8- - ** Rust stable** (see ` rust-toolchain.toml ` — includes rustfmt and clippy)
9- - ** protoc** (Protocol Buffers compiler) — only needed if modifying ` .proto ` files
10- - ** Python 3.9+** — for the Python SDK
11- - ** maturin** — for building the PyO3 Python bindings (` pip install maturin ` )
12- - ** Go 1.21+** — for the Go SDK
13- - ** cargo-deny** — for dependency auditing (` cargo install cargo-deny ` )
8+ - ** Rust stable** (see ` rust-toolchain.toml ` - includes rustfmt and clippy)
9+ - ** protoc** (Protocol Buffers compiler) - only needed if modifying ` .proto ` files
10+ - ** Python 3.9+** - for the Python SDK
11+ - ** maturin** - for building the PyO3 Python bindings (` pip install maturin ` )
12+ - ** Go 1.21+** - for the Go SDK
1413
1514## Development Setup
1615
1716``` bash
1817git clone https://github.com/userFRM/ThetaDataDx.git
19- cd thetadatadx
18+ cd ThetaDataDx
2019
2120# Run the full workspace test suite
2221cargo test --workspace
@@ -26,102 +25,156 @@ cargo test --workspace
2625# Line 2: password
2726```
2827
29- ## Code Style
28+ ## Pre-commit Checks
3029
31- All Rust code must pass formatting and linting checks:
30+ Run these ** before every commit ** . CI will reject anything that fails.
3231
3332``` bash
34- cargo fmt --all - --check
35- cargo clippy --workspace - -D warnings
36- ```
33+ # 1. Format
34+ cargo fmt --all -- --check
3735
38- ## Pre-commit Checks
36+ # 2. Lint
37+ cargo clippy --workspace -- -D warnings
3938
40- Run the full CI-equivalent check locally before pushing:
41-
42- ``` bash
43- # Core workspace
44- cargo fmt --all - --check && cargo test --workspace && cargo clippy --workspace - -D warnings
39+ # 3. Test
40+ cargo test --workspace
4541
46- # FFI crate
42+ # 4. FFI build (if modified)
4743cargo build --release -p thetadatadx-ffi
4844
49- # Python SDK (if modified)
50- cd sdks/python && maturin develop && python -m pytest
45+ # 5. Python SDK (if modified)
46+ cd sdks/python && maturin develop && cd ../..
5147```
5248
53- Do not push code that fails any of these checks. CI will reject it.
49+ One-liner for the common case:
5450
55- ## How to Add a New Endpoint
51+ ``` bash
52+ cargo fmt --all -- --check && cargo clippy --workspace -- -D warnings && cargo test --workspace
53+ ```
5654
57- All 61 DirectClient methods are generated by the ` define_endpoint! ` macro in ` direct.rs ` .
58- Adding a new endpoint is a single macro invocation - no hand-coded boilerplate.
59-
60- 1 . ** Update the proto definition** (if the endpoint uses a new message type)
61- - Edit the relevant ` .proto ` file under ` crates/thetadatadx/proto/ `
62- - Regenerate Rust types with ` cargo build ` (prost build script handles codegen)
63-
64- 2 . ** Add a ` define_endpoint! ` invocation in ` direct.rs ` **
65- - Specify the gRPC method name, request type, parameter fields, and response parser
66- - The macro generates the full async method including auth injection, QueryInfo setup,
67- gRPC streaming, zstd decompression, and DataTable parsing
68- - Example pattern (see existing invocations in ` direct.rs ` for exact syntax):
69- ``` rust
70- define_endpoint! (
71- /// Doc comment for the method.
72- stock_history_eod , // Rust method name
73- GetStockHistoryEod , // gRPC RPC name
74- StockHistoryEodRequest , // protobuf request type
75- { symbol , start_date , end_date }, // parameter fields
76- parse_eod_ticks // response parser function
77- );
78- ```
79- - Add unit tests for any new parsing logic
80-
81- 3 . * * Expose in the FFI layer **
82- - Add the corresponding `extern " C" function in `ffi / src / lib . rs`
83- - The FFI crate also uses macros for endpoint generation - follow the existing pattern
84- - For FPSS - related functions , see the 7 existing `thetadatadx_fpss_ * ` functions as examples
85- - Update the C header , Go SDK , and C ++ SDK wrappers accordingly
86-
87- 4 . * * Expose in the Python SDK **
88- - Add the PyO3 binding in `sdks / python / src / `
89- - Add a Python test in `sdks / python / tests / `
55+ ## Commit Convention
9056
91- 5 . * * Update CHANGELOG . md ** under `[ Unreleased ]`
57+ We follow [ Conventional Commits ] ( https://www.conventionalcommits.org/en/v1.0.0/ ) .
9258
93- ## How to Update After a ThetaData Terminal Update
59+ ### Format
9460
95- When ThetaData releases a new terminal version that changes the wire protocol :
61+ ```
62+ <type>(<scope>): <description>
9663
97- 1 . Refer to `docs / reverse - engineering . md` for methodology
98- 2 . Capture new traffic and compare against existing FIT / FIE codec expectations
99- 3 . Update frame parsers , tick types , or message definitions as needed
100- 4 . Run the full test suite to verify backwards compatibility
64+ [optional body]
10165
102- ## Running Against Dev Servers
66+ [optional footer(s)]
67+ ```
10368
104- Set environment variables to point at a non - production terminal :
69+ ### Types
70+
71+ | Type | When to use |
72+ | ------| -------------|
73+ | ` feat ` | New feature or endpoint |
74+ | ` fix ` | Bug fix |
75+ | ` docs ` | Documentation only |
76+ | ` chore ` | Build, CI, tooling, dependency updates |
77+ | ` refactor ` | Code change that neither fixes a bug nor adds a feature |
78+ | ` perf ` | Performance improvement |
79+ | ` test ` | Adding or updating tests |
80+ | ` style ` | Formatting, whitespace (no code change) |
81+
82+ ### Scopes (optional)
83+
84+ | Scope | What it covers |
85+ | -------| ---------------|
86+ | ` core ` | ` crates/thetadatadx/ ` |
87+ | ` ffi ` | ` ffi/ ` |
88+ | ` python ` | ` sdks/python/ ` |
89+ | ` go ` | ` sdks/go/ ` |
90+ | ` cpp ` | ` sdks/cpp/ ` |
91+ | ` cli ` | ` tools/cli/ ` |
92+ | ` mcp ` | ` tools/mcp/ ` |
93+ | ` server ` | ` tools/server/ ` |
94+ | ` docs ` | ` docs/ ` , ` docs-site/ ` |
95+
96+ ### Examples
10597
106- ```bash
107- export THETADX_HOST = " 127.0.0.1"
108- export THETADX_PORT = " 11000"
109- cargo test -- workspace
98+ ```
99+ feat(core): add stock_history_vwap endpoint
100+ fix(python): correct event key from "type" to "kind"
101+ docs: update streaming examples for all languages
102+ chore: bump version to 3.2.2
103+ perf(core): use precomputed pow10 table in price decoding
110104```
111105
106+ ### Breaking changes
107+
108+ Add ` ! ` after the type or a ` BREAKING CHANGE: ` footer:
109+
110+ ```
111+ feat(core)!: replace DirectClient with ThetaDataDx unified client
112+ ```
113+
114+ ## How to Add a New Endpoint
115+
116+ All 61 endpoint methods are generated by the ` parsed_endpoint! ` macro in ` direct.rs ` .
117+ Tick type structs and parsers are generated from ` endpoint_schema.toml ` by ` build.rs ` .
118+
119+ 1 . ** Update the proto** (if the endpoint uses a new message type)
120+ - Edit ` crates/thetadatadx/proto/v3_endpoints.proto `
121+ - ` cargo build ` regenerates Rust types automatically
122+
123+ 2 . ** Add the column schema** (if the response has a new layout)
124+ - Add a ` [types.YourTick] ` block to ` crates/thetadatadx/endpoint_schema.toml `
125+ - ` cargo build ` generates the struct and parser
126+ - See ` docs/endpoint-schema.md ` for the TOML format
127+
128+ 3 . ** Wire up the endpoint in ` direct.rs ` **
129+ ``` rust
130+ parsed_endpoint! {
131+ /// Doc comment.
132+ fn stock_history_vwap (symbol : & str , start : & str , end : & str ) -> Vec <VwapTick >;
133+ grpc : get_stock_history_vwap ;
134+ request : StockHistoryVwapRequest ;
135+ query : StockHistoryVwapParams {
136+ root : symbol . to_string (),
137+ start_date : start . to_string (),
138+ end_date : end . to_string (),
139+ };
140+ parse : decode :: parse_vwap_ticks ;
141+ dates : start , end ;
142+ }
143+ ```
144+
145+ 4 . ** Expose in downstream SDKs**
146+ - FFI: add ` extern "C" ` function in ` ffi/src/lib.rs `
147+ - Python: add PyO3 method in ` sdks/python/src/lib.rs `
148+ - Go: add method in ` sdks/go/client.go `
149+ - C++: add method in ` sdks/cpp/include/thetadx.hpp ` and ` sdks/cpp/src/thetadx.cpp `
150+
151+ 5 . ** Update CHANGELOG.md** under ` [Unreleased] `
152+
153+ See ` crates/thetadatadx/proto/MAINTENANCE.md ` for the full step-by-step guide.
154+
112155## Pull Request Process
113156
114- 1 . ** Branch** — create a feature branch from ` main ` ( ` feat/description ` or ` fix/description ` )
115- 2 . ** Test ** — run the full pre-commit check suite (see above)
116- 3 . ** PR** — open a pull request against ` main `
117- 4 . ** Review ** — address reviewer feedback; all CI checks must pass
118- 5 . ** Merge ** — squash- merge preferred for clean history
157+ 1 . ** Branch** from ` main ` using the convention: ` feat/description ` , ` fix/description ` , ` docs/description `
158+ 2 . ** Pre-commit checks ** must all pass (see above)
159+ 3 . ** Open PR** against ` main ` with a clear title following commit convention
160+ 4 . ** CI must pass ** - format, lint, test, FFI build
161+ 5 . ** Squash merge** for clean history
119162
120163Every PR must include:
121- - Passing CI (fmt, clippy, test, deny)
164+ - Passing CI
122165- Updated ` CHANGELOG.md ` if user-facing
123166- Updated documentation if any public API changed
124167
168+ ## How to Update After a ThetaData Terminal Update
169+
170+ When ThetaData releases a new terminal version:
171+
172+ 1 . Refer to ` docs/reverse-engineering.md ` for methodology
173+ 2 . Drop new proto files in ` crates/thetadatadx/proto/ `
174+ 3 . Update ` endpoint_schema.toml ` if column schemas changed
175+ 4 . Run the full test suite to verify backwards compatibility
176+ 5 . See ` crates/thetadatadx/proto/MAINTENANCE.md ` for the detailed guide
177+
125178## Community
126179
127180Join the ThetaData Discord for questions and discussion: ** [ discord.thetadata.us] ( https://discord.thetadata.us/ ) **
0 commit comments