Add the options resource#10
Conversation
|
The author of this PR, MarketDataDev03, is not an activated member of this organization on Codecov. |
Captures the request-object vs. params-bag vs. fluent-terminal decision that the options resource introduces SDK-wide. Leads with the cross-SDK comparison (sdk-py / sdk-js bags vs. the four Java candidates) and recommends the inert request object plus an additive Consumer<Builder> overload, preserving the compile-time sealed-filter guarantee.
|
Added ADR-008: Endpoint Parameter Convention (Request Objects) in This PR introduces the per-endpoint request-class convention SDK-wide (the template It leads with the cross-SDK comparison and weighs four candidates against a constant filter set (calls /
Recommendation: A + B. Keep the inert, decoupled request object as canonical (reuse, testability, trivial sync/async parity), add the lambda overload as the ergonomic front door. C and D are not recommended — C sacrifices the decoupled spec, adds an un-enforceable dangling-terminal footgun, amends ADR-006, and makes Java the cross-SDK call-shape outlier; D discards the compile-time sealed-filter guarantee ( Two notes for reviewers:
This is a convention call that propagates to every future resource, so it'd be good to settle it here before the pattern multiplies. |
ADR-008 — Companion: resource & response design (consumer-facing foundations)
TL;DRADR-008 framed the decision as "how does an endpoint accept its parameters" (the input axis) and lands on the request‑object + builder. That input conclusion is right. But the ADR only brushed past a second, coupled question: how does the consumer choose the response format — and in a statically‑typed language that is a harder problem than it looks, because the format changes the response type, and Java types are not dynamic. This note: (1) names that constraint explicitly, (2) argues why the builder is the best input convention from the consumer's seat, (3) answers the output/format question by selecting the format through a typed resource facet (a compile‑time choice the compiler enforces), and (4) turns all of that into a construction recipe every resource follows — so the SDK stays uniform as 1. The constraint Java has that Python and JS don'tThe sibling SDKs answer "what format do I want back?" with a runtime parameter, and the return type changes with it: # Python — one method, the RETURN TYPE changes at runtime
client.stocks.candles("AAPL", output_format="dataframe") # -> DataFrame
client.stocks.candles("AAPL", output_format="json") # -> dict
client.stocks.candles("AAPL", output_format="csv") # -> str// JavaScript — same idea, return type resolved at runtime
client.stocks.candles("AAPL", { format: "csv" }); // -> string | objectThey can do this because they're dynamically typed: the caller sorts out what came back. Java cannot. A Java method has exactly one static return type, fixed at compile time: // Impossible: a method can't return StockCandles OR String based on a runtime flag.
??? result = client.stocks().candles(req, format); // what type is `result`?flowchart LR
subgraph DYN["Python / JS — dynamic"]
A["candles(req, format)"] --> B{format at runtime}
B --> C["DataFrame"]
B --> D["dict / object"]
B --> E["CSV string"]
end
subgraph STAT["Java — static"]
F["candles(req)"] --> G["exactly ONE compile-time type"]
end
So the output‑format problem — trivial in py/js — is structural in Java. We can't bolt a This is also why it's worth separating two things the ADR grouped together under "universal parameters". They live on different axes:
So "universal parameters" is not one bucket. 2. Input: why the builder/request object is the best call for the consumerFor the input axis, the request‑object + builder (ADR-008 Option A) is the strongest option from the consumer's seat. The hard case proves it: OptionsChain chain = client.options().chain(
OptionsChainRequest.builder("AAPL")
.expirationFilter(ExpirationFilter.dte(45)) // pick exactly ONE expiration variant
.strikeFilter(StrikeFilter.range(150, 250)) // pick exactly ONE strike variant
.side(OptionSide.CALL)
.strikeLimit(10)
.build());Advantages (consumer side)
Disadvantages (consumer side)
The cost is real but bounded, and it buys the one advantage that justifies writing an SDK in a typed language at all. 3. Output: select the format through a typed resource facetBecause the format changes the response type, the consumer chooses it by selecting a typed view (facet) of the resource. Each facet's endpoints return the matching response type; the default — no facet — is the typed JSON path. Cross‑cutting modifiers that don't change the type (the universal parameters) are set fluently on the resource before the endpoint call. flowchart TD
R["client.stocks()"] -->|default| T["typed facet"]
R -->|".asCsv()"| CSV["CSV facet"]
R -->|".asHtml()"| HTML["HTML facet"]
T -->|"candles(req)"| TR["StockCandles<br/>(typed rows + raw json)"]
CSV -->|"candles(req)"| CR["CsvResponse"]
HTML -->|"candles(req)"| HR["HtmlResponse"]
The format is no longer a runtime value the return type has to anticipate — it's a compile‑time selection: which facet you call decides which type you get, and the compiler enforces it from there. How the consumer uses itTyped (default) — the 99% path: StockCandles c = client.stocks().candles(
StocksCandlesRequest.builder("AAPL", StockResolution.daily()).countback(30).build());
for (StockCandle bar : c.candles()) { /* typed rows */ }
String rawJson = c.json(); // the original payload, available if you want itWith universal parameters (set fluently on the resource): StockCandles c = client.stocks()
.dateFormat(DateFormat.TIMESTAMP).limit(50)
.candles(StocksCandlesRequest.of("AAPL", StockResolution.daily()));
// the resource is an immutable configured value — configure once, reuse across calls:
StocksResource live = client.stocks().mode(Mode.LIVE).limit(100);
StockCandles candles = live.candles(candlesReq);
StockQuotes quotes = live.quotes(quotesReq);
// or bundle the universal params and pass them in one shot:
StockCandles d = client.stocks().params(myUniversalParams).candles(candlesReq);
CSV: CsvResponse csv = client.stocks().asCsv()
.candles(StocksCandlesRequest.builder("AAPL", StockResolution.daily()).countback(30).build());
csv.saveToFile(Path.of("aapl.csv")); // or csv.csv() for the textHTML: HtmlResponse html = client.stocks().asHtml()
.candles(StocksCandlesRequest.of("AAPL", StockResolution.daily()));
html.saveToFile(Path.of("aapl.html"));Async — same shape, both axes: CompletableFuture<StockCandles> f = client.stocks().candlesAsync(req);
CompletableFuture<CsvResponse> g = client.stocks().asCsv().candlesAsync(req);Full example — sealed filters (input) + universal params + CSV (output): CsvResponse chain = client.options().mode(Mode.DELAYED).asCsv().chain(
OptionsChainRequest.builder("AAPL")
.expirationFilter(ExpirationFilter.dte(45))
.strikeFilter(StrikeFilter.range(150, 250))
.side(OptionSide.CALL).strikeLimit(10)
.build());
chain.saveToFile(Path.of("aapl-chain.csv"));The compile‑time guarantee: CsvResponse ok = client.stocks().asCsv().candles(req); // ✓
StockCandles bad1 = client.stocks().asCsv().candles(req); // ✗ won't compile (CsvResponse ≠ StockCandles)
HtmlResponse bad2 = client.stocks().asCsv().candles(req); // ✗ won't compile (CsvResponse ≠ HtmlResponse)Every resource, one grammarThe same // stocks — single symbol
StockCandles candles = client.stocks().candles(
StocksCandlesRequest.builder("AAPL", StockResolution.daily()).countback(30).build());
// stocks — MANY symbols in one call (required input is a list, not a string)
StockPrices prices = client.stocks().prices(
StocksPricesRequest.of("AAPL", "MSFT", "GOOG"));
// options.chain — the sealed‑filter showcase (input safety + universal param + CSV)
CsvResponse chainCsv = client.options().dateFormat(DateFormat.TIMESTAMP).asCsv().chain(
OptionsChainRequest.builder("AAPL")
.expirationFilter(ExpirationFilter.dte(45)) // pick exactly ONE expiration variant
.strikeFilter(StrikeFilter.range(150, 250)) // pick exactly ONE strike variant
.side(OptionSide.CALL).strikeLimit(10)
.build());
// options.quotes — required input is an OCC OPTION symbol, not a ticker
OptionsQuotes oq = client.options().quote(
OptionsQuoteRequest.of("AAPL250117C00150000"));
// options.lookup — required input is a free‑text human string; the response is a scalar
OptionsLookup sym = client.options().lookup(
OptionsLookupRequest.of("AAPL 1/17/25 $150 call"));
String occSymbol = sym.optionSymbol(); // natural per‑endpoint accessor
// options.expirations — typed response is a LIST, not rows
OptionsExpirations exps = client.options().expirations(
OptionsExpirationsRequest.of("AAPL"));
for (LocalDate d : exps.expirations()) { /* ... */ }
// options.strikes — typed response is a date‑KEYED MAP, not rows
OptionsStrikes strikes = client.options().strikes(
OptionsStrikesRequest.of("AAPL"));
List<Double> jan = strikes.byExpiration().get(LocalDate.of(2025, 1, 17));
// markets.status — NO required input at all
MarketsStatus status = client.markets().status(
MarketsStatusRequest.builder().country("US").build());
// funds — same candle shape as stocks, different resource
FundCandles fc = client.funds().candles(
FundsCandlesRequest.of("VFIAX", StockResolution.daily()));Every line above reads the same way: pick the resource, optionally set type‑preserving universal params and/or a format facet, then call the endpoint with its request object. The return type differs per endpoint (rows, a list, a date‑keyed map, a scalar) — but that's the typed model doing its job, not a break in the grammar. The response modelclassDiagram
class MarketDataResponse {
<<interface>>
+statusCode() int
+requestId() String
+isNoData() boolean
+saveToFile(path) void
}
class StockCandles {
+candles() List~StockCandle~
+json() String
}
class CsvResponse {
+csv() String
}
class HtmlResponse {
+html() String
}
MarketDataResponse <|.. StockCandles
MarketDataResponse <|.. CsvResponse
MarketDataResponse <|.. HtmlResponse
Advantages / disadvantages (consumer side)Advantages
Disadvantages
Prior artSelecting a typed view that changes the return type is a well‑trodden SDK shape, most visibly in sync/async client pairs:
And where SDKs also expose raw bytes, they keep it as a separate surface rather than overloading the typed response — e.g. Elasticsearch's low‑level Alternative considered — a single runtime‑typed responseOne 4. How a resource is built (the author's view)The consumer grammar in §3 only stays uniform if every resource is assembled the same way. This section is the recipe: what an SDK author writes to add a resource or an endpoint, and — just as important — what they don't write because it's shared infrastructure. The goal is that adding The anatomy: four layersflowchart TD
subgraph PUB["public — what the consumer touches"]
F["resource facade<br/>StocksResource (+ .asCsv() facet)"]
RQ["request object<br/>StocksCandlesRequest (builder, sealed filters)"]
RS["response record<br/>StockCandles : MarketDataResponse"]
end
subgraph INT["package-private — shared infra, written once"]
T["HttpTransport<br/>(retry, rate-limit, semaphore, timeouts)"]
PA["ParallelArrays + deserializer module"]
SP["RequestSpec (path + query builder)"]
end
F -->|"builds a"| SP --> T
RQ -->|"feeds"| F
T -->|"decodes via"| PA --> RS
The public layer is per‑endpoint and is all an author writes. The package‑private layer (per ADR‑007) is the transport/retry/rate‑limit/decoding machinery that already exists — endpoints plug into it, they don't reimplement it. The per‑endpoint checklistTo add one endpoint, an author writes exactly four things:
That's it. The facade is Facet exposure: one decision per endpointThe typed method above lives on the default resource. To also offer it as CSV, the CSV facet exposes the same endpoint returning What an author never touches
Bringing
|
| Requirement | Status under this design |
|---|---|
§1.2 five resources stocks/options/funds/markets/utilities |
✓ each is a client.<resource>() facet root |
| §2 endpoint coverage | ✓ one method per endpoint, request object carries its params |
§3 universal params (dateformat, columns, headers, human) |
✓ all four supported — but split across facets by type‑safety (see Decision 1) |
| §11.1 decode wire format into typed models by default | ✓ the default (no‑facet) path is the typed model |
| §11.3 additional formats "only when idiomatic", "no ecosystem‑misaligned formats for parity" | ✓ CSV via a typed facet is idiomatic; we do not clone Python's format="csv" runtime string |
§11.5 isJson()/isCsv()/isHtml(), saveToFile, no‑data, toString() |
✓ on the shared MarketDataResponse interface |
The spec does not require format to be a runtime parameter and does not require HTML output — it only requires the isHtml() detector. That is the latitude this design spends: format is a compile‑time facet, not a runtime string.
The decisions the resource landscape forces
Decision 1 — Where do columns / human / headers live?
This is the sharpest one, and it turns out to reinforce the facet split rather than fight it. Of the four spec‑universal params, only dateformat is type‑preserving. The other three are not:
columnsprojects the payload to a subset → the typed record can no longer deserialize.humanrenames every field → the typed record can no longer deserialize.headerstoggles a CSV header row → meaningless for a typed/JSON response.
So the facet boundary coincides with the type‑safety boundary of the params themselves. The typed facet exposes the type‑preserving modifiers (dateformat, plus the Java‑doc extras mode/limit/offset); the CSV facet additionally exposes columns/headers/human, which only make sense once you've left the typed world. All four are "supported" (compliance holds), but they sort onto the facet where they don't corrupt anything. This is an argument for the facet, not a wart.
CsvResponse csv = client.stocks().asCsv()
.columns("t", "c").human(true) // legal here — output is already opaque text
.candles(StocksCandlesRequest.of("AAPL", StockResolution.daily()));
// client.stocks().columns("t","c").candles(req) // intentionally NOT offered:
// it would hand back a StockCandles that can't decodeDecision 2 — Ship .asHtml() now, or keep HtmlResponse dormant?
The backend implements no HTML output on any endpoint, and the spec doesn't require producing HTML. Shipping .asHtml() today means shipping a facet whose every call fails against the live API. Recommendation: keep the HtmlResponse type and the isHtml() detector (cheap, spec‑required as a detector, future‑ready), but do not expose .asHtml() facet methods until the server supports it — or expose them behind an explicitly "experimental" marker. We get HTML‑readiness without a facet that 400s.
Decision 3 — Facets are a subset, not a clone, of the typed surface.
Not every endpoint has every format. options.lookup takes a free‑text string, returns a scalar optionSymbol, and carries no format param at all on the backend. So the CSV facet simply does not expose lookup — facets are allowed to omit endpoints a format doesn't fit. Practically: client.options().asCsv() returns an object with chain/quotes/strikes/expirations but no lookup. That's a feature (you can't write a CSV call that can't exist), but the team must accept that the CSV facet's method set is a principled subset of the typed facet's, decided per‑endpoint, not a mechanical 1:1 mirror.
Decision 4 — Fan‑out endpoints mirror their container; the format facet swaps only the value type.
First, separate two kinds of fan‑out — only one poses a question:
- Merge‑fan‑out (
stocks.candlesintraday auto‑split): N date‑chunk requests are merged into oneStockCandles; the consumer never sees the fan‑out. CSV is a singleCsvResponse(merge the chunks' CSV, assdk-py'smerge_csv_textsdoes). No decision. - Map‑fan‑out (
options.quotes): the backend path is single‑symbol (options/quotes/{optionSymbol}/— comma‑bulk isn't actually supported, verified by reading the handler), so the SDK fires one request per symbol and returnsMap<String, Response<OptionsQuotes>>keyed by symbol, each value carrying its ownstatusCode/isNoData/requestId. This is the only shape that poses the facet question.
Whether an endpoint is map‑fan‑out is forced by the backend (does it accept the batch in one request?), not by taste — stocks.quotes takes a comma‑list in one request and returns a single multi‑row StockQuotes, so it is not a map‑fan‑out.
For map‑fan‑out, the facet should mirror the container and swap only the value type, because a format facet changes the response type, not its cardinality:
client.options().quotes(req) // Map<String, OptionsQuotes>
client.options().asCsv().quotes(req) // Map<String, CsvResponse>This is the most consistent with the facet thesis, needs no merge logic, and loses no per‑symbol metadata or error isolation. Merging all symbols into one CsvResponse (the sdk-py shape) is a nice‑to‑have for "open it in one spreadsheet", but it costs us merge logic and collapses per‑symbol metadata — so it belongs as an optional convenience helper on top, not the primitive. (Flattening the typed response keeps this working: a flattened OptionsQuotes implements MarketDataResponse, so Map<String, OptionsQuotes> still carries per‑symbol metadata that previously lived on the Response<T> wrapper.) One rule, applied to every future per‑symbol endpoint.
What does not break (the reassuring part)
- Input variety is a non‑issue. Single symbol, symbol list, OCC option symbol, free‑text lookup, no symbol at all — every one is just a different request object behind an unchanged call grammar. This is exactly what ADR‑008's request‑object decision already bought us; the facet inherits it for free.
- Response‑shape variety is a non‑issue. Rows (
candles), a list (expirations), a date‑keyed map (strikes), a scalar (lookup), a per‑symbol map (quotesfan‑out) — each is just a per‑endpoint typed record exposing its natural accessor plus.json(). The "flattened typed" convention generalizes from "has.rows()" to "exposes whatever shape fits, uniformly behindMarketDataResponse." No endpoint forces a grammar change. utilitiesstays itself.status/headers/userare diagnostic endpoints with no CSV/HTML need;utilitiessimply doesn't grow format facets. The grammar tolerates a resource opting out of facets entirely.
Net assessment
The architecture stays homogeneous at the grammar level across all five resources while subsetting honestly where the API genuinely differs. The four decisions above are real, but each is a one‑time choice that then applies uniformly to every future resource — which is precisely what an ADR is for. None of them threatens the core: typed‑by‑default, format‑as‑compile‑time‑facet, sealed‑filter input safety. The migration cost is the honest caveat — options shipped before this convention settled, so adopting the proposal means the one‑time flattening pass spelled out in §4 ("Bringing options into line"); funds and markets are built to the convention from the start.
6. Summary for discussion
- Input: keep the request‑object + builder (ADR-008 Option A). It's the only option that turns the API's mutually‑exclusive filters into compile errors — the typed SDK's defining advantage.
- Output/format: recognize it as a separate axis that, in a typed language, cannot be a runtime parameter — and resolve it by selecting the format as a typed resource facet, so the compiler carries the format guarantee.
- The two axes compose cleanly:
client.<resource>()[.universalParams…][.asCsv()/.asHtml()].<endpoint>(request). - Cross‑resource: the grammar holds across all five resources and every required‑input/response shape the API has. Four conscious decisions fall out of the resource landscape — (1)
columns/human/headerslive on the CSV facet, where the facet boundary coincides with the params' own type‑safety boundary; (2) keepHtmlResponsedormant until the server supports HTML; (3) format facets are a principled subset of the typed surface (options.lookuphas no CSV); (4) per‑symbol fan‑out endpoints mirror their container (typedMap<String, OptionsQuotes>→ CSVMap<String, CsvResponse>), with single‑file merge as an optional helper. Each is decided once and applied uniformly.
7009e48 to
d895c8f
Compare
Add the
optionsresource (+ the response & parameter conventions it establishes)Branch:
10_options_resource→mainThis PR adds the second consumer-facing resource,
client.options()(every options endpoint), and with it the response-model and parameter conventions every future resource will follow.utilitiesis migrated onto the same model in this PR;stocks/funds/marketswill copy the template.For a flow-by-flow walkthrough aimed at reviewers, see
docs/OPTIONS_REVIEW_GUIDE.md.What's added
Endpoints (sync + async parity, ADR-006)
client.options()exposes all endpoints, each with afoo(...)+fooAsync(...)pair returning a named response type:.values())lookup/v1/options/lookup/{userInput}OptionsLookupResponse→String(OCC symbol)expirations/v1/options/expirations/{symbol}OptionsExpirationsResponse→List<ZonedDateTime>(+updated())strikes/v1/options/strikes/{symbol}OptionsStrikesResponse→List<ExpirationStrikes>(+updated())quote/v1/options/quotes/{symbol}OptionsQuotesResponse→List<OptionQuote>quotes/v1/options/quotes/{symbol}Map<String, OptionsQuotesResponse>(one concurrent request per symbol, fail-fast)chain/v1/options/chain/{symbol}OptionsChainResponse→List<OptionQuote>Response model (new, SDK-wide) — replaces the old
Response<T>MarketDataResponse<T>interface: one uniformvalues()data accessor (typed per endpoint) plus the metadata surface (statusCode,isNoData,requestId,requestUrl,json,isJson/isCsv/isHtml,saveToFile).OptionsChainResponse, …) implement it;values()is the flat payload (aList, or the scalarStringforlookup) — one hop, no wrapper to unwrap.Response<T>is deleted;utilities(status/headers/user) is migrated to named response types too (Response<T>had no remaining users).Universal parameters (now implemented on the resource)
Set fluently on the resource (immutable configured copies via
RequestConfig):dateFormat(),mode(),limit(),offset().columns(String...)— projection (see below).human()/headers()live only on the CSV facet — they reshape the payload, so they don't cohere with the typed path.columns+ nullable fields + Option A (correctness-critical)OptionQuotefield is@Nullable(boxed), so acolumnsprojection decodes cleanly — a field the consumer didn't ask for comes backnull.validateRequestedColumnsfails loudly (ParseError) when a required column that was requested (or wasn't projected away) is missing. So anullonly ever means "I projected it away" or "legitimately-optional (greek/IV)" — never "the backend silently dropped a required field." Strict-by-default survives the nullable fields.Format facets
asCsv()→OptionsCsvResource→CsvResponse(raw CSV text). Carries the universal-param config and additionally exposescolumns/human/headers. Omitslookup(a scalar — no CSV). Fan-out mirrors the container:Map<String, CsvResponse>.asHtml()→OptionsHtmlResource/HtmlResponseis built but package-private (not exposed) — the backend serves no HTML for data endpoints. Flip the entry point topublicwhen it does; theisHtml()detector ships regardless.Greeks helpers
OptionQuote.presentGreeks()→Set<Greek>andgreek(Greek)→@Nullable Double.Greekenum =DELTA/GAMMA/THETA/VEGA/RHO(IV is not a greek; read viaiv()).Request-class convention (SDK-wide pattern)
Every endpoint takes a Builder-based request object in
com.marketdata.sdk.options— noStringconvenience overloads, uniform call shape:of(required...)shortcut for no-optional endpoints;builder(required...)otherwise; cross-field validation inbuild().chaingroups are sealed types (compiler-enforced "pick one variant"):ExpirationFilter→OnDate/Dte/Between/MonthYear/All;StrikeFilter→Exact/Range/Comparison.Infra reused by future resources
ParallelArrayslenient accessors —dblOrNullplus newtextOrNull/lngOrNull/boolOrNull/nodeOrNull(absent column or null cell →null; still strict on a present-but-wrong-type cell). The stricttext/dbl/lng/boolpath is unchanged for existing callers.RequestConfig— the immutable universal-param bundle a resource carries across its endpoints.JsonResponseParser.parse(env, type, requestedColumns)— threads the requestedcolumnsinto deserializers (via a Jackson context attribute) so Option A can enforce them.MarketDataDates,PathSegments— date parsing, per-segment URL encoding.Testing
OptionsResourceTest(URL/filter translation, sealed-type dispatch, envelope handling, columns projection, Option A failures, strict-by-default, greeks helpers, CSV/HTML facets, universal params on the wire, response metadata) +UtilitiesResourceTest(migrated) +ParallelArraysTest../gradlew buildgreen (unit + Spotless + JaCoCo).OptionsIntegrationTest— 10 tests, shape assertions, run against the live API (MARKETDATA_RUN_INTEGRATION_TESTS=true ./gradlew integrationTest).OptionsApp(./gradlew runOptions/make demo-options) — every endpoint with the full parameter surface, the CSV facet (plain +columns/human/headers+ fan-out),columnsprojection, and both Option-A failure paths. Verified end-to-end against the mock server.Findings that shaped the design
203 Non-Authoritative Informationfor cached/delayed data; tests accept200 || 203.countbackquery returnednulliv/greeks — the original driver for nullable model values; generalized here to "all fields nullable forcolumns", with Option A keeping it safe.Deliberately left for later (please don't flag as missing)
HtmlResponse+ the facet are built and tested, butasHtml()is package-private until the backend servesformat=html. Enabling it is then a one-line change.sourceparameter — INTENTIONALLY NOT ADDED. Internal data-provider param, not in the public schema / cross-language requirements / Python SDK. Left out for v1.client.getRateLimits(); a small follow-up.stocks/funds/markets— out of scope; they adopt this convention next.Review focus
The load-bearing, non-obvious pieces: the response model (
MarketDataResponse<T>+ named types +values()), the nullable + Option A mechanism (buildOptionRowlenient decode +validateRequestedColumnsstrictness — correctness of thecolumns/no-silent-failure guarantee), the chain filter→query translation (applyChainParams/applyExpirationFilter/strikeFilterWireValue), the facets (CSV exposed, HTML built-not-exposed), and thequotesfan-out semantics. See the review guide for the suggested reading order.