make stocks compliance

Author: MarketDataDev03

CHANGELOG.md

@@ -40,7 +40,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
   on synthesized forward-quarter rows. Mixed date/timestamp wire shapes (a daily
   candle's date-only `t` vs. an intraday full timestamp) decode uniformly. Carries
   the same universal-parameter setters, `columns` projection (with the Option A
-  strict guarantee), and `asCsv()` facet as `options`.
+  strict guarantee), and `asCsv()` facet as `options`. Intraday candle requests
+  spanning more than ~one year are **auto-split** into year-sized sub-requests,
+  fetched concurrently through the 50-permit pool and merged into one response
+  (SDK requirements §12), on both the typed and CSV paths.
+- **Per-response rate limits** — every `MarketDataResponse` now exposes
+  `rateLimit()` returning a `RateLimitSnapshot` parsed from that response's own
+  `x-api-ratelimit-*` headers (request-scoped, SDK requirements §8.2), distinct
+  from the client-level `MarketDataClient.getRateLimits()`. Applies to every
+  resource (options/utilities/stocks) and the CSV/HTML responses.
 - **Options resource** (`client.options()`) — all six endpoints, each in sync +
   async form: `lookup`, `expirations`, `strikes`, `quote` (single contract),
   `quotes` (multi-contract fan-out returning a per-symbol

CLAUDE.md

@@ -85,7 +85,7 @@ The Java SDK must also satisfy the canonical, cross-language [SDK Requirements](
 **Deliberately deferred (require the per-resource layer to land first):**
 - §1.2 resource groupings — `client.utilities()`, `client.options()`, and `client.stocks()` are wired today; `client.funds`, `client.markets` still to come.
 - §2 endpoint method coverage; §3 universal parameters; §11 wire-format decoding for the remaining resources (funds, markets). The plumbing (`ParallelArrays.zip` helper for the parallel-arrays shape, `JsonResponseParser`, the `MarketDataResponse<T>` named-response types) is in place — each new endpoint just declares its fields and row builder.
-- §8 request-scoped rate-limit attachment — today the snapshot is client-level (via `client.getRateLimits()`); attaching a per-response snapshot to `Response<T>` is a small follow-up when a consumer needs it.
+- ~~§8 request-scoped rate-limit attachment~~ **DONE** — every `MarketDataResponse` now exposes `rateLimit()`, parsed from that response's own `x-api-ratelimit-*` headers (request-scoped), alongside the client-level `client.getRateLimits()`.
 - §13 100% coverage threshold via JaCoCo `violationRules`; deferred until the resource layer lands so the threshold meaningfully ratchets functional code, not just scaffolding.
 
 When picking up new work, check this list before reaching for the SDK requirements doc — most foundational rules are already encoded in code; missing pieces are deferred deliberately, not by accident.

README.md

@@ -48,8 +48,10 @@ MarketDataClient().use { client ->
 
 Every response implements `MarketDataResponse<T>`: `values()` returns the typed payload
 (typed per endpoint — a `List`, a scalar `String`, …), and the same metadata accessors
-(`statusCode()`, `isNoData()`, `requestId()`, `json()`, `saveToFile(path)`) are available on
-every response, on every resource.
+(`statusCode()`, `isNoData()`, `requestId()`, `rateLimit()`, `json()`, `saveToFile(path)`) are
+available on every response, on every resource. `rateLimit()` returns the rate-limit snapshot
+parsed from *that* response's headers (request-scoped), distinct from the client-level
+`client.getRateLimits()`.
 
 ## Options
 
@@ -172,7 +174,9 @@ a typed `MarketDataResponse` (payload via `values()`). The universal-parameter s
 ### Candles
 
 `StockResolution` is a value type, not an enum — the API accepts an open-ended family of
-resolutions, so use the factories (`DAILY`, `minutes(15)`, `hours(1)`, `days(2)`, …):
+resolutions, so use the factories (`DAILY`, `minutes(15)`, `hours(1)`, `days(2)`, …). An
+**intraday** request spanning more than ~one year is automatically split into year-sized
+sub-requests, fetched concurrently and merged into one response — transparent to the caller:
 
 #### Java
 

docs/STOCKS_REVIEW_GUIDE.md

@@ -2,7 +2,7 @@
 
 This guide walks a reviewer through the `stocks` resource added on the `11_stocks_resource` branch. It is organized by **flow**, not by file.
 
-This PR **adopts the conventions the [`options` PR](OPTIONS_REVIEW_GUIDE.md) established** — the `MarketDataResponse<T>` + named-response model, the Builder-based per-endpoint request, nullable fields + `columns` + Option A, and the CSV/HTML facets — and applies them to stocks. The shared layers (transport, retry, rate-limit, response model, `ParallelArrays`, `JsonResponseParser`) are reused **unchanged**; the only shared-layer addition is one tolerant date parser (`MarketDataDates.parseDateOrTimestampField`). If you reviewed the options PR, the load-bearing shape will be familiar — focus your time on §4 (per-endpoint query translation), §5 (the per-endpoint required-column sets + the `news` deserializer) and §7–§8 (batch vs. fan-out, news/earnings specifics).
+This PR **adopts the conventions the [`options` PR](OPTIONS_REVIEW_GUIDE.md) established** — the `MarketDataResponse<T>` + named-response model, the Builder-based per-endpoint request, nullable fields + `columns` + Option A, and the CSV/HTML facets — and applies them to stocks. The shared layers (transport, retry, rate-limit parsing, `ParallelArrays`, `JsonResponseParser`) are reused unchanged; there are **two additive shared-layer changes** — a tolerant date parser (`MarketDataDates.parseDateOrTimestampField`) and a per-response rate-limit accessor (`MarketDataResponse.rateLimit()`, §8.2). If you reviewed the options PR, the load-bearing shape will be familiar — focus your time on §4 (per-endpoint query translation), §5 (the per-endpoint required-column sets + the `news` deserializer), §7–§8 (batch vs. fan-out, news/earnings specifics), and the two stocks-/SDK-specific additions: **§12 candle auto-chunking** (§9.9) and **§8.2 per-response rate limits** (§9.10).
 
 Suggested reading order: §1 (what's here) → §5 (deserializers: nullable + columns + Option A) → §3/§4 (requests + query translation) → §7 (batch) → §8 (news/earnings) → §9 (subtle corners). ~30 minutes.
 
@@ -65,24 +65,29 @@ com.marketdata.sdk.stocks.StockResolution      (candle-resolution value type)
 |---|---|---|
 | Resource façade | `StocksResource.java` | universal-param config, per-endpoint `*Spec` builders, the generic row deserializer + Option A, `asCsv()`/`asHtml()` |
 | CSV/HTML facets | `StocksCsvResource.java`, `StocksHtmlResource.java` | reuse of the static `*Spec` builders, `format=csv/html` |
-| Responses | `Stock*Response.java` | `values()` payload typing; `StockNewsResponse.updated()` scalar accessor |
+| Responses | `Stock*Response.java`, `MarketDataResponse.java`, `AbstractMarketDataResponse.java` | `values()` payload typing; `StockNewsResponse.updated()` scalar accessor; the new `rateLimit()` (§8.2) on the interface + base |
+| Candle chunking | `StocksResource.candleChunks`/`candlesAsync`, `StockResolution.isIntraday`, `StocksCsvResource.mergeCsvBodies` | §12 intraday split → concurrent fetch → merge |
 | Requests | `stocks/Stock*Request.java`, `StockRequests.java`, `StockResolution.java` | Builder validation, shared `validateWindow`, the value-type resolution |
 | Row records | `stocks/StockCandle/StockQuote/StockPrice/StockNewsArticle/StockEarning.java` | `@Nullable` fields; `StockNewsArticle` non-null (always emitted) |
 | News deserializer | `StockNewsDeserializer.java` | per-row arrays + scalar `updated`; envelope handling |
-| Reused infra (changed) | `MarketDataDates.java` | the new `parseDateOrTimestampField` only |
+| Reused infra (changed) | `MarketDataDates.java`, `MarketDataResponse.java`, `AbstractMarketDataResponse.java` | additive only: `parseDateOrTimestampField`; `rateLimit()` |
 | Wiring | `MarketDataClient.java` | `client.stocks()` |
 
-### 1.3 What changed in shared layers (vs. "unchanged")
+### 1.3 What changed in shared layers (additive only)
 
-- **`MarketDataDates`** gained `parseDateOrTimestampField` — additive; the existing `parseDateField` / `parseTimestampField` are untouched. Everything else (`ParallelArrays`, `JsonResponseParser`, `RequestConfig`, `MarketDataResponse`, transport/retry/rate-limit/status-cache/exceptions) is reused exactly as the options PR left it. Confirm no existing parser was modified.
+- **`MarketDataDates`** gained `parseDateOrTimestampField` — the existing `parseDateField` / `parseTimestampField` are untouched. Confirm no existing parser was modified.
+- **`MarketDataResponse`** gained `rateLimit()` (§8.2), implemented once in `AbstractMarketDataResponse` (it parses `RateLimitHeaders.parse(envelope.headers())` in the constructor). Since that base is the **only** implementor of the interface, every existing response type — options, utilities, `CsvResponse`, `HtmlResponse` — gets it with no per-type change. Confirm the interface addition didn't miss an implementor and that options/utilities tests still pass.
+- Everything else (`ParallelArrays`, `JsonResponseParser`, `RequestConfig`, `RateLimitHeaders`, transport/retry/status-cache/exceptions) is reused exactly as the options PR left it.
 
 ---
 
 ## 2. The response model (reused)
 
-No new model concepts — `Stock*Response` are thin subclasses of `AbstractMarketDataResponse<T>` (see [Options guide §2](OPTIONS_REVIEW_GUIDE.md#2-the-response-model) for the full shape). `values()` is the flat payload per endpoint (`List<StockCandle>`, `List<StockQuote>`, …). The one extra: `StockNewsResponse.updated()` exposes the feed's scalar update time (it sits at the response root, not on each row).
+No new model concepts — `Stock*Response` are thin subclasses of `AbstractMarketDataResponse<T>` (see [Options guide §2](OPTIONS_REVIEW_GUIDE.md#2-the-response-model) for the full shape). `values()` is the flat payload per endpoint (`List<StockCandle>`, `List<StockQuote>`, …). The one endpoint extra: `StockNewsResponse.updated()` exposes the feed's scalar update time (it sits at the response root, not on each row).
 
-What to check: `values()` return types match the wire shape per endpoint; `updated()` is only on `StockNewsResponse`.
+**New SDK-wide accessor:** `MarketDataResponse.rateLimit()` returns the `RateLimitSnapshot` parsed from this response's own `x-api-ratelimit-*` headers (§8.2) — request-scoped, distinct from the client-level `client.getRateLimits()`. `null` when the four headers weren't all present. §9.10.
+
+What to check: `values()` return types match the wire shape per endpoint; `updated()` is only on `StockNewsResponse`; `rateLimit()` is populated from the per-response headers (the merged-chunk candle response reflects the final slice's headers, §9.9).
 
 ---
 
@@ -194,6 +199,8 @@ Verify: `quotesBatchesSymbolsInOneRequest` / `pricesBatchesSymbolsInOneRequest`
 | 9.6 | **`StockResolution` is a value type** | Not an enum — the API's resolution family is open-ended. Factories validate positivity; `of(String)` passes an arbitrary token through. |
 | 9.7 | **wire vs. Java names** | Builder `week52(...)` → wire `52week`; builder `candle(...)` → wire `candle`; `adjustSplits`/`adjustDividends` → wire `adjustsplits`/`adjustdividends`. |
 | 9.8 | **`mode=cached` is quote-only** | The backend rejects `cached` on list endpoints (candles/news/earnings) — a consumer concern, not enforced by the SDK. |
+| 9.9 | **Candle auto-chunking (§12)** | An intraday request with a `from` bound spanning > ~1 year is split into year-sized sub-requests (`candleChunks`), fanned out through the 50-permit pool and merged (`candlesAsync` typed; `StocksCsvResource.mergeCsvBodies` for CSV). `StockResolution.isIntraday()` is the trigger. When split, the merged response's metadata (status/json/`rateLimit`) reflects the final slice. |
+| 9.10 | **Per-response rate limit (§8.2)** | `MarketDataResponse.rateLimit()` is parsed from each response's own `x-api-ratelimit-*` headers (in `AbstractMarketDataResponse`, SDK-wide) — request-scoped, distinct from client-level `getRateLimits()`. |
 
 ---
 
@@ -204,20 +211,22 @@ Do **not** flag as missing — deferred, documented in [`PR.md`](../PR.md):
 - **ADR-008 accept + ADR-009**, and the `docs/java-sdk-requirements.md` per-resource section that depends on an accepted source ADR.
 - **HTML facet exposure** — package-private until the backend serves `format=html`.
 - **SDK-wide setter de-duplication** — the universal-param setters are copy-pasted per resource; a self-typed-base refactor is tracked for before v1.
-- **§13 JaCoCo 100% threshold**, **§8 per-response rate-limit snapshot** — unchanged from the options PR.
+- **§13 JaCoCo 100% threshold** — unchanged from the options PR.
 - **`funds`/`markets`** — adopt this convention next.
 
 ---
 
 ## Reviewer checklist
 
-- [ ] Shared layers untouched except the additive `MarketDataDates.parseDateOrTimestampField`; no existing parser modified.
+- [ ] Shared layers changed only additively: `MarketDataDates.parseDateOrTimestampField` and `MarketDataResponse.rateLimit()` (implemented once in `AbstractMarketDataResponse`); no existing parser modified; options/utilities responses inherit `rateLimit()` with no per-type change.
 - [ ] Requests: Builder per endpoint, no `String` overloads; `StockResolution` value-type factories validate; shared `validateWindow` (date XOR range, countback pairs with `to`).
 - [ ] Query translation: every `*Spec` reads every getter; wire names correct (`52week`, `candle`, `adjustsplits`, …); `symbols` comma-joined; paths encoded.
 - [ ] Nullable + Option A: every row field `@Nullable`; row builders lenient; per-endpoint **required-column sets** are right (quote 11-standard required, OHLC/52-week optional; earnings only symbol/date/updated required); strict-by-default preserved.
 - [ ] `news` deserializer: per-row arrays + scalar `updated` (null on date-bounded); article fields strict; envelope handling.
 - [ ] `quotes`/`prices` are a single batched request (one response, N rows) — not a fan-out map.
 - [ ] Facets: `asCsv()` covers every endpoint, returns `CsvResponse`, adds `human`/`headers`; `asHtml()` package-private.
 - [ ] Date handling: daily date-only vs. intraday full timestamp both decode; `updated` uses the plain timestamp parser.
-- [ ] Unit (`./gradlew build`, 719 tests) and integration (`MARKETDATA_RUN_INTEGRATION_TESTS=true ./gradlew integrationTest`) green; `make demo-stocks` runs against the mock server.
+- [ ] §12 candle chunking: `isIntraday()` classifier correct; only intraday + `from`-bounded ranges split; slices contiguous/non-overlapping (`to` exclusive); concurrent via the 50-permit pool; merged in chronological order; non-intraday / no-`from` stay single-request; CSV path merges with header dedup.
+- [ ] §8.2 per-response rate limit: `rateLimit()` parsed from each response's headers; `null` when absent; request-scoped (distinct from `getRateLimits()`).
+- [ ] Unit (`./gradlew build`, 728 tests) and integration (`MARKETDATA_RUN_INTEGRATION_TESTS=true ./gradlew integrationTest`) green; `make demo-stocks` runs against the mock server.
 - [ ] Deferred items (§10) understood and not blocking.

examples/consumer-test/src/main/java/com/marketdata/consumer/QuickstartApp.java

@@ -309,6 +309,15 @@ private static void stocksExamples(MarketDataClient client) {
                       .to(LocalDate.now())
                       .build());
       Console.ok(r.values().size() + " daily candles fetched");
+      // §8.2: each response carries its own rate-limit snapshot (request-scoped).
+      if (r.rateLimit() != null) {
+        Console.info(
+            "rate limit (from this response): "
+                + r.rateLimit().remaining()
+                + "/"
+                + r.rateLimit().limit()
+                + " remaining");
+      }
     } catch (AuthenticationError e) {
       Console.info("401 — set MARKETDATA_TOKEN (env or .env) to exercise the stocks endpoints.");
     } catch (MarketDataException e) {

examples/consumer-test/src/main/java/com/marketdata/consumer/StocksApp.java

@@ -34,7 +34,10 @@
  *   <li>{@code columns} projection: requested fields populate, fields you did <em>not</em> ask for
  *       come back {@code null} with <strong>no error</strong>;
  *   <li>Option A failures: a required column you <em>did</em> request (or didn't project away) that
- *       the API omits raises a {@link ParseError}.
+ *       the API omits raises a {@link ParseError};
+ *   <li>§12 candle auto-chunking: an intraday range over a year splits into concurrent sub-requests
+ *       and merges into one response;
+ *   <li>§8.2 per-response rate limits: {@code rateLimit()} parsed from each response's headers.
  * </ul>
  *
  * <p>Each scenario scripts the mock server's response with {@link MockServerControl#script}.
@@ -95,13 +98,76 @@ public static void main(String[] args) {
 
     try (var client = new MarketDataClient("token", MockServerControl.BASE_URL, null, false)) {
       everyEndpointWithParams(mock, client);
+      candleAutoChunking(mock, client);
+      perResponseRateLimit(mock, client);
       csvFacet(mock, client);
       columnsProjectionDoesNotFail(mock, client);
       optionARequestedColumnMissingFails(mock, client);
       strictByDefaultMissingColumnFails(mock, client);
     }
   }
 
+  // ---------- §12 candle auto-chunking ----------
+
+  private static void candleAutoChunking(MockServerControl mock, MarketDataClient client) {
+    Console.header("§12 candle auto-chunking — intraday range > 1 year splits + merges");
+    mock.reset();
+    // A 3-year HOURLY (intraday) range splits into 4 year-sized sub-requests, fetched concurrently
+    // and merged. Script one candle body per slice; each returns 2 rows → 8 merged.
+    mock.script(
+        List.of(
+            Step.of(200, CANDLES), Step.of(200, CANDLES), Step.of(200, CANDLES),
+            Step.of(200, CANDLES)));
+    Console.step("candles(hours(1), from=2020-01-01, to=2023-01-01) — auto-split");
+    var resp =
+        client
+            .stocks()
+            .candles(
+                StockCandlesRequest.builder(StockResolution.hours(1), "AAPL")
+                    .from(LocalDate.of(2020, 1, 1))
+                    .to(LocalDate.of(2023, 1, 1))
+                    .build());
+    Console.ok(
+        "candles.values() → "
+            + resp.values().size()
+            + " bars merged from "
+            + mock.stats().requests()
+            + " concurrent sub-requests (one continuous series, transparent to the caller)");
+    Console.info("Daily/weekly/… resolutions or no `from` bound → a single request, no chunking.");
+  }
+
+  // ---------- §8.2 per-response rate limit ----------
+
+  private static void perResponseRateLimit(MockServerControl mock, MarketDataClient client) {
+    Console.header("§8.2 per-response rate limit — rateLimit() off each response");
+    mock.reset();
+    // Script the four x-api-ratelimit-* headers on the response; the SDK parses them per response.
+    mock.script(
+        Step.of(200, QUOTE_FULL)
+            .withHeader("x-api-ratelimit-limit", "100000")
+            .withHeader("x-api-ratelimit-remaining", "99997")
+            .withHeader("x-api-ratelimit-reset", "1735689600")
+            .withHeader("x-api-ratelimit-consumed", "3"));
+    Console.step("quote(\"AAPL\").rateLimit() — parsed from THIS response's headers");
+    var resp = client.stocks().quote(StockQuoteRequest.of("AAPL"));
+    var rl = resp.rateLimit();
+    if (rl != null) {
+      Console.ok(
+          "rateLimit() → remaining="
+              + rl.remaining()
+              + "/"
+              + rl.limit()
+              + " consumed="
+              + rl.consumed()
+              + " reset="
+              + rl.reset());
+    } else {
+      Console.fail("expected a rate-limit snapshot from the response headers");
+    }
+    Console.info(
+        "Request-scoped — distinct from client.getRateLimits() (the client-wide latest snapshot).");
+  }
+
   // ---------- every endpoint ----------
 
   private static void everyEndpointWithParams(MockServerControl mock, MarketDataClient client) {