Commit 0031453
[GatewayV2] Fix over-spanning issue when partial partition keys are provided. (#49688)
* Fix thin-client MULTI_HASH prefix partition key EPK over-span
For a partial (prefix) hierarchical (MULTI_HASH) partition key, the thin-client
(Gateway V2) store model previously sent only StartEpkHash/EndEpkHash routing
headers and no doc-level EPK filter. The proxy therefore resolved the request to
the owning physical partition and returned every co-located document (an
over-span) instead of only those matching the prefix.
ThinClientStoreModel.wrapInHttpRequest now detects a prefix MULTI_HASH key and
sets READ_FEED_KEY_TYPE=EffectivePartitionKeyRange, START_EPK and END_EPK on the
request headers before RntbdRequest.from(), so RntbdRequestHeaders serializes the
prefix EPK sub-range [hash(prefix), hash(prefix) + "FF") as the backend doc-level
filter (hex string as UTF-8 bytes), mirroring the Direct/RNTBD FeedRangeEpkImpl
path. StartEpkHash/EndEpkHash routing headers are retained. QueryPlan requests
and full (non-prefix) keys are unchanged.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* Add thin-client query E2E parity regression tests
Ports ThinClientQueryE2ETest and its ThinClientTestBase helper. The suite
self-baselines every query against a Direct (RNTBD) client and the thin-client
(:10250 HTTP/2) path, asserting identical results. This directly guards the
MULTI_HASH prefix partition-key EPK over-span fix: prefix (single-component)
hierarchical-partition-key queries must return only the narrow matching set,
matching Direct, rather than over-spanning to all co-located documents.
Both files are runtime self-enabling (enableThinClientForTest() + builder-driven
HTTP/2 via setEnabled(true)); no module property or TestSuiteBase changes needed.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* Reuse prefix EPK range on thin-client parallel prefix-query path
The earlier fix (499f928) only scoped the prefix MULTI_HASH read when the
request carried a non-null partition key. The parallel prefix-query path
intentionally nulls the partial hierarchical partition key and instead carries
the narrow prefix effective-partition-key range on the request's feedRange, so
that guard never fired there and the thin-client read still over-spanned to the
whole physical partition.
Mark such requests (prefixPartitionKeyQuery) in
ParallelDocumentQueryExecutionContextBase and, in ThinClientStoreModel, reuse
the already-computed FeedRangeEpkImpl range as the doc-level EPK filter
(START_EPK/END_EPK + StartEpkHash/EndEpkHash) instead of recomputing
getEPKRangeForPrefixPartitionKey from a partition key we no longer have. This
mirrors the Direct/GatewayV1 decision points and honors DRY/SRP.
Also relax assertThinClientEndpointUsed so an all-QueryPlan diagnostics context
passes: in thin-client mode QueryPlan calls resolve via the classic gateway and
are the only requests permitted on a non-thin-client endpoint; any non-QueryPlan
(data) request on a non-thin-client endpoint still fails the assertion.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* Consolidate thin-client prefix EPK RNTBD headers, propagate prefix flag on clone, add readAllItems/readMany prefix HPK tests
ThinClientStoreModel.wrapInHttpRequest now sets all five EPK-specific RNTBD headers (ReadFeedKeyType, StartEpk/EndEpk, StartEpkHash/EndEpkHash) in a single block directly on the RNTBD request, replacing the split request-header-map + post-construction approach. Wire encoding is unchanged (hex-string UTF-8 bytes for StartEpk/EndEpk, decoded hash bytes for the hash headers).
RxDocumentServiceRequest.clone() now copies prefixPartitionKeyQuery so hedged/availability-strategy/retry clones keep the prefix signal and do not over-span.
ThinClientQueryE2ETest adds prefix-HPK regression coverage for readAllItems (ReadFeed) and readManyByPartitionKeys, asserting Direct-vs-thin-client parity and per-tenant scoping (no over-span).
* Simplify prefix-query flag assignment in ParallelDocumentQueryExecutionContextBase
Assign isPrefixPartitionKeyQuery directly from isPartialPartitionKeyQuery(...) and guard only the PARTITION_KEY-setting branch with !isPrefixPartitionKeyQuery, removing the if/else. Behavior is unchanged: a partial (prefix) key still leaves partitionKeyInternal null so the prefix FeedRangeEpkImpl survives createDocumentServiceRequestWithFeedRange.
* Add CHANGELOG entry for thin-client prefix hierarchical partition key over-span fix
* Minimize prefix-query flag change vs main in ParallelDocumentQueryExecutionContextBase
Keep main's single 'if (!isPartialPartitionKeyQuery)' structure; only capture the predicate in isPrefixPartitionKeyQuery and pass it to request.setPrefixPartitionKeyQuery. Removes the if/else.
* Order thin-client prefix detection to prefer the cached feed-range path
In ThinClientStoreModel.wrapInHttpRequest, check the cached prefix feed-range path (isPrefixPartitionKeyQuery + FeedRangeEpkImpl) first and fall back to recomputing from the partition key only for the PK-bearing prefix case. Behavior is unchanged (the two cases are mutually exclusive).
* Address Copilot review: validate all requests in thin-client endpoint assertion
TestSuiteBase.assertThinClientEndpointUsed now validates every request instead of early-returning on the first thin-client match, so a mixed scenario (some data requests via the classic gateway) fails; QueryPlan requests remain exempt and null endpoints are handled. ThinClientTestBase.assertThinClientEndpointUsed delegates to the shared TestSuiteBase implementation, removing duplicated logic that could NPE on a null endpoint and reject valid QueryPlan-via-gateway scenarios.
* Simplify thin-client prefix EPK handling to reuse HTTP EPK headers
Replace the explicit MULTI_HASH prefix-detection and pre-serialization range
computation with a header-presence branch: when the request already carries
START_EPK/END_EPK HTTP headers (populated upstream by FeedRangeEpkImpl for
prefix hierarchical partition keys and explicit EPK-range reads), set only the
additive StartEpkHash/EndEpkHash RNTBD tokens. The ReadFeedKeyType/StartEpk/
EndEpk string tokens are copied verbatim from those HTTP headers by
RntbdRequestHeaders#addStartAndEndKeys during RntbdRequest.from(), so they no
longer need to be set here. Removes now-unused imports.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* Remove dead prefix-query flag and expand HPK thin-client test coverage
Following the simplification of the prefix EPK fix to reuse the existing
StartEpk/EndEpk HTTP headers (a56db9a), the RxDocumentServiceRequest
prefixPartitionKeyQuery flag is no longer read anywhere. Remove the field,
its accessors, the clone copy, and the caller assignment in
ParallelDocumentQueryExecutionContextBase.
Test changes:
- Retrofit CosmosMultiHashTest into the thinclient TestNG group so the
MULTI_HASH prefix over-span coverage also runs against GatewayV2 (proxy
:10250) over HTTP/2, not just the emulator gateway.
- ThinClientQueryE2ETest: correct the prefix-scope Javadoc to describe the
actual header-copy behavior and add a deterministic prefix-partition-key
readAllItems regression.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* Revert ParallelDocumentQueryExecutionContextBase to main
The prefix-query flag on RxDocumentServiceRequest was removed in
03a7c67 after the fix was simplified to reuse the StartEpk/EndEpk
HTTP headers. The only functional edit in this file (the setter call)
went away with it, leaving behind unnecessary refactor noise. Restore
the file to its upstream/main state so the hotfix touches no query
execution code.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* Address final review comments on thin-client EPK over-span fix
Comment 1: Tighten the StartEpkHash/EndEpkHash guard in
ThinClientStoreModel#wrapInHttpRequest to require
READ_FEED_KEY_TYPE == EffectivePartitionKeyRange in addition to the
StartEpk/EndEpk headers. RntbdRequestHeaders#addStartAndEndKeys only
emits the ReadFeedKeyType RNTBD token when that header is present, so
the consumer guard now matches the producer contract and keeps the
string and binary EPK tokens coherent.
Comment 2: Add testExplicitFeedRangeShardedReadParity, which exercises
the explicit user-supplied FeedRange (EPK-range) read path directly by
sharding the multi-physical-partition container into its physical feed
ranges and asserting per-shard parity with the Direct baseline plus a
clean, non-overlapping union over the full fan-out.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* Add mode-aware thin-client endpoint routing assertions to CosmosMultiHashTest
Verify data requests route through the thin-client endpoint (:10250) when the
thinclient group is active, and through the gateway otherwise. Also removes
redundant in-code thin-client enablement (owned by the TestNG config / Maven
profile).
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* Address xinlian12 review comments: docstring accuracy, dead scaffolding cleanup, container-leak fix
- ThinClientQueryE2ETest: clarify prefix-HPK test docstring as routing+SQL-predicate parity guard; cross-reference genuine EPK-header guards
- ThinClientTestBase: convert to stateless final helper holder (drop unused lifecycle scaffolding)
- ThinClientQueryE2ETest: best-effort container cleanup in setup catch block to avoid leaking provisioned containers
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>1 parent 0d3040f commit 0031453
6 files changed
Lines changed: 2405 additions & 16 deletions
File tree
- sdk/cosmos
- azure-cosmos-tests/src/test/java/com/azure/cosmos
- rx
- azure-cosmos
- src/main/java/com/azure/cosmos/implementation
Lines changed: 63 additions & 6 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
| 9 | + | |
9 | 10 | | |
10 | 11 | | |
11 | 12 | | |
| |||
42 | 43 | | |
43 | 44 | | |
44 | 45 | | |
| 46 | + | |
| 47 | + | |
45 | 48 | | |
46 | 49 | | |
47 | 50 | | |
| |||
53 | 56 | | |
54 | 57 | | |
55 | 58 | | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
56 | 62 | | |
57 | 63 | | |
58 | 64 | | |
| |||
61 | 67 | | |
62 | 68 | | |
63 | 69 | | |
| 70 | + | |
64 | 71 | | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
65 | 89 | | |
66 | 90 | | |
67 | 91 | | |
| |||
103 | 127 | | |
104 | 128 | | |
105 | 129 | | |
106 | | - | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
107 | 148 | | |
108 | 149 | | |
109 | 150 | | |
| |||
120 | 161 | | |
121 | 162 | | |
122 | 163 | | |
| 164 | + | |
123 | 165 | | |
124 | 166 | | |
125 | 167 | | |
| |||
170 | 212 | | |
171 | 213 | | |
172 | 214 | | |
173 | | - | |
| 215 | + | |
174 | 216 | | |
175 | 217 | | |
176 | 218 | | |
| |||
186 | 228 | | |
187 | 229 | | |
188 | 230 | | |
| 231 | + | |
189 | 232 | | |
190 | 233 | | |
191 | | - | |
| 234 | + | |
192 | 235 | | |
193 | 236 | | |
194 | 237 | | |
| |||
203 | 246 | | |
204 | 247 | | |
205 | 248 | | |
206 | | - | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
207 | 256 | | |
208 | 257 | | |
209 | 258 | | |
| |||
445 | 494 | | |
446 | 495 | | |
447 | 496 | | |
448 | | - | |
| 497 | + | |
449 | 498 | | |
450 | 499 | | |
451 | 500 | | |
| |||
530 | 579 | | |
531 | 580 | | |
532 | 581 | | |
| 582 | + | |
533 | 583 | | |
534 | 584 | | |
535 | 585 | | |
| |||
645 | 695 | | |
646 | 696 | | |
647 | 697 | | |
648 | | - | |
| 698 | + | |
| 699 | + | |
| 700 | + | |
| 701 | + | |
| 702 | + | |
| 703 | + | |
| 704 | + | |
649 | 705 | | |
650 | 706 | | |
651 | 707 | | |
| |||
671 | 727 | | |
672 | 728 | | |
673 | 729 | | |
| 730 | + | |
674 | 731 | | |
675 | 732 | | |
676 | 733 | | |
| |||
Lines changed: 17 additions & 7 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2710 | 2710 | | |
2711 | 2711 | | |
2712 | 2712 | | |
| 2713 | + | |
| 2714 | + | |
| 2715 | + | |
| 2716 | + | |
| 2717 | + | |
2713 | 2718 | | |
2714 | | - | |
2715 | | - | |
2716 | | - | |
| 2719 | + | |
| 2720 | + | |
| 2721 | + | |
| 2722 | + | |
| 2723 | + | |
2717 | 2724 | | |
2718 | | - | |
2719 | 2725 | | |
2720 | | - | |
2721 | | - | |
2722 | | - | |
| 2726 | + | |
| 2727 | + | |
| 2728 | + | |
| 2729 | + | |
| 2730 | + | |
| 2731 | + | |
| 2732 | + | |
2723 | 2733 | | |
2724 | 2734 | | |
2725 | 2735 | | |
| |||
0 commit comments