Fix search profile fetch schema to match server serialization#1147
Fix search profile fetch schema to match server serialization#1147minseoky wants to merge 3 commits into
Conversation
- Change ShardProfile's fetch from a single FetchProfile object to an array, matching FetchProfileShardResult#toXContent which always serializes it with startArray - Add missing inbound_network_time_in_millis and outbound_network_time_in_millis fields to ShardProfile - Rewrite FetchProfileBreakdown to match the server's FetchTimingType enum: add missing timing fields, remove next_reader which the server never emits, and use int64 since breakdown values are Java longs - Add a spec test for search with profile enabled Signed-off-by: Minseok Choi <minseok184@gmail.com>
Signed-off-by: Minseok Choi <minseok184@gmail.com>
- fetch: fetch phase profiling was introduced in OpenSearch 3.2.0 (opensearch-project/OpenSearch#18664) - inbound/outbound_network_time_in_millis: added in OpenSearch 2.0.0 (opensearch-project/OpenSearch#1360) Signed-off-by: Minseok Choi <minseok184@gmail.com>
|
Note on delivery: the weekly code generation in opensearch-java is currently failing against the latest spec ( The schemas in this PR are not affected by that limitation: regenerating with only this change applied on top of the spec snapshot committed in opensearch-java succeeds, changing just |
Description
The
profilesection of the search response is incorrectly specified, causing generated clients to crash when deserializing responses withprofile: true(reported in opensearch-project/opensearch-java#1965).
1.
ShardProfile.fetchis an array, not a single object (the reported bug)The server always serializes
fetchas an array — seeFetchProfileShardResult#toXContent(
builder.startArray(FETCH)). The spec defined it as a singleFetchProfileobject, so the generated Java client fails with:A spec test is required for this change, and the test framework validates the entire response payload against the spec (any field returned by the server but not declared in the spec is an error).
As a result, the
profile: truetest cannot pass with thefetchfix alone — it surfaced two more discrepancies inthe same response, which are fixed in this PR as well:
2.
ShardProfileis missing the network time fieldsThe server unconditionally writes
inbound_network_time_in_millisandoutbound_network_time_in_millisfor every shard profile — seeSearchProfileShardResults#toXContent(present since OpenSearch 2.0.0, opensearch-project/OpenSearch#1360).
3.
FetchProfileBreakdowndoes not match the server'sFetchTimingTypeRewrote the schema to match
FetchTimingType(introduced with fetch profiling in OpenSearch 3.2.0, opensearch-project/OpenSearch#18664):
build_sub_phase_processors,create_stored_fields_visitor,get_next_reader,set_next_reader(+ their_countcounterparts)next_reader/next_reader_count, which the server never emitsformat: int32toint64: breakdown values are Javalongs (AbstractProfileBreakdown#toBreakdownMapreturnsMap<String, Long>), and the timing values are nanoseconds, which canexceed int32's ~2.1s range. This also matches the siblingQueryBreakdown/AggregationBreakdownschemas, which already useint64. Note that Fix type and schema mismatches across the spec #1110 touches the same schema with the same int64 direction, so a small merge conflict is possible depending on ordering.Testing
Added
tests/default/_core/search/profile.yaml(the second and third fixes above are prerequisites for this test to pass) and verified against a local OpenSearch 3.7.0 cluster:RESPONSE PAYLOAD SCHEMAfails with the mismatches abovePASSED, with no regressions across the rest of the_core/searchtest suitenpm run lint:specpassesSince this spec is the source of truth for all generated clients, the fix benefits the other language clients as well.
Issues Resolved
Addresses opensearch-project/opensearch-java#1965 — the Java client fix will follow automatically via the weekly code generation workflow once this spec change is released.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.