fix: CustomItemSerializer not applied correctly in queries and SqlParameter [Cosmos]

[xinlian12]

Summary

Fixes #45521

Two bugs with CustomItemSerializer:

Bug 1 Deserialization failures in ORDER BY/GROUP BY/aggregate/DISTINCT queries

When customItemSerializer is configured on CosmosClientBuilder, complex queries (ORDER BY, GROUP BY, VALUE, COUNT, SUM, DISTINCT, HybridSearch) fail because the custom serializer leaks into the internal query pipeline and is used to deserialize internal SDK structures (OrderByRowResult, Document, etc.).

Root cause: PipelinedDocumentQueryExecutionContext.createBaseComponentFunction() called setCustomItemSerializer(null) on cloned request options, but RxDocumentClientImpl.getEffectiveItemSerializer() falls back to the client-level defaultCustomSerializer when the request-level serializer is null.

Fix: Set CosmosItemSerializer.DEFAULT_SERIALIZER instead of null on internal request options. This causes getEffectiveItemSerializer() to return DEFAULT_SERIALIZER immediately without falling through to the client-level custom serializer.

Bug 2 SqlParameter ignores customItemSerializer

SqlParameter.setValue() delegates to JsonSerializable.set() which hardcodes INTERNAL_DEFAULT_SERIALIZER. Custom serialization settings (e.g., dates as ISO strings) are never applied to query parameters.

Fix: Added rawValue field to SqlParameter to retain the original value, plus applySerializer() to re-serialize with a custom serializer. Called from PipelinedQueryExecutionContextBase.createAsync() where the effective serializer is known.

Changes

File	Change
PipelinedDocumentQueryExecutionContext.java	null DEFAULT_SERIALIZER in 4 internal pipeline paths
PipelinedQueryExecutionContextBase.java	Apply serializer to SqlParameters at query execution time
SqlParameter.java	Added rawValue field + applySerializer() method
SqlQuerySpec.java	Added applySerializerToParameters()

Testing

Build compiles cleanly. Existing tests unaffected. Additional test cases for ORDER BY/GROUP BY/aggregate queries with custom serializer and SqlParameter serialization should be added.

[Copilot]

Pull request overview

Fixes Cosmos DB query behavior when a CustomItemSerializer is configured by preventing it from deserializing internal SDK query structures, and by ensuring query parameters can be serialized with the effective item serializer.

Changes:

• Force internal query pipeline request options to use CosmosItemSerializer.DEFAULT_SERIALIZER (instead of null) to avoid leaking client-level custom serializers into internal deserialization.
• Add internal plumbing to re-serialize SqlParameter values using the effective serializer at query execution time.
• Introduce internal bridge method to apply parameter re-serialization across packages.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/models/SqlQuerySpec.java	Adds internal method to apply an effective serializer to all parameters.
sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/models/SqlParameter.java	Stores raw parameter values and adds internal re-serialization hook.
sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/models/ModelBridgeInternal.java	Exposes internal bridge method to apply serializer to query parameters.
sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/implementation/query/PipelinedQueryExecutionContextBase.java	Applies effective serializer to parameters at query execution time.
sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/implementation/query/PipelinedDocumentQueryExecutionContext.java	Forces DEFAULT serializer on internal pipeline request options to prevent custom serializer leaks.
.coding-harness/spec.json	Adds harness metadata/spec for the reported issue and intended fix.

[xinlian12]

@sdkReviewAgent-2

[xinlian12]

✅ Review complete (54:56)

No new comments — existing review coverage is sufficient.

_{Steps: cross-sdk, history, past-prs, quality, sanity-check, synthesis, test-coverage}

[xinlian12]

/azp run java - cosmos - spark

[xinlian12]

/azp run java - cosmos - kafka

[xinlian12]

/azp run java - cosmos - tests

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[FabianMeiswinkel]

LGTM

[FabianMeiswinkel]

LGTM - but there are still a few test issues

[xinlian12]

/azp run java - cosmos - kafka

[xinlian12]

/azp run java - cosmos - spark

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[xinlian12]

/azp run java - cosmos - tests

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[xinlian12]

/azp run java - cosmos - kafka

[xinlian12]

/azp run java - cosmos - spark

[xinlian12]

/azp run java - cosmos - tests

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[xinlian12]

@sdkReviewAgent-2

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[xinlian12]

🔴 Blocking — Correctness: applySerializer is ineffective for scalar SqlParameter values

The CosmosItemSerializer.serialize() contract returns Map<String, Object>, which fundamentally cannot represent scalar values. For custom serializers like BasicCustomItemSerializer, serialize(scalar) calls customMapper.convertValue(scalar, Map.class), which always throws for non-Map-able types (String, Integer, Instant, etc.). The catch(Throwable) block silently swallows the exception, and the parameter retains its default-serialized representation.

Concrete failure trace for this test:

1. Document created with BasicCustomItemSerializer (WRITE_DATES_AS_TIMESTAMPS=false) → createdAt stored as ISO string "2026-03-15T10:30:00Z"
2. SqlParameter("@createdAt", Instant) → setValue() stores rawValue=Instant and default-serializes to epoch number 1773837000 (SDK's ObjectMapper has WRITE_DATES_AS_TIMESTAMPS=true)
3. Clone (line 78): getValue(Object.class) reads the epoch number from JSON property bag → clone's rawValue = Long(1773837000) (type lost)
4. applySerializer(BasicCustomItemSerializer) → serialize(Long) → convertValue(Long, Map.class) → throws IllegalArgumentException
5. catch(Throwable) fires → parameter keeps epoch number 1773837000
6. Query: c.createdAt = @createdAt compares string "2026-03-15T10:30:00Z" vs number 1773837000 → 0 results
7. assertThat(results).hasSize(1) → test fails

This means Bug 2's fix (SqlParameter custom serialization) is a no-op for the exact scenario described in issue #45521 — scalar query parameters with types where the custom serializer changes the wire format.

Root cause: DefaultCosmosItemSerializer.serialize() handles scalars via PrimitiveJsonNodeMap (an internal type), but custom serializers can't — the Map<String, Object> return type makes it impossible to return a scalar from serialize().

Suggested approaches (pick one):

(a) Use the serializer's ObjectMapper directly for scalar parameter values:

void applySerializer(CosmosItemSerializer serializer) {
    if (this.rawValue != null && serializer != null) {
        ObjectMapper mapper = serializer.getItemObjectMapper();
        JsonNode node = mapper.convertValue(this.rawValue, JsonNode.class);
        this.jsonSerializable.getPropertyBag().set("value", node);
    }
}

But this requires BasicCustomItemSerializer (and customer serializers) to call setItemObjectMapper(customMapper) — document this contract.

(b) Add a serializeToNode(T item) method to CosmosItemSerializer with a default implementation using getItemObjectMapper(), and use that in applySerializer instead of serialize().

⚠️ AI-generated review — may be incorrect. Agree? → resolve the conversation. Disagree? → reply with your reasoning.

[xinlian12]

🔴 Blocking — Correctness: Clone via getValue(Object.class) loses the original Java type

This line reconstructs the SqlParameter value by reading from the JSON property bag (getObject("value", Object.class) → getValue(JsonNode)), not from rawValue. The getValue(JsonNode) method converts JSON nodes back to Java primitives: LongNode → Long, DecimalNode → Double, TextNode → String.

For an Instant parameter that was default-serialized as epoch-seconds (a numeric node), getValue(Object.class) returns a Long — the clone's rawValue is Long, not the original Instant. When applySerializer later tries to re-serialize this Long with the custom serializer, it serializes the wrong type.

Fix: Expose rawValue through a package-private accessor and use it for cloning:

// In SqlParameter.java (package-private):
Object getRawValue() { return this.rawValue; }

// Here:
clonedParams.add(new SqlParameter(p.getName(), p.getRawValue()));

Or add a package-private copy constructor:

SqlParameter(SqlParameter original) {
    this.jsonSerializable = new JsonSerializable(
        original.jsonSerializable.getPropertyBag().deepCopy());
    this.rawValue = original.rawValue;
}

This is a prerequisite for any fix to the applySerializer scalar issue — even after serialize() is made to handle scalars, the clone still needs to preserve the original Java type.

⚠️ AI-generated review — may be incorrect. Agree? → resolve the conversation. Disagree? → reply with your reasoning.

[xinlian12]

🟡 Recommendation — Error Handling: catch(Throwable) catches fatal JVM errors; log level too low

Two issues with this error handling:

1. Throwable catches Error subclasses — OutOfMemoryError, StackOverflowError, ThreadDeath, and VirtualMachineError are fatal JVM conditions that should never be silently swallowed. The intended use case (Spark connector stubs throwing UnsupportedOperationException) is an Exception, not an Error.

2. LOGGER.debug() masks incorrect query results — When the serializer fails and falls back to default serialization, the parameter value may have a different wire representation than the document field (e.g., epoch number vs ISO string). The query silently returns zero results. At DEBUG level, this is invisible in production. The log message says "does not support serialize()" but the actual failure mode (scalar-to-Map conversion failure) is different and much harder to diagnose.

// Suggested fix:
} catch (Exception e) {
    LOGGER.warn(
        "Custom serializer '{}' failed to serialize SqlParameter value of type '{}'; "
            + "falling back to default serialization. Query results may not match if the "
            + "custom serializer changes the wire format for this type.",
        serializer.getClass().getSimpleName(),
        this.rawValue.getClass().getSimpleName(),
        e);
}

⚠️ AI-generated review — may be incorrect. Agree? → resolve the conversation. Disagree? → reply with your reasoning.

[xinlian12]

✅ Review complete (50:22)

Posted 3 inline comment(s).

_{Steps: ✓ context, correctness, cross-sdk, design, history, past-prs, synthesis, test-coverage}