NCBC-4244: Raw byte[] sub-document reads over couchbase2#137
Merged
Conversation
Motivation ========== Completing the byte[] sub-document work over couchbase2 (on top of the NCBC-4243 classic fix) surfaced three gaps: - StellarCollection.LookupInAsync/MutateInAsync silently ignored the per-operation Transcoder and always used the cluster default, unlike the classic SDK (OperationConfigurator honors opts.Transcoder). A custom transcoder passed to LookupIn/MutateIn over couchbase2 was dropped. - contentAs(byte[]) on a sub-document is a raw-bytes passthrough cross-SDK (the Java SDK's default serializer returns byte[] as-is). .NET's default serializers encode byte[] as a Base64 JSON string -- which Queue<byte[]>/ List<byte[]> (CBSE-22994) rely on -- so a bare ContentAs<byte[]> Base64- decodes, and there is no way to read the raw bytes of a fragment. The FIT test contentAsBytesGetLookup needs those raw bytes. - The ContentFlags:0 used by the Stellar sub-document results was an undocumented "// TODO". Modifications ============= - StellarCollection: honor opts.Transcoder in LookupInAsync (transcoder = opts.Transcoder ?? cluster default) and MutateInAsync (pass the already-computed transcoder into MutateInResult). - Add RawByteArraySerializer, an ITypeSerializer decorator: byte[] is raw passthrough both ways, other types delegate to an inner serializer (preserving its JSON semantics/converters/AOT). Wrapping the configured serializer via a per-op transcoder yields raw sub-document bytes without changing the SDK default. Per-op use, not cluster-wide. - Document why Stellar LookupInResult/MutateInResult use ContentFlags:0 (protostellar carries no per-spec content flags; sub-doc fragments are JSON; whole-doc reads that need flags use Get/GetResponse). - FIT performer (fit/Couchbase.FitPerformer): in the LookupIn path, when a spec reads byte[], supply a RawByteArraySerializer via a per-op transcoder so ContentAs<byte[]> returns the raw fragment bytes. - Add and expand the Stellar combination tests: Queue<byte[]> / List<byte[]> round-trip, whole-doc byte[] via RawBinaryTranscoder, and a LookupIn per-op transcoder regression, validating the CBSE-22994 fix over couchbase2 / protostellar. These require the couchbase2 gateway (port 18098) and are not run in CI unless that gateway is available. Results ======= Unit tests pass (including a new RawByteArraySerializer test). The new and updated Stellar combination byte[] tests pass, and the FIT contentAsBytesGetLookup test now passes over couchbase2 (it failed before this change). Co-Authored-By: jmorris <jeffrymorris@gmail.com>
There was a problem hiding this comment.
Pull request overview
This PR completes raw byte[] sub-document read support over couchbase2/protostellar by (1) honoring per-operation transcoders in Stellar subdoc APIs and (2) introducing a serializer decorator which treats byte[] as raw passthrough for opt-in per-operation use, enabling cross-SDK contentAs(byte[]) parity without changing the SDK’s default JSON byte[] Base64 behavior.
Changes:
- Ensure
StellarCollection.LookupInAsync/MutateInAsyncuseopts.Transcoder(fallback to cluster default) when creating subdoc results. - Add
RawByteArraySerializer(ITypeSerializerdecorator) and unit tests validating raw passthrough forbyte[]and delegation for other types. - Update FIT performer and Stellar combination tests to validate byte[] round-trips and per-op transcoder behavior for subdoc/fulldoc scenarios.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/Couchbase.UnitTests/Core/IO/Serializers/RawByteArraySerializerTests.cs | Adds unit coverage for RawByteArraySerializer byte[] passthrough + delegation behavior. |
| tests/Couchbase.Stellar.CombinationTests/KeyValue/StellarByteArraySubdocTests.cs | Adds regression/combination coverage for byte[] round-trips and per-op transcoder behavior over protostellar. |
| src/Couchbase/Stellar/Util/ResultTypes.cs | Documents why subdoc results use ContentFlags: 0 under protostellar. |
| src/Couchbase/Stellar/KeyValue/StellarCollection.cs | Honors per-operation transcoders for LookupIn/MutateIn result decoding. |
| src/Couchbase/Core/IO/Serializers/RawByteArraySerializer.cs | Introduces RawByteArraySerializer decorator enabling raw byte[] passthrough per-operation. |
| fit/Couchbase.FitPerformer/Workload/ClusterBucketScopeCollectionCommandExecutor.cs | Uses a per-op transcoder to enable raw fragment bytes for FIT contentAs<byte[]> subdoc reads. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Decorate the cluster's configured serializer (custom serializer when UseCustomSerializer=true) instead of hard-coding DefaultSerializer, so non-byte[] specs in a mixed-spec LookupIn keep the cluster's serialization. Addresses Copilot review feedback on PR #137. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
jeffrymorris
approved these changes
Jul 14, 2026
FIT (LookupInGetTest.contentAsBytesGetLookup) showed the byte[] raw-passthrough transcoder was only applied to the plain LookupIn path; LookupInAllReplicas and LookupInAnyReplica still used the default serializer, so contentAs<byte[]> over the replica-read APIs base64-decoded and threw JsonReaderException. Factor the transcoder selection into a shared helper and apply it to all three read paths. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
jeffrymorris
approved these changes
Jul 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Completing the byte[] sub-document work over couchbase2 (on top of the NCBC-4243 classic fix) surfaced three gaps:
StellarCollection.LookupInAsync/MutateInAsync silently ignored the per-operation Transcoder and always used the cluster default, unlike the classic SDK (OperationConfigurator honors opts.Transcoder). A custom transcoder passed to LookupIn/MutateIn over couchbase2 was dropped.
contentAs(byte[]) on a sub-document is a raw-bytes passthrough cross-SDK (the Java SDK's default serializer returns byte[] as-is). .NET's default serializers encode byte[] as a Base64 JSON string -- which Queue<byte[]>/ List<byte[]> (CBSE-22994) rely on -- so a bare ContentAs<byte[]> Base64- decodes, and there is no way to read the raw bytes of a fragment. The FIT test contentAsBytesGetLookup needs those raw bytes.
The ContentFlags:0 used by the Stellar sub-document results was an undocumented "// TODO".
Modifications
StellarCollection: honor opts.Transcoder in LookupInAsync (transcoder = opts.Transcoder ?? cluster default) and MutateInAsync (pass the already-computed transcoder into MutateInResult).
Add RawByteArraySerializer, an ITypeSerializer decorator: byte[] is raw passthrough both ways, other types delegate to an inner serializer (preserving its JSON semantics/converters/AOT). Wrapping the configured serializer via a per-op transcoder yields raw sub-document bytes without changing the SDK default. Per-op use, not cluster-wide.
Document why Stellar LookupInResult/MutateInResult use ContentFlags:0 (protostellar carries no per-spec content flags; sub-doc fragments are JSON; whole-doc reads that need flags use Get/GetResponse).
FIT performer (fit/Couchbase.FitPerformer): in the LookupIn path, when a spec reads byte[], supply a RawByteArraySerializer via a per-op transcoder so ContentAs<byte[]> returns the raw fragment bytes.
Add and expand the Stellar combination tests: Queue<byte[]> / List<byte[]> round-trip, whole-doc byte[] via RawBinaryTranscoder, and a LookupIn per-op transcoder regression, validating the CBSE-22994 fix over couchbase2 / protostellar. These require the couchbase2 gateway (port 18098) and are not run in CI unless that gateway is available.
Results
Unit tests pass (including a new RawByteArraySerializer test). The new and updated Stellar combination byte[] tests pass, and the FIT contentAsBytesGetLookup test now passes over couchbase2 (it failed before this change).