Skip to content

NCBC-4244: Raw byte[] sub-document reads over couchbase2#137

Merged
davidkelly merged 3 commits into
masterfrom
dk/ncbc-4244
Jul 14, 2026
Merged

NCBC-4244: Raw byte[] sub-document reads over couchbase2#137
davidkelly merged 3 commits into
masterfrom
dk/ncbc-4244

Conversation

@davidkelly

Copy link
Copy Markdown
Contributor

Motivation

Completing the byte[] sub-document work over couchbase2 (on top of the NCBC-4243 classic fix) surfaced three gaps:

  • StellarCollection.LookupInAsync/MutateInAsync silently ignored the per-operation Transcoder and always used the cluster default, unlike the classic SDK (OperationConfigurator honors opts.Transcoder). A custom transcoder passed to LookupIn/MutateIn over couchbase2 was dropped.

  • contentAs(byte[]) on a sub-document is a raw-bytes passthrough cross-SDK (the Java SDK's default serializer returns byte[] as-is). .NET's default serializers encode byte[] as a Base64 JSON string -- which Queue<byte[]>/ List<byte[]> (CBSE-22994) rely on -- so a bare ContentAs<byte[]> Base64- decodes, and there is no way to read the raw bytes of a fragment. The FIT test contentAsBytesGetLookup needs those raw bytes.

  • The ContentFlags:0 used by the Stellar sub-document results was an undocumented "// TODO".

Modifications

  • StellarCollection: honor opts.Transcoder in LookupInAsync (transcoder = opts.Transcoder ?? cluster default) and MutateInAsync (pass the already-computed transcoder into MutateInResult).

  • Add RawByteArraySerializer, an ITypeSerializer decorator: byte[] is raw passthrough both ways, other types delegate to an inner serializer (preserving its JSON semantics/converters/AOT). Wrapping the configured serializer via a per-op transcoder yields raw sub-document bytes without changing the SDK default. Per-op use, not cluster-wide.

  • Document why Stellar LookupInResult/MutateInResult use ContentFlags:0 (protostellar carries no per-spec content flags; sub-doc fragments are JSON; whole-doc reads that need flags use Get/GetResponse).

  • FIT performer (fit/Couchbase.FitPerformer): in the LookupIn path, when a spec reads byte[], supply a RawByteArraySerializer via a per-op transcoder so ContentAs<byte[]> returns the raw fragment bytes.

  • Add and expand the Stellar combination tests: Queue<byte[]> / List<byte[]> round-trip, whole-doc byte[] via RawBinaryTranscoder, and a LookupIn per-op transcoder regression, validating the CBSE-22994 fix over couchbase2 / protostellar. These require the couchbase2 gateway (port 18098) and are not run in CI unless that gateway is available.

Results

Unit tests pass (including a new RawByteArraySerializer test). The new and updated Stellar combination byte[] tests pass, and the FIT contentAsBytesGetLookup test now passes over couchbase2 (it failed before this change).

Motivation
==========
Completing the byte[] sub-document work over couchbase2 (on top of the
NCBC-4243 classic fix) surfaced three gaps:

- StellarCollection.LookupInAsync/MutateInAsync silently ignored the
  per-operation Transcoder and always used the cluster default, unlike the
  classic SDK (OperationConfigurator honors opts.Transcoder). A custom
  transcoder passed to LookupIn/MutateIn over couchbase2 was dropped.

- contentAs(byte[]) on a sub-document is a raw-bytes passthrough cross-SDK
  (the Java SDK's default serializer returns byte[] as-is). .NET's default
  serializers encode byte[] as a Base64 JSON string -- which Queue<byte[]>/
  List<byte[]> (CBSE-22994) rely on -- so a bare ContentAs<byte[]> Base64-
  decodes, and there is no way to read the raw bytes of a fragment. The FIT
  test contentAsBytesGetLookup needs those raw bytes.

- The ContentFlags:0 used by the Stellar sub-document results was an
  undocumented "// TODO".

Modifications
=============
- StellarCollection: honor opts.Transcoder in LookupInAsync (transcoder =
  opts.Transcoder ?? cluster default) and MutateInAsync (pass the
  already-computed transcoder into MutateInResult).

- Add RawByteArraySerializer, an ITypeSerializer decorator: byte[] is raw
  passthrough both ways, other types delegate to an inner serializer
  (preserving its JSON semantics/converters/AOT). Wrapping the configured
  serializer via a per-op transcoder yields raw sub-document bytes without
  changing the SDK default. Per-op use, not cluster-wide.

- Document why Stellar LookupInResult/MutateInResult use ContentFlags:0
  (protostellar carries no per-spec content flags; sub-doc fragments are
  JSON; whole-doc reads that need flags use Get/GetResponse).

- FIT performer (fit/Couchbase.FitPerformer): in the LookupIn path, when a
  spec reads byte[], supply a RawByteArraySerializer via a per-op transcoder
  so ContentAs<byte[]> returns the raw fragment bytes.

- Add and expand the Stellar combination tests: Queue<byte[]> /
  List<byte[]> round-trip, whole-doc byte[] via RawBinaryTranscoder, and a
  LookupIn per-op transcoder regression, validating the CBSE-22994 fix over
  couchbase2 / protostellar. These require the couchbase2 gateway (port
  18098) and are not run in CI unless that gateway is available.

Results
=======
Unit tests pass (including a new RawByteArraySerializer test). The new and
updated Stellar combination byte[] tests pass, and the FIT
contentAsBytesGetLookup test now passes over couchbase2 (it failed before
this change).

Co-Authored-By: jmorris <jeffrymorris@gmail.com>
@CLAassistant

CLAassistant commented Jul 14, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR completes raw byte[] sub-document read support over couchbase2/protostellar by (1) honoring per-operation transcoders in Stellar subdoc APIs and (2) introducing a serializer decorator which treats byte[] as raw passthrough for opt-in per-operation use, enabling cross-SDK contentAs(byte[]) parity without changing the SDK’s default JSON byte[] Base64 behavior.

Changes:

  • Ensure StellarCollection.LookupInAsync/MutateInAsync use opts.Transcoder (fallback to cluster default) when creating subdoc results.
  • Add RawByteArraySerializer (ITypeSerializer decorator) and unit tests validating raw passthrough for byte[] and delegation for other types.
  • Update FIT performer and Stellar combination tests to validate byte[] round-trips and per-op transcoder behavior for subdoc/fulldoc scenarios.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/Couchbase.UnitTests/Core/IO/Serializers/RawByteArraySerializerTests.cs Adds unit coverage for RawByteArraySerializer byte[] passthrough + delegation behavior.
tests/Couchbase.Stellar.CombinationTests/KeyValue/StellarByteArraySubdocTests.cs Adds regression/combination coverage for byte[] round-trips and per-op transcoder behavior over protostellar.
src/Couchbase/Stellar/Util/ResultTypes.cs Documents why subdoc results use ContentFlags: 0 under protostellar.
src/Couchbase/Stellar/KeyValue/StellarCollection.cs Honors per-operation transcoders for LookupIn/MutateIn result decoding.
src/Couchbase/Core/IO/Serializers/RawByteArraySerializer.cs Introduces RawByteArraySerializer decorator enabling raw byte[] passthrough per-operation.
fit/Couchbase.FitPerformer/Workload/ClusterBucketScopeCollectionCommandExecutor.cs Uses a per-op transcoder to enable raw fragment bytes for FIT contentAs<byte[]> subdoc reads.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Decorate the cluster's configured serializer (custom serializer when
UseCustomSerializer=true) instead of hard-coding DefaultSerializer, so
non-byte[] specs in a mixed-spec LookupIn keep the cluster's serialization.
Addresses Copilot review feedback on PR #137.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
FIT (LookupInGetTest.contentAsBytesGetLookup) showed the byte[] raw-passthrough
transcoder was only applied to the plain LookupIn path; LookupInAllReplicas and
LookupInAnyReplica still used the default serializer, so contentAs<byte[]> over
the replica-read APIs base64-decoded and threw JsonReaderException. Factor the
transcoder selection into a shared helper and apply it to all three read paths.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@davidkelly davidkelly merged commit 020ddc8 into master Jul 14, 2026
11 checks passed
@davidkelly davidkelly deleted the dk/ncbc-4244 branch July 14, 2026 22:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants