Skip to content

Added support to connect and perform CRUD operations with couchbase #3138

Merged
wwbmmm merged 57 commits intoapache:masterfrom
couchbaselabs:master
Nov 25, 2025
Merged

Added support to connect and perform CRUD operations with couchbase #3138
wwbmmm merged 57 commits intoapache:masterfrom
couchbaselabs:master

Conversation

@giriraj-singh-couchbase
Copy link
Copy Markdown
Contributor

@giriraj-singh-couchbase giriraj-singh-couchbase commented Nov 5, 2025

This pull request introduces Couchbase Binary Protocol support to bRPC, enabling communication with Couchbase Server and Capella deployments. It adds a new protocol handler, request/response builders, comprehensive example clients, performance benchmarking tools, and detailed documentation. The changes are grouped below by theme:

Couchbase Binary Protocol Integration

Protocol Layer (src/brpc/policy/):

  • Added couchbase_protocol.h and couchbase_protocol.cpp implementing the complete binary protocol framing, parsing, with support for all standard Couchbase commands (GET, SET, ADD, REPLACE, DELETE, etc.)

High-Level API (src/brpc/):

  • Implemented CouchbaseOperations class in src/brpc/couchbase.h and src/brpc/couchbase.cpp (~3,500 lines), providing simplified APIs for:

    • Authentication: authenticate() and authenticateSSL() for both non-SSL and SSL connections (required for Couchbase Capella)
    • Bucket management: selectBucket() for bucket selection
    • CRUD operations: get(), upsert(), add(), replace(), append(), prepend(), delete_() with Result struct for clean error handling
    • Pipeline batching: beginPipeline(), pipelineRequest(), executePipeline(), clearPipeline() for batching multiple operations in a single network call.
  • Implemented CouchbaseRequest and CouchbaseResponse classes (inherit from NonreflectableMessage) providing low-level request construction and response parsing

  • Implemented CouchbaseManifestManager singleton for thread-safe collection manifest management with automatic caching and refresh

Example Clients

Basic Client (example/couchbase_c++/couchbase_client.cpp - 462 lines):

  • Demonstrates authentication (both SSL and non-SSL), bucket selection, basic CRUD operations, collection-scoped operations, and error handling patterns

Multithreaded Client (example/couchbase_c++/multithreaded_couchbase_client.cpp - 375 lines):

  • Demonstrates two threading patterns:
    • Multiple threads sharing a single CouchbaseOperations instance (same bucket, shared connection)
    • Multiple threads with dedicated CouchbaseOperations instances (different buckets, isolated connections)

Documentation

Comprehensive Guide (docs/en/couchbase_example.md):

Key Features

  • SSL/TLS Support: for Couchbase Capella and secure on-premises deployments
  • Collection Management: Native Couchbase 7.0+ collections with automatic ID resolution, manifest caching.
  • Pipeline Batching: Batch multiple operations.
  • Error Handling: Clean Result struct with success flag, error messages, status codes, and values

What problem does this PR solve?

Added native support to interact with Couchbase servers using the binary protocol, enabling efficient CRUD operations, SSL/TLS encrypted connections and collection management.

Side effects:

  • Performance effects: None

  • Breaking backward compatibility: NO


Check List:

…functionality to handle server side manifest updates.
Unnecessary file
Added an example where a single instance is being shared across the threads when operating on single bucket.
updated the documentation on thread safe operations and fixed small small discrepancies.
Added support to connect and perform CRUD operations with couchbase
Comment thread src/brpc/couchbase.h
Comment thread src/brpc/couchbase.h
Comment thread src/brpc/couchbase.h
Comment thread src/brpc/couchbase.cpp
@wwbmmm wwbmmm requested a review from Copilot November 7, 2025 02:42
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request adds comprehensive support for Couchbase Binary Protocol to bRPC, enabling communication with Couchbase Server clusters using the native binary protocol. The implementation provides both high-level and low-level APIs for key-value operations with support for collections, pipelining, SSL/TLS, and multithreading.

Key Changes:

  • Complete Couchbase Binary Protocol implementation with request/response parsing and serialization
  • High-level CouchbaseOperations API with simple CRUD operations and pipeline support
  • SSL/TLS support for secure connections to Couchbase Capella (cloud)
  • Collection-scoped operations with automatic manifest caching and refresh
  • Multithreading support with both shared and separate instance patterns

Reviewed Changes

Copilot reviewed 12 out of 13 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
src/brpc/proto_base.proto Adds base message types for Couchbase request/response
src/brpc/policy/couchbase_protocol.h Defines protocol structures, enums, and function declarations for Couchbase Binary Protocol
src/brpc/policy/couchbase_protocol.cpp Implements message parsing, serialization, and protocol handling
src/brpc/couchbase.h Defines high-level CouchbaseOperations API with request/response classes and manifest management
src/brpc/couchbase.cpp Implements CRUD operations, authentication, collection management, and pipeline functionality
src/brpc/options.proto Adds PROTOCOL_COUCHBASE enum value for protocol registration
src/brpc/global.cpp Registers Couchbase protocol with bRPC framework
example/couchbase_c++/couchbase_client.cpp Single-threaded example demonstrating all features
example/couchbase_c++/multithreaded_couchbase_client.cpp Multithreaded example with shared and separate instance patterns
example/couchbase_c++/Makefile Build configuration for examples
example/couchbase_c++/CMakeLists.txt CMake build configuration for examples
docs/en/couchbase_example.md Comprehensive documentation with usage patterns and examples
CMakeLists.txt Updates C++ standard to C++17 for modern features

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/brpc/couchbase.cpp Outdated
Comment thread src/brpc/couchbase.cpp Outdated
Comment thread src/brpc/couchbase.h Outdated
Comment thread src/brpc/policy/couchbase_protocol.cpp Outdated
Comment thread example/couchbase_c++/multithreaded_couchbase_client.cpp Outdated
Comment thread example/couchbase_c++/multithreaded_couchbase_client.cpp Outdated
@wwbmmm
Copy link
Copy Markdown
Contributor

wwbmmm commented Nov 7, 2025

@giriraj-singh-couchbase Thank you for your contribution! This is quite a complicated job. Your code looks very good! I have add some comments.

@wwbmmm
Copy link
Copy Markdown
Contributor

wwbmmm commented Nov 7, 2025

A small advice: you can add an unit test for the couchbase protocol, just base on the code in example/couchbase_c++/couchbase_client.cpp

Traditional bRPC coding approach doesn't uses high level functions but provides more control to the user

fixed formatting issues.

fixed the bug in couchbase.cpp where logic to check the cache is empty was inverted
@giriraj-singh-couchbase
Copy link
Copy Markdown
Contributor Author

Hi @wwbmmm, I have added the unit test cases, traditional approach example and updated the documentation regarding it.
Can you take a look at it? Thank you.

@wwbmmm
Copy link
Copy Markdown
Contributor

wwbmmm commented Nov 12, 2025

Hi @wwbmmm, I have added the unit test cases, traditional approach example and updated the documentation regarding it. Can you take a look at it? Thank you.

Most look good, can you remove the using namespace std; in the couchbase.h file? This header file may be included by user, so the using namespace std; may cause some conflicts in user code.

@giriraj-singh-couchbase
Copy link
Copy Markdown
Contributor Author

Hi @wwbmmm, I have added the unit test cases, traditional approach example and updated the documentation regarding it. Can you take a look at it? Thank you.

Most look good, can you remove the using namespace std; in the couchbase.h file? This header file may be included by user, so the using namespace std; may cause some conflicts in user code.

Removed.

@wwbmmm
Copy link
Copy Markdown
Contributor

wwbmmm commented Nov 12, 2025

LGTM
I think the unittest failure is not related to this PR, we can ignore it.

@wwbmmm wwbmmm requested a review from chenBright November 12, 2025 13:07
@giriraj-singh-couchbase
Copy link
Copy Markdown
Contributor Author

LGTM I think the unittest failure is not related to this PR, we can ignore it.

Can you re-run the checks?

@wwbmmm
Copy link
Copy Markdown
Contributor

wwbmmm commented Nov 15, 2025

LGTM I think the unittest failure is not related to this PR, we can ignore it.

Can you re-run the checks?

I have fixed the unittest in this PR: #3153 . You can merge this PR and then re-run the checks.

@giriraj-singh-couchbase
Copy link
Copy Markdown
Contributor Author

giriraj-singh-couchbase commented Nov 18, 2025

Hi @wwbmmm & @chenBright, can I get further feedback on this PR?

@wwbmmm wwbmmm merged commit d2ea819 into apache:master Nov 25, 2025
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants