Introduce clustered servers by farost · Pull Request #239 · typedb/typedb-protocol

farost · 2025-11-19T14:56:00Z

Release notes: usage and product changes

Introduce cluster support in the TypeDB protocol.

In TypeDB 2.x, each database response carried its own replica list (DatabaseReplicas). In TypeDB 3.x, servers are the unit of replication, and all databases on that server share the replication context.

Replace per-database replica discovery with server-level discovery. On connection open, return ServerManager.All.Res (servers with replica status) instead of DatabaseManager.All.Res (databases with replicas).
Add servers_get RPC to retrieve the specific server's replica status.
Add server_version RPC to retrieve server distribution and version.
Remove DatabaseReplicas message. Instead, return Database (name only) instead of DatabaseReplicas (name + replica info). Replica topology is a server-level concern.
Replica information is now on Server.ReplicaStatus (replica ID, role, term).

Breaking: not compatible with previous versions.

Implementation

server.proto — Extended the Server message with:

optional ReplicaStatus replica_status (field 2) containing replica ID, role enum, and term
Nested Version request/response messages for the new server_version RPC
New ServerManager.Get request/response messages for the new servers_get RPC

database.proto — Simplified:

Removed DatabaseReplicas message and its nested Replica message
Added name field (field 1) to the Database message (previously it only contained nested schema/delete messages)
Updated DatabaseManager.Get/All/Create responses to return Database instead of DatabaseReplicas

connection.proto — Updated handshake:

Connection.Open.Res field 3 changed from DatabaseManager.All.Res databases_all to ServerManager.All.Res servers_all
Added import "proto/server.proto" dependency

typedb-service.proto → typedb_service.proto — Renamed to follow proto naming conventions (snake_case). Added servers_get and server_version RPC methods.

For the cleanup purposes within a major update, the fields are reorganized, so the users of this protocol will not be compatible with the older versions.

farost · 2026-04-15T12:14:03Z

      filter:
        owner: typedb
-        branch: [master]
+        branch: [master, cluster-support-feature-branch]


TODO: remove

TODO: Update protocol version since we break backwards compatibility

farost · 2026-04-15T12:14:43Z

    srcs = ["version.proto"],
 )

-# TODO: THIS SHOULD BE MADE TO STOP EXISTING


Only happy to help

farost · 2026-04-15T12:22:46Z

-      // pre-send all databases and replica info
-      DatabaseManager.All.Res databases_all = 3;
-
+      ServerManager.All.Res servers_all = 3;


databases are not actually used or useful. Since replication moved from the database to the server layer, it's only logical to send the list of available servers.

farost · 2026-04-15T12:23:18Z

+    rpc servers_get (ServerManager.Get.Req) returns (ServerManager.Get.Res);
+
+    // Server API
+    rpc server_version (Server.Version.Req) returns (Server.Version.Res);


Mirroring HTTP

farost · 2026-04-15T12:41:11Z

Renamed to follow proto naming conventions (snake_case). Can be rollbacked

farost · 2026-04-16T13:20:15Z

 proto_library(
    name = "typedb-service",
-    srcs = [":typedb-service.proto"],
+    srcs = ["typedb_service.proto"],


https://protobuf.dev/programming-guides/style/
While we break some stuff, I thought it might be a good moment to touch this

farost · 2026-04-16T13:20:45Z

@@ -1,5 +1,8 @@
 Documentation: https://typedb.com/docs/drivers/

+**This is an alpha release for CLUSTERED TypeDB 3.x. Do not use this as a stable version of TypeDB.**


TODO: Remove

farost · 2026-04-16T16:32:39Z


 message Server {
-    string address = 1;
+    optional string address = 1;


It contains ONLY the connection address of the grpc endpoint. No serving addresses, no http addresses. I think it's correct.

flyingsilverfin · 2026-04-19T00:19:43Z

-        "//proto:database-proto",
-        "//proto:migration-proto",
-        "//proto:error-proto",
+        "//proto:analyze-proto",


errr do we even still need to support java typedb-protocol

We don't use it. I just keep supporting it since it's there, maybe somebody depends on it -- it's very cheap for us (you do these global searches of deps anyway). Can be removed easily

nah just deleted it

flyingsilverfin · 2026-04-19T00:19:57Z

-        "//proto:database-proto",
-        "//proto:migration-proto",
-        "//proto:error-proto",
+        "//proto:analyze-proto",


and also here? Could delete IMO, less maintenance long term

I'd personally leave this since we still have the nodejs dir in typedb-driver, which is just not updated. If we decide "nah, we're not doing this in the coming years", then we just need to completely remove both.

i think we can delete it - http seems to work quite well for most people in the js environment IMO. We can always resurrect it if we need to, what do you think @lolski

Agreed @flyingsilverfin. Less maintenance that way!

## Release notes: usage and product changes Introduce cluster support for TypeDB 3.x, featuring server replicas and the new version message. ## Implementation Remove database replicas. Instead, add server replicas (by extending the usual `Server` message) with a similar set of fields. Instead of adding a `is_primary` flag, introduce an extensible enum `ReplicaType` for more granular split between primary and supporting nodes. To reduce the network overhead, this new information is provided in the initial server's connection response.

## Release notes: usage and product changes Add server_status message to the raft peering service and rename the old raft peering messages to adjust to the existing naming conventions. ## Implementation

Split the protocol into 3 services: * TypeDBService (contains common TypeDB logic): server, database, user, and transaction management, etc. * TypeDBClusteringService (contains TypeDB logic regarding clustering): replica registration, server status. * RaftService (contains Raft logic): replication operations. Split the old RaftPeeringService into TypeDBClusteringService and RaftService. Move separate services and their implementation files into separate subpackages. Rename methods. Introduce `ServersGet` method for single server status retrieval. Introduce `registration` method to verify the replica registration operation before submitting it to Raft.

Add protobuf schema for Raft's 'state mutation request'. The schema defines how the data is transported over the network and stored in the Raft log. - Add a protobuf definition for State Mutation Request - Create a mock RPC endpoint to handle that message as a hack to get the code generator to work

## Release notes: usage and product changes Reorganize Raft service's request messages to mirror the structure of TypeDB service's methods. Additionally, optionalize the Server message to allow returning pure Raft Agent id without other server information available for Clustered operations (in case the replica is not available). ## Implementation I just did it! --------- Co-authored-by: Ganeshwara Hananda <ganesh@vaticle.com> Co-authored-by: Ganeshwara Hananda <ganesh@typedb.com>

## Release notes: usage and product changes Introduce protocol changes required by typedb/typedb-cluster#642 : * User management requests now contain specific request information like username and password hashes (not actual passwords) instead of database commits. ## Implementation

… connection (#242) ## Release notes: usage and product changes Substitute `ServerManager.All.Res` with `ServerManager.Get.Res` in the result of the connection opening message. Returning all servers in a cloud environment does not make sense, since you cannot know the actual address of other servers until you connect to them (they are probably hidden behind proxies, and the GRPC protocol does not even know about them). So, it makes more sense to return only the target server's server information, which helps map the connection address with the id of the replica and all the other parameters for the future requests. ## Implementation

## Release notes: usage and product changes Remove unrelated packages and flatten the project's structure. ## Implementation Move raft and clustering packages to the repo that uses these protocol definitions instead of exposing them here: typedb/typedb-cluster#673

## Release notes: usage and product changes Remove replica registration/deregistration methods from the public protocol as a part of the move to the admin tool. These functions will be available only for the system administrators with access to the server's localhost. Additionally, remove the unnecessary address fields from `Server` messages. ## Implementation

farost force-pushed the cluster-support-feature-branch branch from a10b859 to 88d31ba Compare December 17, 2025 20:34

farost force-pushed the cluster-support-feature-branch branch 2 times, most recently from 8e5d232 to 5fbda34 Compare April 13, 2026 08:43

farost commented Apr 15, 2026

View reviewed changes

farost added type: feature priority: blocker labels Apr 16, 2026

farost changed the title ~~Add cluster support~~ Introduce clustered servers Apr 16, 2026

farost commented Apr 16, 2026

View reviewed changes

farost marked this pull request as ready for review April 16, 2026 13:21

farost requested a review from flyingsilverfin as a code owner April 16, 2026 13:21

farost requested a review from lolski April 16, 2026 15:58

farost commented Apr 16, 2026

View reviewed changes

flyingsilverfin reviewed Apr 19, 2026

View reviewed changes

flyingsilverfin approved these changes Apr 19, 2026

View reviewed changes

farost and others added 13 commits April 28, 2026 17:37

Add server register/deregister endpoint

471e590

Update the numbering of ReplicaStatus fields

64d5caf

Change type of term from in64 to uint64

0483964

Move raft peering protocol to this repo

9318480

Add server_status message to the raft peering service (#229)

91ad370

## Release notes: usage and product changes Add server_status message to the raft peering service and rename the old raft peering messages to adjust to the existing naming conventions. ## Implementation

Rename ReplicaType to ReplicaRole

7b06e2c

Return servers_all to the connection result

cd7fa15

farost added 9 commits April 28, 2026 17:38

Add connection address to the Server message

1585ec8

Update version to 3.0.7-alpha-0 with release notes

a4173e1

Update factory

a59cdf5

Fix field ordering

72dcc54

Bump protocol version

43615ea

Rename replica-related fields

4ef5af2

Update deps

8460b0a

farost force-pushed the cluster-support-feature-branch branch from e5eddc0 to 8460b0a Compare April 28, 2026 16:47

Remove java and nodejs builds

c069fac

		@@ -1,5 +1,8 @@
		Documentation: https://typedb.com/docs/drivers/

		This is an alpha release for CLUSTERED TypeDB 3.x. Do not use this as a stable version of TypeDB.

Conversation

farost commented Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Release notes: usage and product changes

Implementation

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

farost Apr 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

farost commented Nov 19, 2025 •

edited

Loading

farost Apr 16, 2026 •

edited

Loading