Skip to content

Add @bufbuild/protobuf (protobuf-es) payload converters #2045

Description

@wcygan

Is your feature request related to a problem? Please describe.

The SDK ships payload converters for protobufjs only (@temporalio/common/lib/protobufs). Users who generate their proto types with @bufbuild/protobuf v2 can't pass those types to a Temporal client or workflow without writing their own payload converter. The two libraries produce incompatible message shapes (Schema + create(Schema, init) vs Message.create(init)), so the existing converters can't be reused.

Describe the solution you'd like

A parallel converter trio under @temporalio/common/lib/protobufs-es, mirroring the existing …/protobufs exports:

  • ProtobufEsBinaryPayloadConverter
  • ProtobufEsJsonPayloadConverter
  • DefaultPayloadConverterWithProtobufsEs

Today — every protobuf-es user writes their own (~150 LOC) converter:

// payload-converter.ts
import { create, toBinary, fromBinary, toJson, fromJson, createRegistry } from '@bufbuild/protobuf';
// ...custom Binary + JSON converters, composite ordering,
//    messageType metadata, Registry forwarding for google.protobuf.Any...

Proposed — first-class support, matching the protobufjs ergonomics:

// payload-converter.ts
import { createRegistry } from '@bufbuild/protobuf';
import { DefaultPayloadConverterWithProtobufsEs } from '@temporalio/common/lib/protobufs-es';
import { MyMessageSchema } from './gen/messages_pb';

export const payloadConverter = new DefaultPayloadConverterWithProtobufsEs({
  registry: createRegistry(MyMessageSchema),
});

Sketch of the desired properties:

  • @bufbuild/protobuf as an optional peer dependency so users on the protobufjs path pay nothing.
  • Wire-compatible with the existing protobufjs converters:
    • A payload produced by either converter (binary or JSON) round-trips correctly through the other.
    • Same encoding markers (binary/protobuf, json/protobuf) and fully-qualified messageType metadata.
    • Composite ordering matches DefaultPayloadConverterWithProtobufs (Undefined → Binary → ProtobufJson → ProtobufBinary → Json), so on-the-wire converter selection is identical.
  • Constructor accepts either a pre-built Registry or a schema array (calls createRegistry internally).

Additional context

The existing docs/protobuf-libraries.md is the SDK's original (Dec '21) decision log for protobuf library choice. It defines three criteria — (A) TS types, (B) runtime-checkable message instances, (C) spec-compliant proto3 JSON — and notes that protobufjs scores A+B but not C, which is why the SDK bolts on proto3-json-serializer and patchProtobufRoot. @bufbuild/protobuf v2 satisfies A+B+C natively, and its generated output addresses every item in that doc's "Future work" section (spec-compliant to/fromJSON, fully-qualified typename, "this is a generated file" header).

Concrete use case driving this

I'm building protoc-gen-ts-temporal and protoc-gen-rust-temporal (TypeScript and Rust analogues of protoc-gen-go-temporal) so that a single .proto file generates type-safe Temporal clients, workers, and activity stubs across languages.

I'm trying to build a Node service that accepts a webhook and starts a workflow whose worker is implemented in Rust, with no intermediate Rust HTTP server that exists only to translate the webhook into a workflow start. For that to work, the TS client and the Rust worker have to agree on the wire format of the workflow args. protoc-gen-ts-temporal is built on @bufbuild/protoc-gen-es, so its generated message types are @bufbuild/protobuf v2.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions