[Feature Request]: Support PROTO protobuf in WriteToPubSub Beam YAML

[jonathaningram]

What would you like to happen?

In WriteToPubSubLite it supports PROTO:

format string : The encoding format for the data stored in Pubsub Lite. Valid options are: RAW,JSON,AVRO,PROTO

But in WriteToPubSub it does not:

format string : How to format the message payload. Currently suported formats are

RAW: Expects a message with a single field (excluding attribute-related fields) whose contents are used as the raw bytes of the pubsub message.
AVRO: Encodes records with a given Avro schema, which may be inferred from the input PCollection schema.
JSON: Formats records with a given JSON schema, which may be inferred from the input PCollection schema.

Is it possible to add support for PROTO when writing to Pub/Sub?

Is there an existing workaround?

Issue Priority

Priority: 2 (default / most feature requests should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[liferoad]

https://github.com/apache/beam/blob/master/sdks/python/apache_beam/yaml/yaml_io.py#L254 needs to support proto.

[jonathaningram]

We're sending a lot of JSON bytes to Pub/Sub, so the reading and writing of that is likely increasing the overall bytes written/read compared to if we could use Protobuf.

What is the likelihood of implementing this feature any time soon?