|
19 | 19 | # configuration in /sdks/standard_expansion_services.yaml. |
20 | 20 | # Refer to gen_xlang_wrappers.py for more info. |
21 | 21 | # |
22 | | -# Last updated on: 2026-05-06 |
| 22 | +# Last updated on: 2026-06-10 |
23 | 23 |
|
24 | 24 | - default_service: sdks:java:io:expansion-service:shadowJar |
25 | 25 | description: 'Outputs a PCollection of Beam Rows, each containing a single INT64 |
|
80 | 80 | type: str |
81 | 81 | identifier: beam:schematransform:org.apache.beam:mongodb_write:v1 |
82 | 82 | name: MongodbWrite |
| 83 | +- default_service: sdks:java:io:expansion-service:shadowJar |
| 84 | + description: 'Reads messages from an MQTT broker and outputs each payload as a single |
| 85 | + `bytes` field. |
| 86 | +
|
| 87 | +
|
| 88 | + By default the read is unbounded (streaming): it keeps consuming messages from |
| 89 | + the subscribed topic until the pipeline is stopped. Setting `maxNumRecords` and/or |
| 90 | + `maxReadTimeSeconds` bounds the read, producing a bounded (batch) PCollection. |
| 91 | +
|
| 92 | +
|
| 93 | + Note: streaming reads require a runner that supports portable streaming (e.g. |
| 94 | + Prism, Flink, or Dataflow). The legacy local Python DirectRunner does not execute |
| 95 | + portable streaming cross-language reads.' |
| 96 | + destinations: |
| 97 | + python: apache_beam/io |
| 98 | + fields: |
| 99 | + - description: Configuration options to set up the MQTT connection. |
| 100 | + name: connection_configuration |
| 101 | + nullable: false |
| 102 | + type: Row(client_id=typing.Optional[str], password=typing.Optional[str], server_uri=<class |
| 103 | + 'str'>, topic=typing.Optional[str], username=typing.Optional[str]) |
| 104 | + - description: The max number of records to receive. Setting this will result in |
| 105 | + a bounded PCollection. |
| 106 | + name: max_num_records |
| 107 | + nullable: true |
| 108 | + type: int64 |
| 109 | + - description: The maximum time for this source to read messages. Setting this will |
| 110 | + result in a bounded PCollection. |
| 111 | + name: max_read_time_seconds |
| 112 | + nullable: true |
| 113 | + type: int64 |
| 114 | + identifier: beam:schematransform:org.apache.beam:mqtt_read:v1 |
| 115 | + name: ReadFromMqtt |
| 116 | +- default_service: sdks:java:io:expansion-service:shadowJar |
| 117 | + description: 'Publishes messages to an MQTT broker. Expects an input PCollection |
| 118 | + of rows with a single `bytes` field, each of which is published as one MQTT message. |
| 119 | +
|
| 120 | +
|
| 121 | + Works with both bounded (batch) and unbounded (streaming) input PCollections.' |
| 122 | + destinations: |
| 123 | + python: apache_beam/io |
| 124 | + fields: |
| 125 | + - description: Configuration options to set up the MQTT connection. |
| 126 | + name: connection_configuration |
| 127 | + nullable: false |
| 128 | + type: Row(client_id=typing.Optional[str], password=typing.Optional[str], server_uri=<class |
| 129 | + 'str'>, topic=typing.Optional[str], username=typing.Optional[str]) |
| 130 | + - description: Whether or not the publish message should be retained by the messaging |
| 131 | + engine. When a subscriber connects, it gets the latest retained message. Defaults |
| 132 | + to `False`, which will clear the retained message from the server. |
| 133 | + name: retained |
| 134 | + nullable: true |
| 135 | + type: boolean |
| 136 | + identifier: beam:schematransform:org.apache.beam:mqtt_write:v1 |
| 137 | + name: WriteToMqtt |
83 | 138 | - default_service: sdks:java:io:expansion-service:shadowJar |
84 | 139 | description: '' |
85 | 140 | destinations: |
|
0 commit comments