|
| 1 | +# ElasticGraph::Protobuf |
| 2 | + |
| 3 | +An ElasticGraph extension that generates Protocol Buffers schema artifacts from ElasticGraph schemas. |
| 4 | +It emits `proto3` by default and can emit `proto2`, and supports arbitrary file-level headers (such |
| 5 | +as `option` declarations). |
| 6 | + |
| 7 | +## Dependency Diagram |
| 8 | + |
| 9 | +```mermaid |
| 10 | +graph LR; |
| 11 | + classDef targetGemStyle fill:#FADBD8,stroke:#EC7063,color:#000,stroke-width:2px; |
| 12 | + classDef otherEgGemStyle fill:#A9DFBF,stroke:#2ECC71,color:#000; |
| 13 | + classDef externalGemStyle fill:#E0EFFF,stroke:#70A1D7,color:#2980B9; |
| 14 | + elasticgraph-protobuf["elasticgraph-protobuf"]; |
| 15 | + class elasticgraph-protobuf targetGemStyle; |
| 16 | + elasticgraph-support["elasticgraph-support"]; |
| 17 | + elasticgraph-protobuf --> elasticgraph-support; |
| 18 | + class elasticgraph-support otherEgGemStyle; |
| 19 | +``` |
| 20 | + |
| 21 | +## Usage |
| 22 | + |
| 23 | +First, add `elasticgraph-protobuf` to your `Gemfile`, alongside the other ElasticGraph gems: |
| 24 | + |
| 25 | +```diff |
| 26 | +diff --git a/Gemfile b/Gemfile |
| 27 | +index 4a5ef1e..5c16c2b 100644 |
| 28 | +--- a/Gemfile |
| 29 | ++++ b/Gemfile |
| 30 | +@@ -8,6 +8,7 @@ gem "elasticgraph-query_registry", *elasticgraph_details |
| 31 | + |
| 32 | + # Can be elasticgraph-elasticsearch or elasticgraph-opensearch based on the datastore you want to use. |
| 33 | + gem "elasticgraph-opensearch", *elasticgraph_details |
| 34 | ++gem "elasticgraph-protobuf", *elasticgraph_details |
| 35 | + |
| 36 | + gem "httpx", "~> 1.3" |
| 37 | + |
| 38 | +``` |
| 39 | + |
| 40 | +Next, update your `Rakefile` so that `ElasticGraph::Protobuf::SchemaDefinition::APIExtension` is |
| 41 | +included in the schema-definition extension modules: |
| 42 | + |
| 43 | +```diff |
| 44 | +diff --git a/Rakefile b/Rakefile |
| 45 | +index 2943335..26633c3 100644 |
| 46 | +--- a/Rakefile |
| 47 | ++++ b/Rakefile |
| 48 | +@@ -3,5 +3,6 @@ |
| 49 | + require "elastic_graph/json_ingestion/schema_definition/api_extension" |
| 50 | + require "elastic_graph/local/rake_tasks" |
| 51 | ++require "elastic_graph/protobuf/schema_definition/api_extension" |
| 52 | + require "elastic_graph/query_registry/rake_tasks" |
| 53 | + require "rspec/core/rake_task" |
| 54 | + require "standard/rake" |
| 55 | +@@ -16,6 +17,7 @@ ElasticGraph::Local::RakeTasks.new( |
| 56 | + # Determines casing of field names. Can be either `:camelCase` or `:snake_case`. |
| 57 | + tasks.schema_element_name_form = :camelCase |
| 58 | + tasks.schema_definition_extension_modules << ElasticGraph::JSONIngestion::SchemaDefinition::APIExtension |
| 59 | ++ tasks.schema_definition_extension_modules << ElasticGraph::Protobuf::SchemaDefinition::APIExtension |
| 60 | + |
| 61 | + # Customizes the names of fields generated by ElasticGraph. |
| 62 | + tasks.schema_element_name_overrides = { |
| 63 | +``` |
| 64 | + |
| 65 | +Then opt into proto generation from your schema definition: |
| 66 | + |
| 67 | +```ruby |
| 68 | +# in config/schema/protobuf.rb |
| 69 | + |
| 70 | +ElasticGraph.define_schema do |schema| |
| 71 | + schema.proto_schema_artifacts package_name: "myapp.events.v1" |
| 72 | +end |
| 73 | +``` |
| 74 | + |
| 75 | +After running `bundle exec rake schema_artifacts:dump`, ElasticGraph will generate: |
| 76 | + |
| 77 | +- `schema.proto` |
| 78 | +- `proto_field_numbers.yaml` |
| 79 | + |
| 80 | +## Schema Definition Options |
| 81 | + |
| 82 | +### Protobuf Syntax (`proto2` / `proto3`) |
| 83 | + |
| 84 | +`proto_schema_artifacts` emits `proto3` by default. Pass `syntax: :proto2` to emit a proto2 file |
| 85 | +instead (every field is then labeled `optional` or `repeated`). This is useful when the generated |
| 86 | +messages need to reference proto2 types — for example, `protoc` forbids a `proto3` message from |
| 87 | +referencing a `proto2` enum: |
| 88 | + |
| 89 | +```ruby |
| 90 | +ElasticGraph.define_schema do |schema| |
| 91 | + schema.proto_schema_artifacts package_name: "myapp.events.v1", syntax: :proto2 |
| 92 | +end |
| 93 | +``` |
| 94 | + |
| 95 | +### Custom Headers |
| 96 | + |
| 97 | +Pass `headers:` an array of strings to inject file-level lines (such as `option` declarations) |
| 98 | +verbatim, as a contiguous section immediately after the `package` declaration. This lets you set |
| 99 | +language-specific options without the gem baking in any particular convention: |
| 100 | + |
| 101 | +```ruby |
| 102 | +ElasticGraph.define_schema do |schema| |
| 103 | + schema.proto_schema_artifacts( |
| 104 | + package_name: "myapp.events.v1", |
| 105 | + headers: [ |
| 106 | + %(option java_package = "com.myapp.events";), |
| 107 | + "option java_multiple_files = true;" |
| 108 | + ] |
| 109 | + ) |
| 110 | +end |
| 111 | +``` |
| 112 | + |
| 113 | +produces: |
| 114 | + |
| 115 | +```proto |
| 116 | +syntax = "proto3"; |
| 117 | +
|
| 118 | +package myapp.events.v1; |
| 119 | +
|
| 120 | +option java_package = "com.myapp.events"; |
| 121 | +option java_multiple_files = true; |
| 122 | +
|
| 123 | +// ...messages... |
| 124 | +``` |
| 125 | + |
| 126 | +### Custom Scalar Types |
| 127 | + |
| 128 | +Built-in ElasticGraph scalar types are automatically mapped to proto scalar types. |
| 129 | +For custom scalar types, use `proto_field` to define the proto scalar type: |
| 130 | + |
| 131 | +```ruby |
| 132 | +# in config/schema/money.rb |
| 133 | + |
| 134 | +ElasticGraph.define_schema do |schema| |
| 135 | + schema.scalar_type "Money" do |t| |
| 136 | + t.mapping type: "long" |
| 137 | + t.json_schema type: "integer" |
| 138 | + t.proto_field type: "int64" |
| 139 | + end |
| 140 | +end |
| 141 | +``` |
| 142 | + |
| 143 | +### Sourcing Enum Values From Existing Protobuf Mappings |
| 144 | + |
| 145 | +If your project already maintains GraphQL-to-proto enum mappings (for example in tests), |
| 146 | +you can reuse them for proto schema generation: |
| 147 | + |
| 148 | +```ruby |
| 149 | +# in config/schema/proto_enum_mappings.rb |
| 150 | + |
| 151 | +ElasticGraph.define_schema do |schema| |
| 152 | + schema.proto_enum_mappings( |
| 153 | + SalesEg::ProtoEnumMappings::PROTO_ENUMS_BY_GRAPHQL_ENUM |
| 154 | + ) if defined?(SalesEg::ProtoEnumMappings) |
| 155 | +end |
| 156 | +``` |
| 157 | + |
| 158 | +When a mapping exists for an enum, `elasticgraph-protobuf` uses the mapped proto enum(s) |
| 159 | +as the source of enum values (respecting `exclusions`, `expected_extras`, and `name_transform`). |
| 160 | + |
| 161 | +### Referencing Existing Protobuf Types |
| 162 | + |
| 163 | +For enums that exactly match a canonical proto enum, you can import and reference |
| 164 | +the existing proto type instead of generating a duplicate local enum: |
| 165 | + |
| 166 | +```ruby |
| 167 | +ElasticGraph.define_schema do |schema| |
| 168 | + schema.proto_enum_mappings( |
| 169 | + "CardType" => { |
| 170 | + Squareup::Connect::V2::Resources::Card::Type => {} |
| 171 | + } |
| 172 | + ) |
| 173 | + |
| 174 | + schema.proto_external_types( |
| 175 | + "CardType" => { |
| 176 | + proto: "squareup.connect.v2.resources.Card.Type", |
| 177 | + import: "squareup/connect/v2/resources/card.proto" |
| 178 | + } |
| 179 | + ) |
| 180 | +end |
| 181 | +``` |
| 182 | + |
| 183 | +External type references currently support enums only. The matching |
| 184 | +`proto_enum_mappings` entry must have exactly one source and no transform options; |
| 185 | +otherwise the enum stays generated locally so value curation remains explicit. |
| 186 | + |
| 187 | +### Stable Field Numbers |
| 188 | + |
| 189 | +`schema_artifacts:dump` automatically reads and writes `proto_field_numbers.yaml` |
| 190 | +in the schema artifacts directory. Existing numbers stay fixed even if field order |
| 191 | +changes, and new fields get the next available numbers. |
| 192 | + |
| 193 | +`schema.proto` always uses the public GraphQL field names. When a field uses a |
| 194 | +different `name_in_index`, the sidecar YAML stores that override privately: |
| 195 | + |
| 196 | +```yaml |
| 197 | +messages: |
| 198 | + Widget: |
| 199 | + fields: |
| 200 | + id: 1 |
| 201 | + display_name: |
| 202 | + field_number: 2 |
| 203 | + name_in_index: displayName |
| 204 | +``` |
| 205 | +
|
| 206 | +If a field is renamed with `field.renamed_from`, `elasticgraph-protobuf` reuses the |
| 207 | +existing field number under the new public field name. |
| 208 | + |
| 209 | +## Type Mappings |
| 210 | + |
| 211 | +The generated `schema.proto` uses these built-in scalar mappings: |
| 212 | + |
| 213 | +| ElasticGraph Type | Protobuf Type | |
| 214 | +|-------------------|------------| |
| 215 | +| `Boolean` | `bool` | |
| 216 | +| `Cursor` | `string` | |
| 217 | +| `Date` | `string` | |
| 218 | +| `DateTime` | `string` | |
| 219 | +| `Float` | `double` | |
| 220 | +| `ID` | `string` | |
| 221 | +| `Int` | `int32` | |
| 222 | +| `JsonSafeLong` | `int64` | |
| 223 | +| `LocalTime` | `string` | |
| 224 | +| `LongString` | `int64` | |
| 225 | +| `String` | `string` | |
| 226 | +| `TimeZone` | `string` | |
| 227 | +| `Untyped` | `string` | |
| 228 | + |
| 229 | +Additionally: |
| 230 | +- List types become `repeated` fields. |
| 231 | +- Nested list types generate wrapper messages so the output remains valid `proto3`. |
| 232 | +- Enum types generate `enum` definitions whose values are prefixed with the enum type name in `UPPER_SNAKE_CASE`, including a zero-valued `*_UNSPECIFIED` entry. |
0 commit comments