diff --git a/proto/substrait/algebra.proto b/proto/substrait/algebra.proto index 6cea2c849..55b71bd1c 100644 --- a/proto/substrait/algebra.proto +++ b/proto/substrait/algebra.proto @@ -447,20 +447,23 @@ message SetRel { } } -// Stub to support extension with a single input +// A custom relational operator with a single input. Producers and consumers +// must agree on how to derive the output schema of the relation. message ExtensionSingleRel { RelCommon common = 1; Rel input = 2; google.protobuf.Any detail = 3; } -// Stub to support extension with a zero inputs +// A custom relational operator with zero inputs. Producers and consumers +// must agree on how to derive the output schema of the relation. message ExtensionLeafRel { RelCommon common = 1; google.protobuf.Any detail = 2; } -// Stub to support extension with multiple inputs +// A custom relational operator with multiple inputs. Producers and consumers +// must agree on how to derive the output schema of the relation. message ExtensionMultiRel { RelCommon common = 1; repeated Rel inputs = 2; diff --git a/site/docs/extensions/index.md b/site/docs/extensions/index.md index 51c69531c..d982c12d3 100644 --- a/site/docs/extensions/index.md +++ b/site/docs/extensions/index.md @@ -263,6 +263,10 @@ The third form of advanced extensions provides entirely new relational operation These extension relations are first-class relation types in Substrait and can be used anywhere a standard relation would be used. +#### Output Schema + +Unlike standard relations, Substrait cannot specify how to derive the output schema of extension relations. It is up to producers and consumers to agree on how to derive the schema from the inputs and the `detail` field. This derivation should be documented in the protobuf definition (or equivalent) used for the `detail` field. + ### When to Use What Custom relations are the most flexible option, but also the least interoperable. Prefer enhancements to existing relations when they can express your use case, since this preserves existing patterns and compatibility. As a general rule, choose the least powerful extension mechanism that solves the problem.