Proposal: Gradual Enum Migration When type_name_overrides Collapses Input Enum Names

[ashkan25]

Proposal: Gradual Enum Migration When type_name_overrides Collapses Input Enum Names

Context

How Enum Migration Normally Works

ElasticGraph separates input and output enum types by default. For example, defining a Color enum produces both Color (used in output positions) and ColorInput (used in filter input positions):

# Output type
enum Color {
  RED
  GREEN
  BLUE
}

# Input type (used in filters)
enum ColorInput {
  RED
  GREEN
  BLUE
}

input ColorFilterInput {
  equalToAnyOf: [ColorInput!]
  # ...
}

This separation enables a safe, 3-stage migration workflow when adding a new enum value:

1. Add the new value to the ColorInput enum only (schema change, no client impact)
2. Clients adopt the new value at their own pace
3. Add the new value to the Color output enum

This works because clients' filter variables reference ColorInput, which can evolve independently from Color.

The Problem: type_name_overrides Collapses the Separation

Some schemas use type_name_overrides to collapse input enum names onto their output counterparts:

ElasticGraph::SchemaDefinition::RakeTasks.new(
  type_name_overrides: {"ColorInput" => "Color"}
)

With this override active, both input and output positions use Color. There is no separate ColorInput type:

enum Color {
  RED
  GREEN
  BLUE
}

input ColorFilterInput {
  equalToAnyOf: [Color!]  # Uses Color, not ColorInput
  # ...
}

This is a cleaner API for clients, but it breaks the 3-stage migration. Removing the override to restore the separate types is a breaking change — it simultaneously:

• Creates the ColorInput type
• Changes all filter field signatures from [Color!] to [ColorInput!]
• Breaks every client whose variables are typed as [Color!] (GraphQL type checking rejects the mismatch)

There is no transition period. The change is all-or-nothing.

Why This Is Hard to Work Around

The type_name_overrides system is deeply woven into three code paths, which together prevent both Color and ColorInput from coexisting while the override is active:

1. EnumType#as_input — calls to_final_form, which collapses "ColorInput" back to "Color", so as_input returns self (no input enum is created)
2. Field#type — calls to_final_form(as_input:), which applies the InputEnum format and then the override immediately collapses it back
3. EnumType#initialize — calls to_final_form on the type ref, so even constructing an enum named "ColorInput" registers it under "Color"

The net effect is that removing the override is a single atomic change that simultaneously affects every filter field across the schema — there's no way to migrate clients incrementally.

Options Considered

Option 1: Coordinated Big-Bang Migration

Remove the override and migrate all clients simultaneously.

This requires no framework changes, but it means coordinating every client of the affected schema to update their variable types at the exact same time. For schemas with many clients, this is impractical — a single missed client breaks on deploy.

Option 2: Gateway-Level Transforms

Use router-level request transforms (e.g., Apollo Router) to rewrite variable types at the gateway.

Router transforms operate on the serialized request, not the type system. GraphQL variable type checking happens before transforms run, so the type mismatch still causes validation failures. This doesn't actually solve the problem.

Option 3: Post-Process Generated SDL

Generate the schema normally, then inject the transition types as raw SDL after the fact.

The injected types wouldn't flow through ElasticGraph's schema definition pipeline — no runtime metadata, no filter interpreter registration, no shard routing optimization. An equalToAnyOfInput field injected this way would be unrecognized by FilterNodeInterpreter at runtime (classified as a :sub_field instead of an :operator), breaking queries.

Option 4: First-Class enums_in_transition Configuration (Proposed)

Add a configuration option that temporarily generates a transition input enum and a companion filter field, while keeping the existing override active. This is the approach I'm proposing below.

Proposal

I propose adding an enums_in_transition configuration option that allows schemas using type_name_overrides to gradually migrate their clients before removing the override.

Configuration

enums_in_transition accepts a list of enum type names (the output/collapsed names):

ElasticGraph::SchemaDefinition::RakeTasks.new(
  type_name_overrides: {"ColorInput" => "Color", "StatusInput" => "Status"},
  enums_in_transition: ["Color", "Status"]
)

Note

The names in enums_in_transition are validated against registered enum types after schema definition is complete. Misspelled names produce a warning with DidYouMean suggestions, consistent with how type_name_overrides and enum_value_overrides_by_type are validated.

Schema Changes

For each enum in the list, the generated schema gains two additions:

A transition input enum type:

# Existing (unchanged)
enum Color {
  RED
  GREEN
  BLUE
}

# New (generated during transition)
enum ColorInput {
  RED
  GREEN
  BLUE
}

The transition enum mirrors all values, documentation, and directives from the output enum.

An equalToAnyOfInput filter field:

input ColorFilterInput {
  equalToAnyOf: [Color!]           # Existing, unchanged
  equalToAnyOfInput: [ColorInput!] # New, added during transition
  # ...other filter fields unchanged...
}

The new field appears on both the standard filter type and the list element filter type.

Runtime Behavior

equalToAnyOfInput is processed identically to equalToAnyOf at query time:

• Filter interpretation: Both operators map to the same lambda in FilterNodeInterpreter — same terms query, same null handling, same ids query optimization for ID fields
• Shard routing: RoutingPicker extracts routing values from both operators
• Index pruning: IndexExpressionBuilder uses both operators for rollover index expression pruning
• Datastore values: Color.RED and ColorInput.RED both produce the string "RED" via to_datastore_value — the Elasticsearch queries are identical

Migration Workflow

Here's the full lifecycle for migrating away from a collapsed enum override:

Phase 1 — Add the enum to enums_in_transition. The schema now includes ColorInput and equalToAnyOfInput. No client impact — all existing queries continue to work.

Phase 2 — Clients migrate at their own pace from:

# Before
query($colors: [Color!]) {
  widgets(filter: { color: { equalToAnyOf: $colors } }) { ... }
}

to:

# After
query($colors: [ColorInput!]) {
  widgets(filter: { color: { equalToAnyOfInput: $colors } }) { ... }
}

Phase 3 — Once all clients have migrated, remove the type_name_overrides entry for Color. The equalToAnyOf field now naturally uses [ColorInput!]. Clients still using equalToAnyOfInput are unaffected.

Phase 4 (optional) — Clients migrate back from equalToAnyOfInput to equalToAnyOf, since both now accept [ColorInput!].

Phase 5 — Remove the enum from enums_in_transition. The transition equalToAnyOfInput field and the explicitly-generated ColorInput enum are removed from the schema. ColorInput continues to exist as EG's normal input enum.

Implementation Details

The implementation requires two internal bypass mechanisms, both defaulting to false (zero behavior change for all existing schemas):

skip_name_override on EnumType constructor

When true, the type ref is used as-is without calling to_final_form. This allows constructing a ColorInput enum that isn't collapsed to Color by the override. This flag is only used internally by the transition enum builder — it is not exposed to schema definition users.

# Simplified view of the change
def initialize(schema_def_state, name, skip_name_override: false, &block)
  type_ref = if skip_name_override
    schema_def_state.type_ref(name)
  else
    schema_def_state.type_ref(name).to_final_form
  end
  super(schema_def_state, type_ref, true, {})
  yield self if block_given?
end

type_already_final on Field

When true, the type and type_for_derived_types methods return the original type reference without calling to_final_form. This allows the equalToAnyOfInput filter field to reference [ColorInput!] exactly, without to_final_form rewriting it. Again, only used internally.

def type
  return original_type if type_already_final
  original_type.to_final_form(as_input: as_input)
end

New schema element: equalToAnyOfInput

Added to SchemaElementNames so it participates in the standard name generation pipeline (respects casing configuration, can be overridden via schema_element_name_overrides). The corresponding filter operator is registered in FilterNodeInterpreter, sharing the same lambda as equalToAnyOf.

Note

equalToAnyOfInput is intentionally a new, distinct schema element name rather than an alias for equalToAnyOf. This keeps the two fields independently addressable in the schema and avoids any ambiguity during the transition period.

What Doesn't Change

• Existing overrides stay active — the override is not removed or modified during transition
• Existing filter fields are untouched — equalToAnyOf: [Color!] continues to work exactly as before
• No Elasticsearch index or mapping changes — enum values are stored as plain keyword strings; "RED" is "RED" regardless of which enum type it came from
• No runtime metadata impact — user-defined enums don't produce entries in enum_types_by_name
• No breaking changes — enums_in_transition defaults to [], so all existing schemas are completely unaffected
• No impact on schemas that don't use type_name_overrides — the feature is only meaningful when overrides have collapsed input enum names

Proof of Concept

A draft implementation is available at #1093. It includes:

• All schema definition changes (State, API, RakeTasks, EnumType, Field, Factory, ScalarType)
• Runtime changes (FilterNodeInterpreter, RoutingPicker, IndexExpressionBuilder)
• RBS type signatures
• Validation with DidYouMean suggestions
• Unit tests for both camelCase and snake_case configurations

All existing tests pass with no regressions.

Consequences

• Schemas using type_name_overrides to collapse enum names will be able to migrate away from those overrides gradually, without breaking clients.
• The framework gains a small amount of new surface area: one new configuration option (enums_in_transition), one new schema element (equalToAnyOfInput), and two internal flags (skip_name_override, type_already_final). All are inert when not explicitly activated.
• The enums_in_transition configuration is designed to be temporary — once the override is removed and clients have migrated, it can be cleaned up. The feature is a bridge, not a permanent addition to the schema.

[jwondrusch]

@ashkan25 do you propose any config updates to allow enforcement of the input/return enum split? How do you envision the separation being maintained over time?

[ashkan25]

Once a subgraph finishes the migration and removes the type_name_overrides entry, the input/output enum separation is just EG’s default behavior, so there’s nothing extra to maintain going forward.

As for preventing new subgraphs from collapsing enum names in the first place, I think the simplest starting point would be documentation calling out this pattern and maybe exploring a lint/warning at schema definition time as a follow-up. Open to ideas though!

[BrianSigafoos-SQ]

@myronmarston this looks fine to me. Any concerns or comments?

[myronmarston]

Yes, I'm working on coming up with a response. Hope to have more to say later today.

[myronmarston]

Thanks for the well-articulated proposal, @ashkan25! I didn't really understand what problem you were trying to solve when I first saw #1093 but now I understand it well.

However, I have concerns about adding this to ElasticGraph itself, and I think there's a better path: implementing this entirely as an extension.

Why I'm Not in Favor of the Proposed Core Changes

This is a niche migration problem, not a general-purpose feature. It only affects projects that used type_name_overrides to collapse input/output enums — something that new EG projects using standard naming conventions will never need. (I suspect that Block is likely to be the only company that would need this feature).

The surface area is broad. The proposal touches SchemaElementNames, TypeNamer/TypeReference (with skip_name_override and type_already_final flags), EnumType, filter field generation, and configuration — spread across multiple gems. That's a significant ongoing maintenance burden for EG maintainers, especially for something that's inherently temporary (the config is removed once migration completes).

It can be implemented as an extension instead. EG's extension system is powerful enough to handle this without any core changes. The migration logic can live in a standalone gem (even an internal one), and when the migration is complete, the gem is simply removed, which leaves zero residual complexity in EG core.

How to Implement This as an Extension

The extension has two parts: (1) schema definition changes to generate the transitional types/fields, and (2) a query-time hook to make the new filter field work.

Part 1: Schema Definition Extension

This follows the same patterns as elasticgraph-apollo (which uses ApiExtension + FactoryExtension).

API Extension

Registers the factory and state extensions, and exposes a method on the API to register transitioning enums:

module EnumTransition
  module SchemaDefinition
    module APIExtension
      def self.extended(api)
        api.state.extend(StateExtension)
        api.factory.extend(FactoryExtension)
      end

      # Register enums as "in transition" — generates transitional input
      # enums and companion filter fields to enable gradual client migration.
      def transitioning_enums(*enum_names)
        enum_names.each { |name| state.transitioning_enums << name.to_s }
      end
    end
  end
end

Usage in config/schema.rb:

schema.extend EnumTransition::SchemaDefinition::APIExtension
schema.transitioning_enums "Color", "Size"

State Extension

Stores the set of transitioning enum names on the schema definition state:

module EnumTransition
  module SchemaDefinition
    module StateExtension
      def transitioning_enums
        @transitioning_enums ||= Set.new
      end
    end
  end
end

Factory Extension

Overrides new_enum_type to detect transitioning enums (by checking state) and extend them:

module EnumTransition
  module SchemaDefinition
    module FactoryExtension
      def new_enum_type(name, &block)
        super(name) do |enum_type|
          enum_type.extend(EnumTypeExtension) if @state.transitioning_enums.include?(name)
          block&.call(enum_type)
        end
      end
    end
  end
end

Enum Type Extension

Overrides derived_graphql_types to generate the transitional input enum and register the companion filter field. All setup is done lazily when derived_graphql_types is called — no separate setup step needed:

module EnumTransition
  module SchemaDefinition
    module EnumTypeExtension
      def derived_graphql_types
        transition_input_name = compute_transition_input_name
        register_transition_filter_field(transition_input_name)
        input_enum = build_transition_input_enum(transition_input_name)

        [input_enum] + super
      end

      private

      def compute_transition_input_name
        # The "natural" input name without the type_name_override that
        # collapsed input/output into one type.
        "#{name}Input" # Or derive via TypeNamer format if customized.
      end

      def register_transition_filter_field(transition_input_name)
        # Derive the filter type name for THIS (output) enum.
        # E.g., for "Color" with standard naming, the filter type is "ColorFilterInput".
        filter_type_name = type_ref
          .as_static_derived_type(:filter_input)
          .name

        # Add equalToAnyOfInput field to the existing filter type.
        # customize_derived_types just registers a block — it runs later
        # when Results applies customizations, so calling it here is fine.
        customize_derived_types(filter_type_name) do |filter_type|
          filter_type.field "equalToAnyOfInput", "[#{transition_input_name}!]" do |f|
            f.documentation "Transitional filter field. Behaves identically to `equalToAnyOf` " \
              "but accepts `#{transition_input_name}` values. Use this to migrate to the new " \
              "input enum type before the type_name_override is removed."
          end
        end
      end

      def build_transition_input_enum(transition_input_name)
        schema_def_state.factory.new_enum_type(transition_input_name) do |t|
          t.for_output = false
          t.graphql_only true
          t.documentation doc_comment
          directives.each { |dir| dir.duplicate_on(t) }
          values_by_name.each { |_, val| val.duplicate_on(t) }
        end
      end
    end
  end
end

Part 2: GraphQL Extension (Query-Time Rewriting)

The schema now has a equalToAnyOfInput field on filter types, but the query engine doesn't know how to process it (as FilterNodeInterpreter only recognizes operators from SchemaElementNames). The solution: rewrite the filter args before they reach the standard pipeline.

GraphQL Extension

Overrides filter_args_translator to extend the translator instance:

module EnumTransition
  module GraphQLExtension
    def filter_args_translator
      @filter_args_translator ||= super.tap do |translator|
        translator.extend(FilterArgsTranslatorExtension)
      end
    end
  end
end

This GraphQL extension can be configured via schema.register_graphql_extension (as shown in the elasticgraph-health_check README) or defined in YAML config as shown in the elasticgraph-query_interceptor README.

Filter Args Translator Extension

Rewrites equalToAnyOfInput → equalToAnyOf in the incoming filter hash before convert processes it:

module EnumTransition
  module FilterArgsTranslatorExtension
    FIELD_REWRITES = {
      "equalToAnyOfInput" => "equalToAnyOf"
      # Add more rewrites here if needed for other filter operators
    }.freeze

    def translate_filter_args(field:, args:)
      if (filter_hash = args[filter_arg_name])
        rewritten = deep_rewrite_filter_keys(filter_hash)
        super(field: field, args: args.merge(filter_arg_name => rewritten))
      else
        super
      end
    end

    private

    def deep_rewrite_filter_keys(obj)
      case obj
      when ::Hash
        obj.to_h do |key, value|
          rewritten_key = FIELD_REWRITES.fetch(key, key)
          [rewritten_key, deep_rewrite_filter_keys(value)]
        end
      when ::Array
        obj.map { |item| deep_rewrite_filter_keys(item) }
      else
        obj
      end
    end
  end
end

Why This Works End-to-End

1. Schema definition: The extension generates a ColorInput enum (identical values to Color) and adds equalToAnyOfInput: [ColorInput!] to ColorFilterInput.
2. Schema artifacts: Both enums appear in the SDL. Filter types have both equalToAnyOf: [Color!] and equalToAnyOfInput: [ColorInput!].
3. Client query: filter: { color: { equalToAnyOfInput: [RED, BLUE] } }
4. FilterArgsTranslator extension: Rewrites equalToAnyOfInput → equalToAnyOf in the filter hash.
5. Standard FilterArgsTranslator.convert: Sees equalToAnyOf, finds the field on ColorFilterInput, maps to name_in_index, resolves enum values — all standard processing.
6. FilterNodeInterpreter: Sees equalToAnyOf in its filter_operators hash, builds the Elasticsearch terms query.
7. Result: Identical behavior to using equalToAnyOf directly.

Migration Workflow

The migration workflow stays essentially the same as the proposal, but powered by an extension:

1. Add extension gem + call schema.transitioning_enums "Color" → Schema now has both Color and ColorInput enums, plus equalToAnyOfInput filter fields.
2. Clients gradually migrate from equalToAnyOf (typed as [Color!]) to equalToAnyOfInput (typed as [ColorInput!]).
3. Remove type_name_overrides entry for this enum → equalToAnyOf now naturally uses [ColorInput!].
4. Clients migrate back from equalToAnyOfInput to equalToAnyOf (both now use ColorInput).
5. Remove enum from transitioning_enums config (or remove the extension entirely) → clean state.

Note

Claude generated these code snippets for me and I haven't tried them so they probably don't quite work right but it should get you started.

If you run into any problems with this approach, we can work together to improve EG such that it enables an extension like this--but I doubt we need any changes to EG core for this to work.

@ashkan25 / @BrianSigafoos-SQ / @jwondrusch -- what do y'all think about this approach?

[ashkan25]

Thanks for the detailed extension approach, Myron! This makes a lot of sense to me and I agree it's a better fit as an extension rather than core changes.

customize_derived_types is a good fit for adding the equalToAnyOfInput filter field, and the filter args rewrite cleanly sidesteps any changes to FilterNodeInterpreter or SchemaElementNames.

I put this through Claude Code and it did flag one potential blocker: EnumType#initialize unconditionally calls schema_def_state.type_ref(name).to_final_form on the name, so calling factory.new_enum_type("ColorInput") when type_name_overrides: {"ColorInput" => "Color"} is active would produce a second Color enum rather than a ColorInput enum. The override fires inside the constructor before any extension code can intervene.

Do you have thoughts on how to handle that? A few options I can think of:

1. A small core change to EnumType#initialize (or Factory#new_enum_type) that allows bypassing the override, essentially the skip_name_override flag from the PR but scoped to just that one mechanism
2. Some way to temporarily suppress the override from the extension before calling new_enum_type and restoring it after
3. A different approach I'm not seeing?

Happy to go whichever direction makes the most sense for the project.

[myronmarston]

I think your extension can deal with this. Something like:

module EnumTransition
  module SchemaDefinition
    module FactoryExtension
      def new_enum_type(name, &block)
        super(name) do |enum_type|
          if @state.transitioning_enums.include?(name)
            enum_type.type_ref = .... # fix the type ref here.
            enum_type.extend(EnumTypeExtension)
          end
          block&.call(enum_type)
        end
      end
    end
  end
end

Ruby structs are mutable and since EnumType is created from a struct, it's attributes can be updated. We need to be careful about mutating them (some surprising things could happen if you randomly mutated attributes at some later point) but I think it's safe to do it here--if you update type_ref in the block then it happens during initialization before any computation has happened based on the original type_ref. Just make sure you do it before the block&.call(enum_type) line (which allows arbitrary computation based on the EnumType)--once that happens you should not mutate type_ref again.

[ashkan25]

I'll give that a shot. Thanks for the help!

[myronmarston]

Great. I'm gong to close this, but feel free to re-open if you have followup questions/concerns.