RecordMetadata fields holding raw SQL text are opaque to schema-evolution tooling

[hatyo]

The schema-template metadata persists several pieces of user-defined SQL as raw text strings: PRawSqlFunction for user-defined SQL functions and PView for views are on main today, and PStoredQuery is being added by PR #4157 for stored queries.

fdb-record-layer/fdb-record-layer-core/src/main/proto/record_metadata.proto

Lines 192 to 195 in 9026081

	message PRawSqlFunction {
	optional string name = 1;
	optional string definition = 2;
	}

fdb-record-layer/fdb-record-layer-core/src/main/proto/record_metadata.proto

Lines 223 to 226 in 9026081

	message PView {
	optional string name = 1;
	optional string definition = 2;
	}

All three have the identical { name, definition } shape where definition holds the original SQL text captured verbatim. The representation is fine for storage but it leaves the catalog blind to what each definition actually references. Anything that mutates schema metadata such as a record-type rename, a column rename, an index drop, a removed function, has no way to find affected definitions, much less rewrite them.

The canonical example is MetaDataProtoEditor#renameRecordTypes, which today walks record types, fields, and unnested-record usages and rewrites every place a renamed identifier appears, but does not touch the SQL text inside any definition. The method handles the two flavors of metadata differently. For any PUserDefinedFunction (and by extension, PRawSqlFunction), it bails on the entire rename with a MetaDataException:

fdb-record-layer/fdb-record-layer-core/src/main/java/com/apple/foundationdb/record/provider/foundationdb/MetaDataProtoEditor.java

Lines 553 to 556 in 9026081

	if (metaDataBuilder.getUserDefinedFunctionsCount() > 0) {
	// UserDefinedFunctions may be a string that needs parsing to figure out the types it references
	throw new MetaDataException("Renaming record types with UserDefinedFunctions is not supported");
	}

For PView there is no such guardrail at all; the rename proceeds and the definition string is silently left referencing the old name, with two downstream failure modes from there: either the definition parses but resolves to nothing at plan time and surfaces as a runtime error, or worse, during rename-and-replace migrations it now silently points at a different object that happens to share the new name. Both behaviors stem from the same root cause and the same gap will recur for column renames, dropped columns, dropped indexes referenced by hints, and dropped user-defined functions; every future metadata-holder that persists SQL text inherits this problem by construction.

The fix is structural rather than per-call. Most mature SQL systems materialize a first-class dependency graph in the catalog (PostgreSQL's pg_depend, SQL Server's sys.sql_expression_dependencies with the per-object SCHEMABINDING opt-in for binding strength, Oracle's DBA_DEPENDENCIES with invalidate-and-recompile semantics. The record-layer analogue would be to capture, alongside (or in place of) each raw definition text, the resolved set of identifiers the definition binds to: record types, columns, indexes, functions, other definitions, so that renames propagate automatically and drops are detectable at the catalog level.

There are multiple design choices here that we can work with, but the absence of any model is what makes the brittleness fundamentally unfixable inside MetaDataProtoEditor alone, there is no scalable path other than re-parsing SQL on every DDL (which is exactly the workaround the function-path throw is a placeholder for), and that is exactly what a dependency graph exists to avoid.

Once the catalog has a dependency graph, a number of capabilities become tractable that today are either impossible or require ad-hoc, potentially expensive, fleet-wide, reparsing:

• Mass rename: propagate identifier renames to all dependent functions, views, and stored queries without re-parsing SQL on every DDL.
• Cascade drop: DROP RECORD TYPE foo can refuse with a clear list of dependent definitions, or cascade-delete them on request.
• DDL-time validation: block a column drop or a type-narrowing change when a definition depends on it, before the change commits, with a precise error pointing at the offending dependent.
• Plan-cache invalidation: RelationalPlanCache (warmed by OfflineStoredQueriesProcessor) can correctly invalidate entries whose dependencies changed, instead of re-warming or blindly using stale plans.
• Impact analysis: "show me everything that would break if I change customer_id" reduces to a single graph traversal rather than a fleet-wide grep / re-parse.
• Orphan detection: identify definitions referencing identifiers that no longer exist after manual schema edits, surface them in tooling.
• Lineage and documentation: "which definitions read from orders?" becomes a first-class catalog query.
• Intelligent (re)planning: depending on the modeling design, we can assist the adopter answering fundamental questions related to the impact of schema and business logic query evolution of e.g. adding a new query predicate, will it cause the planner to choose a poor index? or will it continue to work as expected since the predicate is qualifying null fields and we know the field is null in the index chosen to plan the query before the addition of the new predicate.