compute: sync controller config before create-time setup

[antiguru]

Motivation

A replica creates its ComputeState when it handles CreateInstance, seeding worker_config with dyncfg defaults. handle_create_instance then immediately runs apply_worker_config, but the controller's synced configuration only arrives one command later, in the first UpdateConfiguration. Both the live command stream (Instance::run sends Hello then CreateInstance before the first update_configuration) and the reconciliation order (ComputeCommandHistory::reduce places the single UpdateConfiguration after CreateInstance) put configuration after creation. Reordering is not viable either: there is no ComputeState/worker_config to apply configuration to before CreateInstance.

As a result, every create-time read that depends on configuration silently used defaults: lgalloc, the pager, the column-paged batcher, the memory limiter, ore_overflowing_behavior, and the introspection dataflows rendered during CreateInstance. This is also why create-time-frozen scoped flags from #37158 had to be worked around.

Description

Carry the configuration with CreateInstance. InstanceConfig gains an initial_config snapshot of the controller's dyncfg with the replica's scoped overrides applied on top. The controller builds it in Instance::specialize_command_for_replica, the same layer that already injects scoped overrides into UpdateConfiguration, and it has the live dyncfg and override map there. It is evaluated fresh on every send and on every add_replica replay, so the snapshot always reflects current values. The replica applies it to worker_config before apply_worker_config.

The snapshot is excluded from InstanceConfig::compatible_with, like dictionary compression, so reconnecting to a running replica does not halt on configuration that has changed since creation. An empty snapshot leaves the worker at its defaults until the first UpdateConfiguration, preserving prior behavior.

Soundness:

• Fresh startup or replica restart: reconcile applies CreateInstance, seeding worker_config from the snapshot before create-time setup.
• environmentd reconnect to a running replica: reconcile only compatibility-checks CreateInstance; the existing already-synced worker_config is retained and the replayed UpdateConfiguration keeps it current.
• UpdateConfiguration still applies globally and remains hoistable; the snapshot is a redundant create-time seed of the same values, not a new ordering dependency.

Tests

Adds unit tests in the compute controller covering the create-time snapshot and the scoped-override merge, and confirming the existing UpdateConfiguration override merge is unchanged. Adds a clusterd-test-driver scenario and a parse test exercising the create-time snapshot plumbing end to end.

Checklist

• This PR has adequate test coverage / or no new functionality requires testing.

[antiguru]

Reviewed the change end to end. The fix is correct and well-scoped, with a sound soundness argument. One non-blocking question below.

Correctness

Traced the three paths the soundness argument depends on:

1. Fresh startup / replica restart (server.rs, compute_state is None): todo_commands = new_commands, so handle_create_instance runs and the new initial_config.apply(&worker_config) seeds the worker before apply_worker_config. The following UpdateConfiguration re-applies idempotently. ✅
2. environmentd reconnect to a running replica (server.rs:644): CreateInstance is only compatibility-checked, never pushed to todo_commands, so handle_create_instance does not re-run and the already-synced worker_config is retained. ✅
3. compatible_with exclusion: initial_config is destructured to _ and excluded, mirroring arrangement_dictionary_compression. Without this a reconnect after any config change would halt!. ✅

Other checks:

• Both the controller dyncfg and the worker worker_config are built from mz_dyncfgs::all_dyncfgs(), so the dense snapshot from From<&ConfigSet> only contains configs the worker knows. No spurious Sentry errors from the unknown-config path in ConfigUpdates::apply.
• send stores the base (un-specialized) command in history and specializes per-replica at dispatch; add_replica re-specializes from history. The snapshot always reflects current dyncfg + override, never a stale baked value. The override-wins merge matches the existing UpdateConfiguration semantics.
• The controller's dyncfg is kept current via update_configuration, so by the time CreateInstance is sent the snapshot carries synced values rather than defaults.
• InstanceConfig uses serde (no protobuf) and ConfigUpdates already derives Serialize/Deserialize, so the new field is wire-compatible with no proto changes. All call sites updated.

Tests

Good coverage: three controller unit tests (instance-wide snapshot, override-wins, unchanged UpdateConfiguration merge), a parse test, and an end-to-end clusterd-test-driver scenario.

Question (non-blocking)

Persist config / metrics are not seeded at create time. handle_create_instance applies the snapshot only to worker_config. handle_update_configuration additionally does persist_clients.cfg().apply_from(...) and updates metrics, neither of which rides in CreateInstance (the controller.rs comment acknowledges this). Can you confirm that no create-time setup reads a persist-side dyncfg? Anything fetched from persist_clients.cfg() while handling CreateInstance would still observe defaults. If create-time reads only touch worker_config, this is fine as-is, but it's the one remaining gap in an otherwise complete fix.

---

Generated by Claude Code

[antiguru]

Re: persist config / metrics at create time.

Verified that no create-time read touches persist_clients.cfg(), so seeding only worker_config is sufficient.

handle_create_instance does:

• apply_worker_config — reads worker_config, context.scratch_directory, and memory_limiter::get_memory_limit. No persist config.
• stores DICTIONARY_COMPRESSION from the InstanceConfig field, apply_expiration_offset (time only), and peek_stash_persist_location.
• initialize_logging — passes worker_config to logging::initialize, which builds in-memory logging arrangements with no persist source or sink.

Every persist_clients.cfg() read in the compute crate is outside the create path: dataflow_max_inflight_bytes (CreateDataflow time, and it reads the is_cc_active field fixed at client construction, not the UpdateConfiguration dyncfg path), handle_update_configuration, and handle_allow_writes. Persist sources and sinks render at CreateDataflow, which is after the first UpdateConfiguration runs persist_clients.cfg().apply_from.

So persist-side dyncfg is synced before anything reads it. Leaving create-time seeding scoped to worker_config; if a create-time persist read is ever introduced, seeding the persist config there is a one-liner mirroring handle_update_configuration.

[DAlperin]

LGTM, thanks