queue-runner: resolve dynamic derivation dependencies by Ericson2314 · Pull Request #1697 · NixOS/hydra

Ericson2314 · 2026-05-02T20:19:47Z

Reverse dependencies now carry an OutputNameChain (empty for regular deps, non-empty for dynamic ones), stored in a unified rdeps list via ReverseDep. This replaces the previous get_input_drvs method with input_drvs, which walks SingleDerivedPath directly, skipping Opaque (source) inputs and returning only Built derivation dependencies with their relation chain.

When a step completes and has dynamic reverse dependencies (rdeps with a non-empty OutputNameChain), pop one level from the relation, look up the produced output, and create a new step for the resulting derivation. This resolves dynamic derivation chains one level at a time as each step finishes, with no extra DB round-trip needed.

The dyn-drv tests are re-enabled and updated to verify the full build step chains: 4 steps for simple dynamic derivations, 12 for the non-trivial DAG case.

The main work is from @artemist

Depends on #1629

Replace the recursive SQL CTE in `resolve_drv_output_chains` (which cannot see the in-memory resolved-drv map) with a Rust loop that calls a new depth-1 `resolve_drv_output` query at each level, translating intermediate drv paths through `resolved_drv_map` between steps. This fixes resolution of dynamic derivation chains where an intermediate result is an unresolved drv (status=13): the recursive SQL required `status = 0` at every level and had no way to follow the in-memory mapping, so chains like `make-derivations.drv.drv^out^out` would fail when the intermediate `make-derivations.drv` had been resolved to a different path. Results are memoized in a temporary `HashMap` scoped to the `try_resolve` call, so repeated lookups of the same (drv, output_name) pair hit the cache. Once we have the resolved derivation mapping in the database, we can go back to the all-SQL approach, saving ourselves some app <-> DB round trips. This situation is explicitly tested.

Reverse dependencies now carry an `OutputNameChain` (empty for regular deps, non-empty for dynamic ones), stored in a unified `rdeps` list via `ReverseDep`. This replaces the previous `get_input_drvs` method with `input_drvs`, which walks `SingleDerivedPath` directly, skipping `Opaque` (source) inputs and returning only `Built` derivation dependencies with their relation chain. When a step completes and has dynamic reverse dependencies (`rdeps` with a non-empty `OutputNameChain`), pop one level from the relation, look up the produced output, and create a new step for the resulting derivation. This resolves dynamic derivation chains one level at a time as each step finishes, with no extra DB round-trip needed. The dyn-drv tests are re-enabled and updated to verify the full build step chains: 4 steps for simple dynamic derivations, 12 for the non-trivial DAG case. Co-Authored-By: Artemis Tosini <artemis.tosini@obsidian.systems> Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems> Co-Authored-By: Amaan Qureshi <git@amaanq.com>

Ericson2314 and others added 2 commits May 2, 2026 16:17

Ericson2314 enabled auto-merge May 2, 2026 20:21

Ericson2314 added this pull request to the merge queue May 2, 2026

github-merge-queue Bot removed this pull request from the merge queue due to failed status checks May 2, 2026