Fix cargo resolver pruning of optional/feature-gated dependencies

[evandowning]

Fixes #189.

Problem

The cargo resolver marked every declared dependency of a package as already-resolved. But cargo metadata does not add optional/feature-gated dependencies to its resolved package set when the active feature set doesn't enable them. Those deps therefore matched no package in the cache, and resolution.py pruned them and their entire subtrees.

Net effect: for cargo projects, it-depends' core "resolve all possible dependency versions" behavior collapsed to "the single locked dependency tree" (e.g. rayon, backtrace, afl and ~30 other crates reachable only via optional edges were dropped for the siderophile fixture).

Changes

1. resolve_from_source — collect all packages first, then only set_resolved(dep) when a matching package was actually added to the cache. Inactive optional deps fall through to normal resolution instead of being marked resolved-to-nothing and pruned. This also unifies the previously-separate handling of the SourcePackage's direct deps and transitive deps.

2. get_dependencies — wrap the cargo metadata subprocess call in except subprocess.CalledProcessError, mirroring pip.py's johnnydep handling (Add exception handling #125). Previously a single crate whose temp-project cargo metadata failed (exit 101) raised CalledProcessError and aborted the entire resolution; the old pruning shortcut had inadvertently masked this by never resolving those crates.

Tests

Added test_resolve_from_source_only_marks_resolved_deps, which mocks get_dependencies (no cargo binary required) and verifies that a dep with a matching resolved package is marked resolved while optional deps with no matching package are left unresolved.

Open question (draft)

Fix (1) restores full "all possible versions" expansion, which is large — the issue reports ~963 crates for siderophile vs. 163 in the 2021 baseline and 129 on current master. Per the issue, whether the intended cargo semantics are the full possible set, the locked tree, or a middle ground (e.g. union of all features at locked versions) should be decided deliberately. Opening as a draft so that semantics decision can be settled before merge.

Note: cargo is not installed in this environment, so the end-to-end siderophile reproduction and the re-enabled smoke test (#181) were not run here — CI should exercise those.

🤖 Generated with Claude Code

[evandowning]

Ran the siderophile reproduction + re-enabled smoke test

The PR note said cargo wasn't installed here so the end-to-end siderophile reproduction and the re-enabled smoke test (#181) weren't run. I ran them with the rust toolchain (cargo 1.96.1) installed, applying this PR plus #181's re-enabled smoke test on top of master.

Reproduction — confirms the fix works

pytest test/test_smoke.py::TestSmoke::test_cargo --runintegration (10m48s, resolves trailofbits/siderophile@7bca0f5):

• Resolution expands to 971 packages, up from 129 on current master / 163 in the committed baseline. This matches the ~963 figure in Cargo resolver prunes optional/feature-gated dependencies (collapses "all possible versions" to the locked tree) #189 and verifies the fix restores "resolve all possible versions" for cargo.
• The new CalledProcessError handling fired as designed: 2 cargo metadata calls failed on fully-yanked transitive deps (cloudflare-zlib-sys ^0.2.0, core2 ^0.4) and were logged + skipped instead of aborting the whole run (the pre-fix behavior that Add exception handling #125's pattern addresses).
• test/test_cargo.py: 7 passed, including the new test_resolve_from_source_only_marks_resolved_deps.

Smoke test — fails on one invariant, but it's baseline drift, not a regression

The only failure is:

AssertionError: resolution missing expected package(s) ['cargo:cc']

Root cause, verified from the resolved graph:

• In the 163-package baseline, cc was reachable only via afl@0.10.1 → cc.
• The full expansion now selects newer afl (0.15.24, 0.16.0), which dropped the cc build-dependency (afl now deps libc, rustc_version, xdg).
• The remaining paths to cc (flate2 → cloudflare-zlib-sys, bitstream-io → core2) hit the fully-yanked crates above, so those subtrees legitimately error out.

So cc genuinely isn't in the current resolution. The baseline siderophile.expected.json predates both this fix and the current crates.io state — this is exactly the drift the "open question" here and #181 anticipate. The gap needs the semantics decision plus a re-baseline via test/rebuild_expected_output.py, not a code change. I did not rebuild the baseline, since that's the decision this draft defers.

🤖 Generated with Claude Code