Skip to content

merge-queue bar: foreign_cc //:llvm rebuilds in-VM (no remote cache hit), ~11min/run #293

Description

@obj-p

Symptom

During a make bar run (in-VM bazel test //... --remote_cache=grpc://100.121.199.61:9092 --remote_upload_local_results=false), the foreign_cc LLVM cmake targets rebuild locally in the VM instead of hitting the remote cache:

[316 / 352] ... Foreign Cc - CMake: Building llvm; 667s disk-cache, remote-cache, darwin-sandbox

This added ~11 min (llvm + llvm_tblgen) and pushed the bar from ~17 min to ~27 min. Other 358 actions got 133 remote cache hits, so the cache works in general — //:llvm specifically misses.

Context

This surfaced right after #287-era work made zstd hermetic (LLVM_ENABLE_ZSTD=OFF, commit on mergequeue-vz-kit). The host (macbook) rebuilt //:llvm with the new cache_entries and uploaded with --remote_upload_local_results=true, but the VM did not get a cache hit for that target and rebuilt from source. Before the change, libLLVM.dylib was a remote cache hit (which is how the cache writer's Homebrew zstd path leaked into the VM in the first place), so host↔VM sharing for this target worked at least once previously.

Likely cause

rules_foreign_cc cmake() actions are notoriously non-deterministic across machines — absolute paths, CMake/Ninja discovery, compiler wrapper paths, and sandbox layout can all perturb the action key, so the host's uploaded key and the VM's computed key diverge.

Impact

Not a correctness issue. The bar still reports the right verdict. It just rebuilds LLVM (~11 min) on every VM run that doesn't already have a warm local cache, which slows the merge-queue gate.

Options to investigate

  1. Compare the //:llvm action key host vs VM (bazel aquery) to find the diverging input.
  2. Make the mac mini (the cache server host) the canonical //:llvm writer so its key matches future consumers, or pre-warm the VM's bazel disk-cache in the post-ios snapshot.
  3. Accept the per-run LLVM rebuild and bake a warm bazel cache into the VM image.

Found while validating the Bazel merge-queue bar (chunk D).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions