Not compatible with direct linker invocations (bypassing C compiler)

[oopsbagel]

Normally -Wl, should be passed to the compiler to signify the following argument is a linker argument, but when we specify the linker as rust-lld instead of (e.g., cc), we no longer need -Wl,.

Here's a sample error:

error: linking with `rust-lld` failed: exit status: 1
  |
  = note:  "rust-lld" "-flavor" "gnu" "<sysroot>/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/rcrt1.o" "<sysroot>/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crti.o" "<sysroot>/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crtbeginS.o" "/tmp/rustcpkcTj2/symbols.o" "<1 object files omitted>" "--as-needed" "-Bstatic" "-lunwind" "-lc" "<sysroot>/lib/rustlib/x86_64-unknown-linux-musl/lib/{libcompiler_builtins-*}.rlib" "-Bdynamic" "--eh-frame-hdr" "-z" "noexecstack" "-L" "<sysroot>/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained" "-L" "<sysroot>/lib/rustlib/x86_64-unknown-linux-musl/lib" "-o" "/home/fox/prj/rayhunter/target/x86_64-unknown-linux-musl/release/deps/rayhunter_check-553f8aa90a9bc0bb" "--gc-sections" "-static" "-pie" "--no-dynamic-linker" "-z" "text" "-z" "relro" "-z" "now" "--strip-all" "/home/fox/prj/rayhunter/target/x86_64-unknown-linux-musl/release/deps/rayhunter_check_audit_data.o" "-Wl,--undefined=AUDITABLE_VERSION_INFO" "<sysroot>/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crtendS.o" "<sysroot>/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crtn.o"
  = note: some arguments are omitted. use `--verbose` to show all linker arguments
  = note: rust-lld: error: unknown argument '-Wl,--undefined=AUDITABLE_VERSION_INFO'

And just for sanity checking here's the relevant part of our project's .cargo/config.toml (we build for a bunch of targets with these same flags):

[target.x86_64-unknown-linux-musl]
linker = "rust-lld"
rustflags = ["-C", "target-feature=+crt-static"]

The following quick and dirty change was sufficient to get cargo auditable build working for our project, but this would need to be tweaked to check the linker executable before being mainlined:

diff --git a/cargo-auditable/src/rustc_wrapper.rs b/cargo-auditable/src/rustc_wrapper.rs
index 6291dee..790ac27 100644
--- a/cargo-auditable/src/rustc_wrapper.rs
+++ b/cargo-auditable/src/rustc_wrapper.rs
@@ -63,7 +63,7 @@ pub fn main(rustc_path: &OsStr) {
                 } else if is_wasm(&target_info) {
                     // We don't emit the symbol name in WASM, so nothing to do
                 } else {
-                    command.arg("-Clink-arg=-Wl,--undefined=AUDITABLE_VERSION_INFO");
+                    command.arg("-Clink-arg=--undefined=AUDITABLE_VERSION_INFO");
                 }
             } else {
                 // create_binary_file() returned None, indicating an unsupported architecture

[oopsbagel]

If you would like to exactly reproduce building for our project (done on an x86_64 linux machine, but we support building on lots of platforms for lots of targets, partially powered by rust-lld to avoid needing special toolchains for arm32 targets):

rustup toolchain install stable-x86_64-unknown-linux-gnu
rustup target install armv7-unknown-linux-musleabihf
rustup target install x86_64-unknown-linux-musl
cargo install cargo-auditable cargo-audit
git clone https://github.com/EFForg/rayhunter
cd rayhunter
cargo auditable build --release --target armv7-unknown-linux-musleabihf
cargo audit bin target/armv7-unknown-linux-musleabihf/release/rayhunter-daemon
cargo auditable build --bin rayhunter-check --release --target x86_64-unknown-linux-musl
cargo audit bin target/x86_64-unknown-linux-musl/release/rayhunter-check

[Shnatsel]

Yeah, by default Rust uses a C compiler as a frontend to the linker, and always requires passing -Wl. Using a bare linker sort of works but then falls apart whenever you need to do something more advanced. This is why e.g. mold's recipe for usage with Rust recommends

linker = "clang"
rustflags = ["-C", "link-arg=-fuse-ld=/path/to/mold"]

as opposed to just linker = "mold" or some such.

AFAIK there isn't really a reasonable way for me to support projects that configure themselves to use a "bare" linker without going through cc as rustc usually does. Stuff like cargo config get is nightly-only, and trying to accurately replicate the Cargo configuration parsing logic is going to cause more problems than it solve.

I'm afraid the only reasonable solution right now is "don't use a bare linker, use cc as rustc already does by default".

[oopsbagel]

Ah that's a bummer, because adding rustflags = ["-C", "link-arg=-fuse-ld=lld"] undoes all the work we did to get off of needing gcc/glibc. Unfortunately linking with clang isn't a workaround.

[Shnatsel]

Unfortunately Cargo does not provide the facilities to fix this properly. So I'm afraid you'll have to carry a patch if you'd like to use cargo auditable with this linker configuration. I don't think there is anything I can reasonably do about this until cargo config get is stabilized.

[Shnatsel]

I might have been wrong about it being impossible to detect the linker in use. It will eventually get lowered into a rustc codegen flag, and we have access to that. I am investigating if inspecting that would be feasible.

By the way, rayhunter's linker configuration also breaks if the system is configured to use mold via the recommended instructions:

 = note: rust-lld: error: unknown argument '-fuse-ld=/usr/bin/mold'

which just goes to show how brittle the bare linker configuration is.

[Shnatsel]

#203 fixes this on my machine, if I apply the workaround for your linker configuration not being compatible with system-wide mold, and also build specifically for musl platform (native cargo build fails with a panic in a proc macro).

Could you verify that it works in your intended pipelines as well?

[oopsbagel]

Yes it does! I just tested this with all our configured targets and it works as expected, cargo audit can read the embedded dependency tree for each of them.

Thank you very much for implementing this so quickly.

The main challenge that led us to using the bare linker (with all its shortcomings) was acquiring a gcc/glib toolchain to compile for armv7-unknown-linux-{gnu,musl}eabihf on multiple platforms required distinct per-platform/per-operating system instructions. By moving to the bare linker, we were able to remove a lot of custom development environment setup steps and still produce functional binaries. Now users can compile for armv7-unknown-linux-musleabihf and their host machine with tooling just installed by rustup, which is a win for us.

I suspect there are other projects out there with similar requirements, so it seems like if cargo auditable can be made flexible enough to work with bare linkers as you've done, it's helpful even if rust-lld is an inferior linker to mold.

(edit: and yes, we are aware of things like cargo cross, nix, chroots, all the techniques one might use to get a toolchain in place that isn't build for your operating system / distro.)