Unable to use local clone of rust-gpu

[tmvkrpxl0]

Below is Cargo.toml file of my shader crate:

[package]
name = "shader-path"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[lib]
crate-type = ["dylib"]

[dependencies]
spirv-std = { path = "/home/tmvkrpxl0/CLionProjects/rust-gpu/crates/spirv-std" }
beam-types-shared = { workspace = true }

[package.metadata.rust-gpu.build]
capability = ["RuntimeDescriptorArray", "Int8", "InputAttachment", "SampledImageArrayNonUniformIndexing"]
extension = ["SPV_EXT_descriptor_indexing"]
debug = true
spirv-metadata = "Full"

This fails to build with below error:

$ cargo build    
🦀 Cloning `rust-gpu` repo...
[2025-03-13T05:32:41Z ERROR cargo_gpu] couldn't clone `rust-gpu` /home/tmvkrpxl0/CLionProjects/rust-gpu/crates/spirv-std to /home/tmvkrpxl0/.cache/rust-gpu/rust-gpu-repo/_home_tmvkrpxl0_CLionProjects_rust-gpu_crates_spirv-std+v0_9_0
    fatal: repository '/home/tmvkrpxl0/CLionProjects/rust-gpu/crates/spirv-std' does not exist
    
Error: couldn't clone `rust-gpu` /home/tmvkrpxl0/CLionProjects/rust-gpu/crates/spirv-std to /home/tmvkrpxl0/.cache/rust-gpu/rust-gpu-repo/_home_tmvkrpxl0_CLionProjects_rust-gpu_crates_spirv-std+v0_9_0
fatal: repository '/home/tmvkrpxl0/CLionProjects/rust-gpu/crates/spirv-std' does not exist

The path '/home/tmvkrpxl0/CLionProjects/rust-gpu/crates/spirv-std' indeed exist. It's just not root of git repository. I did use git to clone that repo but the .git directory is located at '/home/tmvkrpxl0/CLionProjects/rust-gpu/.git'

[tombh]

Oh! Sorry about that, we did actually at least write some code to support paths, but we've never tested it, so it's fallen by the wayside. We'll fix it soon!

[Firestar99]

Apparently cargo gpu build has this parameter, maybe it also accepts file paths? That way you can point the spirv-std dependency to the /crates/spirv-std subfolder and make cargo gpu point to the repository root.

        --spirv-builder-source <SPIRV_BUILDER_SOURCE>
            Source of `spirv-builder` dependency Eg: "https://github.com/Rust-GPU/rust-gpu"

[tmvkrpxl0]

Apparently cargo gpu build has this parameter, maybe it also accepts file paths? That way you can point the spirv-std dependency to the /crates/spirv-std subfolder and make cargo gpu point to the repository root.

        --spirv-builder-source <SPIRV_BUILDER_SOURCE>
            Source of `spirv-builder` dependency Eg: "https://github.com/Rust-GPU/rust-gpu"

That has same error, it tries to clone /home/tmvkrpxl0/CLionProjects/rust-gpu/crates/spirv-std as if it were git repository

[schell]

This is most definitely a use case I though we supported, so good catch on this bug @tmvkrpxl0 . We'll have to add this to our CI tests as a requirement of closing this ticket.

[tmvkrpxl0]

I'm working this this issue along with #66
Is there reason why we clone spirv-builder-cli as well, not just spirv-builder?
On surface level, just supplying target json file would be enough

[schell]

@tmvkrpxl0 I'm not sure I'm following, can you point me the code you're asking about?

The only instance where cargo-gpu is git cloneing (that I can find ATM) is here, in cargo_gpu::spirv_source::SpirvSource::checkout, but that's of the rust-gpu repo, not spirv-builder-cli?

Or maybe you're talking about .clone()?

[tmvkrpxl0]

line 209 in crates/cargo-gpu/src/install.rs
where it copies spirv-builder-cli from cargo-gpu itself into cache and compiles project-specific one. Why is it necessary?

[schell]

Ah, ok, I think there may just be a misunderstanding here. The binding name is checkout, but it's not being cloned with git.

So, onward - if the built spirv-builder-cli binary does not exist, we copy the files from SPIRV_BUILDER_FILES into the cache dir and then build it, using the checkout of spirv-builder.

The reason we do this is because spirv-builder is pinned to a specific spirv-std and subsequently rustc. This means that any program that includes spirv-builder must also be pinned to a specific rustc. This is a bit of a bummer because we want to be able to develop CPU code that is not pinned to an older rustc, and we also would like cargo-gpu not to be pinned to rustc so we need to provide a "compilation gap" between spirv-builder-cli and cargo-gpu. We also don't want people's shader crates to depend on spirv-builder...

We have talked about merging spirv-builder-cli with spirv-builder but haven't taken on that work yet.

Another complication is that folks use different spirv-std versions in their projects, so we have to be able to support building spirv-builder-cli for each version they may use, which can come from a bunch of different places (local file system being one of them).

Arguably these complications are why cargo-gpu exists.

[Firestar99]

We have talked about merging spirv-builder-cli with spirv-builder but haven't taken on that work yet.

I started working on that, as in fact spirv-builder does not care about the rustc version at all, it just depends on rustc_codegen_spirv which does need a specific rustc version. So to simplify things, I'm making the rustc_codegen_spirv dependency a feature, so that we may use SpirvBuilder from stable, and using cargo gpu to better manage building that rustc_codegen_spirv dylib.

The cli should not change at all, but makes the internal workings a lot simpler, and may significantly simplify the path to build scripts with cargo gpu as a lib.