Casting between address spaces using bitcasts is UB.

[FractalFir]

In both NVVM IR & LLVM in general, using bitcasts to change address spaces is UB:

bitcast on pointers is supported, though LLVM IR forbids bitcast from being used to change the address space of a pointer.

The addrspacecast IR instruction should be used to perform pointer casts across address spaces (generic to non-generic or non-generic to generic). Casting a non-generic pointer to a different non-generic pointer is not supported. Casting from a generic to a non-generic pointer is undefined if the generic pointer does not point to an object in the target non-generic address space.

However, we do use bitcasts quite liberally in cg_nvvm, frequently to change the address space of a pointer.

rust-cuda/crates/rustc_codegen_nvvm/src/builder.rs

Line 512 in ac2674f

llvm::LLVMBuildPointerCast(bx.llbuilder, alloca, self.cx().type_ptr(), UNNAMED)

We should use address space casts here instead. The main problem will probably be catching all the places where we do this sort of thing.

[PMQ9]

Hi, I'm new here. Can I help with this issue.

It seems like NVVM IR forbids LLVMBuildPointerCast and LLVMBuildBitCast to be used to change the addres space of a pointer. We need to use addrspacecast. So we need to fix anywhere we used LLVMBuildPointerCast or LLVMBuildBitCast to LLVMBuildAddrSpaceCast.

I can only find 2 uses of LLVMBuildPointerCast at ~480 in alloca() and ~910 in pointercast() of build.rs.

I saw a partial fix in bitcast() ~890

            if kind == llvm::TypeKind::Pointer {
                let element = self.element_type(ty);
                let addrspace = llvm::LLVMGetPointerAddressSpace(ty);
                let new_ty = self.type_ptr_to_ext(element, AddressSpace::ZERO);
                if addrspace != 0 {
                    trace!("injecting addrspace cast for `{:?}` to `{:?}`", ty, new_ty);
                    val = llvm::LLVMBuildAddrSpaceCast(self.llbuilder, val, new_ty, UNNAMED);
                }

This code porperly uses LLVMBuildAddrSpaceCast when changing non-zero address space to generic

I can apply the same fix to alloca() and pointercast()

• alloca() checks if the alloca address differs from the target, if so, use LLVMBuildSpaceCast
• pointercast() check if address space is changed, if so, you LLVMBuildSpaceCast

[PMQ9]

Checking other bitcasting locations:

1. LLVMBuildBitCast and LLVMBuildPointerCast in llvm.rs, FFI declaration only, not an issue
2. ret() ~220 in build.rs uses bitcasts, but only changes pointer types within the same address space
3. imcp() ~930 in build.rs uses bitcasts, but requires both pointers to be in the same address space
4. transmute_llval() in ~ 110 in int_replace.rs uses bitcasts, but seems to be only handling type conversion, pointer related

[PMQ9]

PMQ9@200d3ec

[PMQ9]

PR @ #289

[PMQ9]

Fixed in #289