Skip to content

[HLSL][DXIL][SPIRV] QuadReadLaneAt intrinsic support#205735

Open
kcloudy0717 wants to merge 2 commits into
llvm:mainfrom
kcloudy0717:kcloudy0717/QuadReadLaneAt
Open

[HLSL][DXIL][SPIRV] QuadReadLaneAt intrinsic support#205735
kcloudy0717 wants to merge 2 commits into
llvm:mainfrom
kcloudy0717:kcloudy0717/QuadReadLaneAt

Conversation

@kcloudy0717

@kcloudy0717 kcloudy0717 commented Jun 25, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

This PR adds QuadReadLaneAt intrinsic support to both DXIL and SPIRV backends. Resolves #99174. Associated offload-test-suite PR: llvm/offload-test-suite#1335.

  • Implement QuadReadLaneAt clang builtin,
  • Link QuadReadLaneAt clang builtin with hlsl_intrinsics.h
  • Add sema checks for QuadReadLaneAt to CheckHLSLBuiltinFunctionCall in SemaChecking.cpp
  • Add codegen for QuadReadLaneAt to EmitHLSLBuiltinExpr in CGBuiltin.cpp
  • Add codegen tests to clang/test/CodeGenHLSL/builtins/QuadReadLaneAt.hlsl
  • Add sema tests to clang/test/SemaHLSL/BuiltIns/QuadReadLaneAt-errors.hlsl
  • Create the int_dx_QuadReadLaneAt intrinsic in IntrinsicsDirectX.td
  • Create the DXILOpMapping of int_dx_QuadReadLaneAt to 122 in DXIL.td
  • Create the QuadReadLaneAt.ll and QuadReadLaneAt_errors.ll tests in llvm/test/CodeGen/DirectX/
  • Create the int_spv_QuadReadLaneAt intrinsic in IntrinsicsSPIRV.td
  • In SPIRVInstructionSelector.cpp create the QuadReadLaneAt lowering and map it to int_spv_QuadReadLaneAt in SPIRVInstructionSelector::selectIntrinsic.
  • Create SPIR-V backend test case in llvm/test/CodeGen/SPIRV/hlsl-intrinsics/QuadReadLaneAt.ll

@llvmorg-github-actions llvmorg-github-actions Bot added clang:frontend Language frontend issues, e.g. anything involving "Sema" clang:codegen IR generation bugs: mangling, exceptions, etc. backend:DirectX HLSL HLSL Language Support backend:SPIR-V llvm:ir labels Jun 25, 2026
@llvmorg-github-actions

llvmorg-github-actions Bot commented Jun 25, 2026 •
edited
Loading

Copy link
Copy Markdown

@llvm/pr-subscribers-backend-directx
@llvm/pr-subscribers-hlsl

@llvm/pr-subscribers-clang-codegen

Author: Kai (kcloudy0717)

Changes

This PR adds QuadReadLaneAt intrinsic support to both DXIL and SPIRV backends. Resolves #99174.

  • Implement QuadReadLaneAt clang builtin,
  • Link QuadReadLaneAt clang builtin with hlsl_intrinsics.h
  • Add sema checks for QuadReadLaneAt to CheckHLSLBuiltinFunctionCall in SemaChecking.cpp
  • Add codegen for QuadReadLaneAt to EmitHLSLBuiltinExpr in CGBuiltin.cpp
  • Add codegen tests to clang/test/CodeGenHLSL/builtins/QuadReadLaneAt.hlsl
  • Add sema tests to clang/test/SemaHLSL/BuiltIns/QuadReadLaneAt-errors.hlsl
  • Create the int_dx_QuadReadLaneAt intrinsic in IntrinsicsDirectX.td
  • Create the DXILOpMapping of int_dx_QuadReadLaneAt to 122 in DXIL.td
  • Create the QuadReadLaneAt.ll and QuadReadLaneAt_errors.ll tests in llvm/test/CodeGen/DirectX/
  • Create the int_spv_QuadReadLaneAt intrinsic in IntrinsicsSPIRV.td
  • In SPIRVInstructionSelector.cpp create the QuadReadLaneAt lowering and map it to int_spv_QuadReadLaneAt in SPIRVInstructionSelector::selectIntrinsic.
  • Create SPIR-V backend test case in llvm/test/CodeGen/SPIRV/hlsl-intrinsics/QuadReadLaneAt.ll

Patch is 30.90 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/205735.diff

17 Files Affected:

  • (modified) clang/include/clang/Basic/Builtins.td (+6)
  • (modified) clang/include/clang/Basic/HLSLIntrinsics.td (+16)
  • (modified) clang/lib/CodeGen/CGHLSLBuiltins.cpp (+9)
  • (modified) clang/lib/CodeGen/CGHLSLRuntime.h (+1)
  • (modified) clang/lib/Sema/SemaHLSL.cpp (+2-1)
  • (added) clang/test/CodeGenHLSL/builtins/QuadReadLaneAt.hlsl (+185)
  • (added) clang/test/SemaHLSL/BuiltIns/QuadReadLaneAt-errors.hlsl (+38)
  • (modified) llvm/include/llvm/IR/IntrinsicsDirectX.td (+1)
  • (modified) llvm/include/llvm/IR/IntrinsicsSPIRV.td (+1)
  • (modified) llvm/lib/Target/DirectX/DXIL.td (+10)
  • (modified) llvm/lib/Target/DirectX/DXILShaderFlags.cpp (+1)
  • (modified) llvm/lib/Target/DirectX/DirectXTargetTransformInfo.cpp (+1)
  • (modified) llvm/lib/Target/SPIRV/SPIRVInstrInfo.td (+1)
  • (modified) llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp (+3)
  • (modified) llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.cpp (+1)
  • (added) llvm/test/CodeGen/DirectX/QuadReadLaneAt.ll (+95)
  • (added) llvm/test/CodeGen/SPIRV/hlsl-intrinsics/QuadReadLaneAt.ll (+61)
diff --git a/clang/include/clang/Basic/Builtins.td b/clang/include/clang/Basic/Builtins.td
index 63cdb787bea16..b37725498fbdc 100644
--- a/clang/include/clang/Basic/Builtins.td
+++ b/clang/include/clang/Basic/Builtins.td
@@ -5555,6 +5555,12 @@ def HLSLWavePrefixProduct : LangBuiltin<"HLSL_LANG"> {
   let Prototype = "void(...)";
 }
 
+def HLSLQuadReadLaneAt : LangBuiltin<"HLSL_LANG"> {
+  let Spellings = ["__builtin_hlsl_quad_read_lane_at"];
+  let Attributes = [NoThrow, Const];
+  let Prototype = "void(...)";
+}
+
 def HLSLQuadReadAcrossX : LangBuiltin<"HLSL_LANG"> {
   let Spellings = ["__builtin_hlsl_quad_read_across_x"];
   let Attributes = [NoThrow, Const];
diff --git a/clang/include/clang/Basic/HLSLIntrinsics.td b/clang/include/clang/Basic/HLSLIntrinsics.td
index 99259046940f1..5dd2cd47b9756 100644
--- a/clang/include/clang/Basic/HLSLIntrinsics.td
+++ b/clang/include/clang/Basic/HLSLIntrinsics.td
@@ -1322,6 +1322,22 @@ def hlsl_pow : HLSLTwoArgBuiltin<"pow", "__builtin_elementwise_pow"> {
   let VaryingMatDims = [];
 }
 
+// Returns the value from the lane with the specified index in the quad.
+def hlsl_quad_read_lane_at : HLSLBuiltin<"QuadReadLaneAt", "__builtin_hlsl_quad_read_lane_at"> {
+  let Doc = [{
+\brief Returns the value from the lane with the specified index in the quad.
+\param Val The value to read.
+\param Index The lane index.
+}];
+  let ReturnType = Varying;
+  let Args = [Varying, UIntTy];
+  let VaryingTypes = AllTypesWithBool;
+  let VaryingScalar = 1;
+  let VaryingVecSizes = [2, 3, 4];
+  let VaryingMatDims = [];
+  let IsConvergent = 1;
+}
+
 // Reads the value from the lane across the X axis of the quad.
 def hlsl_quad_read_across_x :
     HLSLOneArgBuiltin<"QuadReadAcrossX",
diff --git a/clang/lib/CodeGen/CGHLSLBuiltins.cpp b/clang/lib/CodeGen/CGHLSLBuiltins.cpp
index 20a2119e28ce1..ce3af3ffd0467 100644
--- a/clang/lib/CodeGen/CGHLSLBuiltins.cpp
+++ b/clang/lib/CodeGen/CGHLSLBuiltins.cpp
@@ -1557,6 +1557,15 @@ Value *CodeGenFunction::EmitHLSLBuiltinExpr(unsigned BuiltinID,
     return EmitIntrinsicCall(IID, {OpExpr->getType()}, ArrayRef{OpExpr},
                              "hlsl.wave.prefix.product");
   }
+  case Builtin::BI__builtin_hlsl_quad_read_lane_at: {
+    Value *OpExpr = EmitScalarExpr(E->getArg(0));
+    Value *OpIndex = EmitScalarExpr(E->getArg(1));
+    return EmitRuntimeCall(
+        Intrinsic::getOrInsertDeclaration(
+            &CGM.getModule(), CGM.getHLSLRuntime().getQuadReadLaneAtIntrinsic(),
+            {OpExpr->getType()}),
+        ArrayRef{OpExpr, OpIndex}, "hlsl.quad.read.lane.at");
+  }
   case Builtin::BI__builtin_hlsl_quad_read_across_x: {
     Value *OpExpr = EmitScalarExpr(E->getArg(0));
     Intrinsic::ID ID = CGM.getHLSLRuntime().getQuadReadAcrossXIntrinsic();
diff --git a/clang/lib/CodeGen/CGHLSLRuntime.h b/clang/lib/CodeGen/CGHLSLRuntime.h
index a126d4612a5f4..a75ffdba04a8d 100644
--- a/clang/lib/CodeGen/CGHLSLRuntime.h
+++ b/clang/lib/CodeGen/CGHLSLRuntime.h
@@ -160,6 +160,7 @@ class CGHLSLRuntime {
   GENERATE_HLSL_INTRINSIC_FUNCTION(WaveIsFirstLane, wave_is_first_lane)
   GENERATE_HLSL_INTRINSIC_FUNCTION(WaveGetLaneCount, wave_get_lane_count)
   GENERATE_HLSL_INTRINSIC_FUNCTION(WaveReadLaneAt, wave_readlane)
+  GENERATE_HLSL_INTRINSIC_FUNCTION(QuadReadLaneAt, quad_read_lane_at)
   GENERATE_HLSL_INTRINSIC_FUNCTION(QuadReadAcrossX, quad_read_across_x)
   GENERATE_HLSL_INTRINSIC_FUNCTION(QuadReadAcrossY, quad_read_across_y)
   GENERATE_HLSL_INTRINSIC_FUNCTION(QuadReadAcrossDiagonal,
diff --git a/clang/lib/Sema/SemaHLSL.cpp b/clang/lib/Sema/SemaHLSL.cpp
index 075dc97b0aef2..55c96e4f525a8 100644
--- a/clang/lib/Sema/SemaHLSL.cpp
+++ b/clang/lib/Sema/SemaHLSL.cpp
@@ -4613,7 +4613,8 @@ bool SemaHLSL::CheckBuiltinFunctionCall(unsigned BuiltinID, CallExpr *TheCall) {
 
     break;
   }
-  case Builtin::BI__builtin_hlsl_wave_read_lane_at: {
+  case Builtin::BI__builtin_hlsl_wave_read_lane_at:
+  case Builtin::BI__builtin_hlsl_quad_read_lane_at: {
     if (SemaRef.checkArgCount(TheCall, 2))
       return true;
 
diff --git a/clang/test/CodeGenHLSL/builtins/QuadReadLaneAt.hlsl b/clang/test/CodeGenHLSL/builtins/QuadReadLaneAt.hlsl
new file mode 100644
index 0000000000000..9242b5075dc10
--- /dev/null
+++ b/clang/test/CodeGenHLSL/builtins/QuadReadLaneAt.hlsl
@@ -0,0 +1,185 @@
+// RUN: %clang_cc1 -finclude-default-header -x hlsl -triple \
+// RUN:   dxil-pc-shadermodel6.3-compute %s -fnative-half-type -fnative-int16-type \
+// RUN:   -emit-llvm -disable-llvm-passes -o - | FileCheck %s \
+// RUN:   --check-prefixes=CHECK,CHECK-NATIVE_HALF -DTARGET=dx -DCC=""
+// RUN: %clang_cc1 -finclude-default-header -x hlsl -triple \
+// RUN:   dxil-pc-shadermodel6.3-compute %s -emit-llvm -disable-llvm-passes \
+// RUN:   -o - | FileCheck %s --check-prefixes=CHECK,CHECK-NO_HALF -DTARGET=dx -DCC=""
+
+// RUN: %clang_cc1 -finclude-default-header -x hlsl -triple \
+// RUN:   spirv-unknown-vulkan-compute %s -fnative-half-type -fnative-int16-type \
+// RUN:   -emit-llvm -disable-llvm-passes -o - | FileCheck %s \
+// RUN:   --check-prefixes=CHECK,CHECK-NATIVE_HALF -DTARGET=spv -DCC="spir_func "
+// RUN: %clang_cc1 -finclude-default-header -x hlsl -triple \
+// RUN:   spirv-unknown-vulkan-compute %s -emit-llvm -disable-llvm-passes \
+// RUN:   -o - | FileCheck %s --check-prefixes=CHECK,CHECK-NO_HALF -DTARGET=spv -DCC="spir_func " 
+
+// CHECK: %[[RET:.*]] = call [[CC]]i1 @llvm.[[TARGET]].quad.read.lane.at.i1(i1 %[[VAR:.*]], i32 %[[#]])
+// CHECK: ret i1 %[[RET]]
+bool test_bool(bool expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<2 x i1> @llvm.[[TARGET]].quad.read.lane.at.v2i1(<2 x i1> %[[VAR:.*]], i32 %[[#]])
+// CHECK: ret <2 x i1> %[[RET]]
+bool2 test_bool2(bool2 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<3 x i1> @llvm.[[TARGET]].quad.read.lane.at.v3i1(<3 x i1> %[[VAR:.*]], i32 %[[#]])
+// CHECK: ret <3 x i1> %[[RET]]
+bool3 test_bool3(bool3 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<4 x i1> @llvm.[[TARGET]].quad.read.lane.at.v4i1(<4 x i1> %[[VAR:.*]], i32 %[[#]])
+// CHECK: ret <4 x i1> %[[RET]]
+bool4 test_bool4(bool4 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]i32 @llvm.[[TARGET]].quad.read.lane.at.i32(i32 %[[#]], i32 %[[#]])
+// CHECK: ret i32 %[[RET]]
+int test_int(int expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<2 x i32> @llvm.[[TARGET]].quad.read.lane.at.v2i32(<2 x i32> %[[#]], i32 %[[#]])
+// CHECK: ret <2 x i32> %[[RET]]
+int2 test_int2(int2 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<3 x i32> @llvm.[[TARGET]].quad.read.lane.at.v3i32(<3 x i32> %[[#]], i32 %[[#]])
+// CHECK: ret <3 x i32> %[[RET]]
+int3 test_int3(int3 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<4 x i32> @llvm.[[TARGET]].quad.read.lane.at.v4i32(<4 x i32> %[[#]], i32 %[[#]])
+// CHECK: ret <4 x i32> %[[RET]]
+int4 test_int4(int4 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]i32 @llvm.[[TARGET]].quad.read.lane.at.i32(i32 %[[#]], i32 %[[#]])
+// CHECK: ret i32 %[[RET]]
+uint test_uint(uint expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<2 x i32> @llvm.[[TARGET]].quad.read.lane.at.v2i32(<2 x i32> %[[#]], i32 %[[#]])
+// CHECK: ret <2 x i32> %[[RET]]
+uint2 test_uint2(uint2 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<3 x i32> @llvm.[[TARGET]].quad.read.lane.at.v3i32(<3 x i32> %[[#]], i32 %[[#]])
+// CHECK: ret <3 x i32> %[[RET]]
+uint3 test_uint3(uint3 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<4 x i32> @llvm.[[TARGET]].quad.read.lane.at.v4i32(<4 x i32> %[[#]], i32 %[[#]])
+// CHECK: ret <4 x i32> %[[RET]]
+uint4 test_uint4(uint4 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]i64 @llvm.[[TARGET]].quad.read.lane.at.i64(i64 %[[#]], i32 %[[#]])
+// CHECK: ret i64 %[[RET]]
+int64_t test_int64_t(int64_t expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<2 x i64> @llvm.[[TARGET]].quad.read.lane.at.v2i64(<2 x i64> %[[#]], i32 %[[#]])
+// CHECK: ret <2 x i64> %[[RET]]
+int64_t2 test_int64_t2(int64_t2 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<3 x i64> @llvm.[[TARGET]].quad.read.lane.at.v3i64(<3 x i64> %[[#]], i32 %[[#]])
+// CHECK: ret <3 x i64> %[[RET]]
+int64_t3 test_int64_t3(int64_t3 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<4 x i64> @llvm.[[TARGET]].quad.read.lane.at.v4i64(<4 x i64> %[[#]], i32 %[[#]])
+// CHECK: ret <4 x i64> %[[RET]]
+int64_t4 test_int64_t4(int64_t4 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]i64 @llvm.[[TARGET]].quad.read.lane.at.i64(i64 %[[#]], i32 %[[#]])
+// CHECK: ret i64 %[[RET]]
+uint64_t test_uint64_t(uint64_t expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<2 x i64> @llvm.[[TARGET]].quad.read.lane.at.v2i64(<2 x i64> %[[#]], i32 %[[#]])
+// CHECK: ret <2 x i64> %[[RET]]
+uint64_t2 test_uint64_t2(uint64_t2 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<3 x i64> @llvm.[[TARGET]].quad.read.lane.at.v3i64(<3 x i64> %[[#]], i32 %[[#]])
+// CHECK: ret <3 x i64> %[[RET]]
+uint64_t3 test_uint64_t3(uint64_t3 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<4 x i64> @llvm.[[TARGET]].quad.read.lane.at.v4i64(<4 x i64> %[[#]], i32 %[[#]])
+// CHECK: ret <4 x i64> %[[RET]]
+uint64_t4 test_uint64_t4(uint64_t4 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]float @llvm.[[TARGET]].quad.read.lane.at.f32(float %[[#]], i32 %[[#]])
+// CHECK: ret float %[[RET]]
+float test_float(float expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]<2 x float> @llvm.[[TARGET]].quad.read.lane.at.v2f32(<2 x float> %[[#]], i32 %[[#]])
+// CHECK: ret <2 x float> %[[RET]]
+float2 test_float2(float2 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]<3 x float> @llvm.[[TARGET]].quad.read.lane.at.v3f32(<3 x float> %[[#]], i32 %[[#]])
+// CHECK: ret <3 x float> %[[RET]]
+float3 test_float3(float3 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]<4 x float> @llvm.[[TARGET]].quad.read.lane.at.v4f32(<4 x float> %[[#]], i32 %[[#]])
+// CHECK: ret <4 x float> %[[RET]]
+float4 test_float4(float4 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]double @llvm.[[TARGET]].quad.read.lane.at.f64(double %[[#]], i32 %[[#]])
+// CHECK: ret double %[[RET]]
+double test_double(double expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]<2 x double> @llvm.[[TARGET]].quad.read.lane.at.v2f64(<2 x double> %[[#]], i32 %[[#]])
+// CHECK: ret <2 x double> %[[RET]]
+double2 test_double2(double2 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]<3 x double> @llvm.[[TARGET]].quad.read.lane.at.v3f64(<3 x double> %[[#]], i32 %[[#]])
+// CHECK: ret <3 x double> %[[RET]]
+double3 test_double3(double3 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]<4 x double> @llvm.[[TARGET]].quad.read.lane.at.v4f64(<4 x double> %[[#]], i32 %[[#]])
+// CHECK: ret <4 x double> %[[RET]]
+double4 test_double4(double4 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK-NATIVE_HALF: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]half @llvm.[[TARGET]].quad.read.lane.at.f16(half %[[#]], i32 %[[#]])
+// CHECK-NATIVE_HALF: ret half %[[RET]]
+// CHECK-NO_HALF: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]float @llvm.[[TARGET]].quad.read.lane.at.f32(float %[[#]], i32 %[[#]])
+// CHECK-NO_HALF: ret float %[[RET]]
+half test_half(half expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK-NATIVE_HALF: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]<2 x half> @llvm.[[TARGET]].quad.read.lane.at.v2f16(<2 x half> %[[#]], i32 %[[#]])
+// CHECK-NATIVE_HALF: ret <2 x half> %[[RET]]
+// CHECK-NO_HALF: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]<2 x float> @llvm.[[TARGET]].quad.read.lane.at.v2f32(<2 x float> %[[#]], i32 %[[#]])
+// CHECK-NO_HALF: ret <2 x float> %[[RET]]
+half2 test_half2(half2 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK-NATIVE_HALF: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]<3 x half> @llvm.[[TARGET]].quad.read.lane.at.v3f16(<3 x half> %[[#]], i32 %[[#]])
+// CHECK-NATIVE_HALF: ret <3 x half> %[[RET]]
+// CHECK-NO_HALF: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]<3 x float> @llvm.[[TARGET]].quad.read.lane.at.v3f32(<3 x float> %[[#]], i32 %[[#]])
+// CHECK-NO_HALF: ret <3 x float> %[[RET]]
+half3 test_half3(half3 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK-NATIVE_HALF: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]<4 x half> @llvm.[[TARGET]].quad.read.lane.at.v4f16(<4 x half> %[[#]], i32 %[[#]])
+// CHECK-NATIVE_HALF: ret <4 x half> %[[RET]]
+// CHECK-NO_HALF: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]<4 x float> @llvm.[[TARGET]].quad.read.lane.at.v4f32(<4 x float> %[[#]], i32 %[[#]])
+// CHECK-NO_HALF: ret <4 x float> %[[RET]]
+half4 test_half4(half4 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+#ifdef __HLSL_ENABLE_16_BIT
+// CHECK-NATIVE_HALF: %[[RET:.*]] = call [[CC]]i16 @llvm.[[TARGET]].quad.read.lane.at.i16(i16 %[[#]], i32 %[[#]])
+// CHECK-NATIVE_HALF: ret i16 %[[RET]]
+int16_t test_int16_t(int16_t expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK-NATIVE_HALF: %[[RET:.*]] = call [[CC]]<2 x i16> @llvm.[[TARGET]].quad.read.lane.at.v2i16(<2 x i16> %[[#]], i32 %[[#]])
+// CHECK-NATIVE_HALF: ret <2 x i16> %[[RET]]
+int16_t2 test_int16_t2(int16_t2 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK-NATIVE_HALF: %[[RET:.*]] = call [[CC]]<3 x i16> @llvm.[[TARGET]].quad.read.lane.at.v3i16(<3 x i16> %[[#]], i32 %[[#]])
+// CHECK-NATIVE_HALF: ret <3 x i16> %[[RET]]
+int16_t3 test_int16_t3(int16_t3 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK-NATIVE_HALF: %[[RET:.*]] = call [[CC]]<4 x i16> @llvm.[[TARGET]].quad.read.lane.at.v4i16(<4 x i16> %[[#]], i32 %[[#]])
+// CHECK-NATIVE_HALF: ret <4 x i16> %[[RET]]
+int16_t4 test_int16_t4(int16_t4 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK-NATIVE_HALF: %[[RET:.*]] = call [[CC]]i16 @llvm.[[TARGET]].quad.read.lane.at.i16(i16 %[[#]], i32 %[[#]])
+// CHECK-NATIVE_HALF: ret i16 %[[RET]]
+uint16_t test_uint16_t(uint16_t expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK-NATIVE_HALF: %[[RET:.*]] = call [[CC]]<2 x i16> @llvm.[[TARGET]].quad.read.lane.at.v2i16(<2 x i16> %[[#]], i32 %[[#]])
+// CHECK-NATIVE_HALF: ret <2 x i16> %[[RET]]
+uint16_t2 test_uint16_t2(uint16_t2 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK-NATIVE_HALF: %[[RET:.*]] = call [[CC]]<3 x i16> @llvm.[[TARGET]].quad.read.lane.at.v3i16(<3 x i16> %[[#]], i32 %[[#]])
+// CHECK-NATIVE_HALF: ret <3 x i16> %[[RET]]
+uint16_t3 test_uint16_t3(uint16_t3 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK-NATIVE_HALF: %[[RET:.*]] = call [[CC]]<4 x i16> @llvm.[[TARGET]].quad.read.lane.at.v4i16(<4 x i16> %[[#]], i32 %[[#]])
+// CHECK-NATIVE_HALF: ret <4 x i16> %[[RET]]
+uint16_t4 test_uint16_t4(uint16_t4 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+#endif
diff --git a/clang/test/SemaHLSL/BuiltIns/QuadReadLaneAt-errors.hlsl b/clang/test/SemaHLSL/BuiltIns/QuadReadLaneAt-errors.hlsl
new file mode 100644
index 0000000000000..38d22b1f44772
--- /dev/null
+++ b/clang/test/SemaHLSL/BuiltIns/QuadReadLaneAt-errors.hlsl
@@ -0,0 +1,38 @@
+// RUN: %clang_cc1 -finclude-default-header -triple dxil-pc-shadermodel6.6-library %s -verify
+
+bool test_too_few_arg() {
+  return __builtin_hlsl_quad_read_lane_at();
+  // expected-error@-1 {{too few arguments to function call, expected 2, have 0}}
+}
+
+float2 test_too_few_arg_1(float2 p0) {
+  return __builtin_hlsl_quad_read_lane_at(p0);
+  // expected-error@-1 {{too few arguments to function call, expected 2, have 1}}
+}
+
+float2 test_too_many_arg(float2 p0) {
+  return __builtin_hlsl_quad_read_lane_at(p0, p0, p0);
+  // expected-error@-1 {{too many arguments to function call, expected 2, have 3}}
+}
+
+float3 test_index_double_type_check(float3 p0, double idx) {
+  return __builtin_hlsl_quad_read_lane_at(p0, idx);
+  // expected-error@-1 {{passing 'double' to parameter of incompatible type 'unsigned int'}}
+}
+
+float3 test_index_int3_type_check(float3 p0, int3 idxs) {
+  return __builtin_hlsl_quad_read_lane_at(p0, idxs);
+  // expected-error@-1 {{passing 'int3' (aka 'vector<int, 3>') to parameter of incompatible type 'unsigned int'}}
+}
+
+struct S { float f; };
+
+float3 test_index_S_type_check(float3 p0, S idx) {
+  return __builtin_hlsl_quad_read_lane_at(p0, idx);
+  // expected-error@-1 {{passing 'S' to parameter of incompatible type 'unsigned int'}}
+}
+
+S test_expr_struct_type_check(S p0, int idx) {
+  return __builtin_hlsl_quad_read_lane_at(p0, idx);
+  // expected-error@-1 {{invalid operand of type 'S' where a scalar or vector is required}}
+}
diff --git a/llvm/include/llvm/IR/IntrinsicsDirectX.td b/llvm/include/llvm/IR/IntrinsicsDirectX.td
index af360dfc78965..cbcce3bd6b6bc 100644
--- a/llvm/include/llvm/IR/IntrinsicsDirectX.td
+++ b/llvm/include/llvm/IR/IntrinsicsDirectX.td
@@ -277,6 +277,7 @@ def int_dx_wave_prefix_sum : DefaultAttrsIntrinsic<[llvm_any_ty], [LLVMMatchType
 def int_dx_wave_prefix_usum : DefaultAttrsIntrinsic<[llvm_anyint_ty], [LLVMMatchType<0>], [IntrConvergent, IntrNoMem, IntrTriviallyScalarizable]>;
 def int_dx_wave_prefix_product : DefaultAttrsIntrinsic<[llvm_any_ty], [LLVMMatchType<0>], [IntrConvergent, IntrNoMem, IntrTriviallyScalarizable]>;
 def int_dx_wave_prefix_uproduct : DefaultAttrsIntrinsic<[llvm_anyint_ty], [LLVMMatchType<0>], [IntrConvergent, IntrNoMem, IntrTriviallyScalarizable]>;
+def int_dx_quad_read_lane_at : DefaultAttrsIntrinsic<[llvm_any_ty], [LLVMMatchType<0>, llvm_i32_ty], [IntrConvergent, IntrNoMem, IntrTriviallyScalarizable]>;
 def int_dx_quad_read_across_x : DefaultAttrsIntrinsic<[llvm_any_ty], [LLVMMatchType<0>], [IntrConvergent, IntrNoMem, IntrTriviallyScalarizable]>;
 def int_dx_quad_read_across_y : DefaultAttrsIntrinsic<[llvm_any_ty], [LLVMMatchType<0>], [IntrConvergent, IntrNoMem, IntrTriviallyScalarizable]>;
 def int_dx_quad_read_across_diagonal : DefaultAttrsIntrinsic<[llvm_any_ty], [LLVMMatchType<0>], [IntrConvergent, IntrNoMem, IntrTriviallyScalarizable]>;
diff --git a/llvm/include/llvm/IR/IntrinsicsSPIRV.td b/llvm/include/llvm/IR/IntrinsicsSPIRV.td
index 6e4cf8f7e72dc..8c76ef3f9e6c7 100644
--- a/llvm/include/llvm/IR/IntrinsicsSPIRV.td
+++ b/llvm/include/llvm/IR/IntrinsicsSPIRV.td
@@ -166,6 +166,7 @@ def int_spv_rsqrt : DefaultAttrsIntrinsic<[LLVMMatchType<0>], [llvm_anyfloat_ty]
       : DefaultAttrsIntrinsic<[llvm_i32_ty], [], [IntrConvergent]>;
   def int_spv_wave_prefix_sum : DefaultAttrsIntrinsic<[llvm_any_ty], [LLVMMatchType<0>], [IntrConvergent, IntrNoMem]>;
   def int_spv_wave_prefix_product : DefaultAttrsIntrinsic<[llvm_any_ty], [LLVMMatchType<0>], [IntrConvergent, IntrNoMem]>;
+  def int_spv_quad_read_lane_at : DefaultAttrsIntrinsic<[llvm_any_ty], [LLVMMatchType<0>, llvm_i32_ty], [IntrConvergent, IntrNoMem]>;
   def int_spv_quad_read_across_x : DefaultAttrsIntrinsic<[llvm_any_ty], [LLVMMatchType<0>], [IntrConvergent, IntrNoMem]>;
   def int_spv_quad_read_across_y : DefaultAttrsIntrinsic<[llvm_any_ty], [LLVMMatchType<0>], [IntrConvergent, IntrNoMem]>;
   def int_spv_quad_read_across_diagonal : DefaultAttrsIntrinsic<[llvm_any_ty], [LLVMMatchType<0>], [IntrConvergent, IntrNoMem]>;
diff --git a/llvm/lib/Target/Direc...
[truncated]

@llvmorg-github-actions

llvmorg-github-actions Bot commented Jun 25, 2026

Copy link
Copy Markdown

@llvm/pr-subscribers-backend-spir-v

Author: Kai (kcloudy0717)

Changes

This PR adds QuadReadLaneAt intrinsic support to both DXIL and SPIRV backends. Resolves #99174.

  • Implement QuadReadLaneAt clang builtin,
  • Link QuadReadLaneAt clang builtin with hlsl_intrinsics.h
  • Add sema checks for QuadReadLaneAt to CheckHLSLBuiltinFunctionCall in SemaChecking.cpp
  • Add codegen for QuadReadLaneAt to EmitHLSLBuiltinExpr in CGBuiltin.cpp
  • Add codegen tests to clang/test/CodeGenHLSL/builtins/QuadReadLaneAt.hlsl
  • Add sema tests to clang/test/SemaHLSL/BuiltIns/QuadReadLaneAt-errors.hlsl
  • Create the int_dx_QuadReadLaneAt intrinsic in IntrinsicsDirectX.td
  • Create the DXILOpMapping of int_dx_QuadReadLaneAt to 122 in DXIL.td
  • Create the QuadReadLaneAt.ll and QuadReadLaneAt_errors.ll tests in llvm/test/CodeGen/DirectX/
  • Create the int_spv_QuadReadLaneAt intrinsic in IntrinsicsSPIRV.td
  • In SPIRVInstructionSelector.cpp create the QuadReadLaneAt lowering and map it to int_spv_QuadReadLaneAt in SPIRVInstructionSelector::selectIntrinsic.
  • Create SPIR-V backend test case in llvm/test/CodeGen/SPIRV/hlsl-intrinsics/QuadReadLaneAt.ll

Patch is 30.90 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/205735.diff

17 Files Affected:

  • (modified) clang/include/clang/Basic/Builtins.td (+6)
  • (modified) clang/include/clang/Basic/HLSLIntrinsics.td (+16)
  • (modified) clang/lib/CodeGen/CGHLSLBuiltins.cpp (+9)
  • (modified) clang/lib/CodeGen/CGHLSLRuntime.h (+1)
  • (modified) clang/lib/Sema/SemaHLSL.cpp (+2-1)
  • (added) clang/test/CodeGenHLSL/builtins/QuadReadLaneAt.hlsl (+185)
  • (added) clang/test/SemaHLSL/BuiltIns/QuadReadLaneAt-errors.hlsl (+38)
  • (modified) llvm/include/llvm/IR/IntrinsicsDirectX.td (+1)
  • (modified) llvm/include/llvm/IR/IntrinsicsSPIRV.td (+1)
  • (modified) llvm/lib/Target/DirectX/DXIL.td (+10)
  • (modified) llvm/lib/Target/DirectX/DXILShaderFlags.cpp (+1)
  • (modified) llvm/lib/Target/DirectX/DirectXTargetTransformInfo.cpp (+1)
  • (modified) llvm/lib/Target/SPIRV/SPIRVInstrInfo.td (+1)
  • (modified) llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp (+3)
  • (modified) llvm/lib/Target/SPIRV/SPIRVModuleAnalysis.cpp (+1)
  • (added) llvm/test/CodeGen/DirectX/QuadReadLaneAt.ll (+95)
  • (added) llvm/test/CodeGen/SPIRV/hlsl-intrinsics/QuadReadLaneAt.ll (+61)
diff --git a/clang/include/clang/Basic/Builtins.td b/clang/include/clang/Basic/Builtins.td
index 63cdb787bea16..b37725498fbdc 100644
--- a/clang/include/clang/Basic/Builtins.td
+++ b/clang/include/clang/Basic/Builtins.td
@@ -5555,6 +5555,12 @@ def HLSLWavePrefixProduct : LangBuiltin<"HLSL_LANG"> {
   let Prototype = "void(...)";
 }
 
+def HLSLQuadReadLaneAt : LangBuiltin<"HLSL_LANG"> {
+  let Spellings = ["__builtin_hlsl_quad_read_lane_at"];
+  let Attributes = [NoThrow, Const];
+  let Prototype = "void(...)";
+}
+
 def HLSLQuadReadAcrossX : LangBuiltin<"HLSL_LANG"> {
   let Spellings = ["__builtin_hlsl_quad_read_across_x"];
   let Attributes = [NoThrow, Const];
diff --git a/clang/include/clang/Basic/HLSLIntrinsics.td b/clang/include/clang/Basic/HLSLIntrinsics.td
index 99259046940f1..5dd2cd47b9756 100644
--- a/clang/include/clang/Basic/HLSLIntrinsics.td
+++ b/clang/include/clang/Basic/HLSLIntrinsics.td
@@ -1322,6 +1322,22 @@ def hlsl_pow : HLSLTwoArgBuiltin<"pow", "__builtin_elementwise_pow"> {
   let VaryingMatDims = [];
 }
 
+// Returns the value from the lane with the specified index in the quad.
+def hlsl_quad_read_lane_at : HLSLBuiltin<"QuadReadLaneAt", "__builtin_hlsl_quad_read_lane_at"> {
+  let Doc = [{
+\brief Returns the value from the lane with the specified index in the quad.
+\param Val The value to read.
+\param Index The lane index.
+}];
+  let ReturnType = Varying;
+  let Args = [Varying, UIntTy];
+  let VaryingTypes = AllTypesWithBool;
+  let VaryingScalar = 1;
+  let VaryingVecSizes = [2, 3, 4];
+  let VaryingMatDims = [];
+  let IsConvergent = 1;
+}
+
 // Reads the value from the lane across the X axis of the quad.
 def hlsl_quad_read_across_x :
     HLSLOneArgBuiltin<"QuadReadAcrossX",
diff --git a/clang/lib/CodeGen/CGHLSLBuiltins.cpp b/clang/lib/CodeGen/CGHLSLBuiltins.cpp
index 20a2119e28ce1..ce3af3ffd0467 100644
--- a/clang/lib/CodeGen/CGHLSLBuiltins.cpp
+++ b/clang/lib/CodeGen/CGHLSLBuiltins.cpp
@@ -1557,6 +1557,15 @@ Value *CodeGenFunction::EmitHLSLBuiltinExpr(unsigned BuiltinID,
     return EmitIntrinsicCall(IID, {OpExpr->getType()}, ArrayRef{OpExpr},
                              "hlsl.wave.prefix.product");
   }
+  case Builtin::BI__builtin_hlsl_quad_read_lane_at: {
+    Value *OpExpr = EmitScalarExpr(E->getArg(0));
+    Value *OpIndex = EmitScalarExpr(E->getArg(1));
+    return EmitRuntimeCall(
+        Intrinsic::getOrInsertDeclaration(
+            &CGM.getModule(), CGM.getHLSLRuntime().getQuadReadLaneAtIntrinsic(),
+            {OpExpr->getType()}),
+        ArrayRef{OpExpr, OpIndex}, "hlsl.quad.read.lane.at");
+  }
   case Builtin::BI__builtin_hlsl_quad_read_across_x: {
     Value *OpExpr = EmitScalarExpr(E->getArg(0));
     Intrinsic::ID ID = CGM.getHLSLRuntime().getQuadReadAcrossXIntrinsic();
diff --git a/clang/lib/CodeGen/CGHLSLRuntime.h b/clang/lib/CodeGen/CGHLSLRuntime.h
index a126d4612a5f4..a75ffdba04a8d 100644
--- a/clang/lib/CodeGen/CGHLSLRuntime.h
+++ b/clang/lib/CodeGen/CGHLSLRuntime.h
@@ -160,6 +160,7 @@ class CGHLSLRuntime {
   GENERATE_HLSL_INTRINSIC_FUNCTION(WaveIsFirstLane, wave_is_first_lane)
   GENERATE_HLSL_INTRINSIC_FUNCTION(WaveGetLaneCount, wave_get_lane_count)
   GENERATE_HLSL_INTRINSIC_FUNCTION(WaveReadLaneAt, wave_readlane)
+  GENERATE_HLSL_INTRINSIC_FUNCTION(QuadReadLaneAt, quad_read_lane_at)
   GENERATE_HLSL_INTRINSIC_FUNCTION(QuadReadAcrossX, quad_read_across_x)
   GENERATE_HLSL_INTRINSIC_FUNCTION(QuadReadAcrossY, quad_read_across_y)
   GENERATE_HLSL_INTRINSIC_FUNCTION(QuadReadAcrossDiagonal,
diff --git a/clang/lib/Sema/SemaHLSL.cpp b/clang/lib/Sema/SemaHLSL.cpp
index 075dc97b0aef2..55c96e4f525a8 100644
--- a/clang/lib/Sema/SemaHLSL.cpp
+++ b/clang/lib/Sema/SemaHLSL.cpp
@@ -4613,7 +4613,8 @@ bool SemaHLSL::CheckBuiltinFunctionCall(unsigned BuiltinID, CallExpr *TheCall) {
 
     break;
   }
-  case Builtin::BI__builtin_hlsl_wave_read_lane_at: {
+  case Builtin::BI__builtin_hlsl_wave_read_lane_at:
+  case Builtin::BI__builtin_hlsl_quad_read_lane_at: {
     if (SemaRef.checkArgCount(TheCall, 2))
       return true;
 
diff --git a/clang/test/CodeGenHLSL/builtins/QuadReadLaneAt.hlsl b/clang/test/CodeGenHLSL/builtins/QuadReadLaneAt.hlsl
new file mode 100644
index 0000000000000..9242b5075dc10
--- /dev/null
+++ b/clang/test/CodeGenHLSL/builtins/QuadReadLaneAt.hlsl
@@ -0,0 +1,185 @@
+// RUN: %clang_cc1 -finclude-default-header -x hlsl -triple \
+// RUN:   dxil-pc-shadermodel6.3-compute %s -fnative-half-type -fnative-int16-type \
+// RUN:   -emit-llvm -disable-llvm-passes -o - | FileCheck %s \
+// RUN:   --check-prefixes=CHECK,CHECK-NATIVE_HALF -DTARGET=dx -DCC=""
+// RUN: %clang_cc1 -finclude-default-header -x hlsl -triple \
+// RUN:   dxil-pc-shadermodel6.3-compute %s -emit-llvm -disable-llvm-passes \
+// RUN:   -o - | FileCheck %s --check-prefixes=CHECK,CHECK-NO_HALF -DTARGET=dx -DCC=""
+
+// RUN: %clang_cc1 -finclude-default-header -x hlsl -triple \
+// RUN:   spirv-unknown-vulkan-compute %s -fnative-half-type -fnative-int16-type \
+// RUN:   -emit-llvm -disable-llvm-passes -o - | FileCheck %s \
+// RUN:   --check-prefixes=CHECK,CHECK-NATIVE_HALF -DTARGET=spv -DCC="spir_func "
+// RUN: %clang_cc1 -finclude-default-header -x hlsl -triple \
+// RUN:   spirv-unknown-vulkan-compute %s -emit-llvm -disable-llvm-passes \
+// RUN:   -o - | FileCheck %s --check-prefixes=CHECK,CHECK-NO_HALF -DTARGET=spv -DCC="spir_func " 
+
+// CHECK: %[[RET:.*]] = call [[CC]]i1 @llvm.[[TARGET]].quad.read.lane.at.i1(i1 %[[VAR:.*]], i32 %[[#]])
+// CHECK: ret i1 %[[RET]]
+bool test_bool(bool expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<2 x i1> @llvm.[[TARGET]].quad.read.lane.at.v2i1(<2 x i1> %[[VAR:.*]], i32 %[[#]])
+// CHECK: ret <2 x i1> %[[RET]]
+bool2 test_bool2(bool2 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<3 x i1> @llvm.[[TARGET]].quad.read.lane.at.v3i1(<3 x i1> %[[VAR:.*]], i32 %[[#]])
+// CHECK: ret <3 x i1> %[[RET]]
+bool3 test_bool3(bool3 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<4 x i1> @llvm.[[TARGET]].quad.read.lane.at.v4i1(<4 x i1> %[[VAR:.*]], i32 %[[#]])
+// CHECK: ret <4 x i1> %[[RET]]
+bool4 test_bool4(bool4 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]i32 @llvm.[[TARGET]].quad.read.lane.at.i32(i32 %[[#]], i32 %[[#]])
+// CHECK: ret i32 %[[RET]]
+int test_int(int expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<2 x i32> @llvm.[[TARGET]].quad.read.lane.at.v2i32(<2 x i32> %[[#]], i32 %[[#]])
+// CHECK: ret <2 x i32> %[[RET]]
+int2 test_int2(int2 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<3 x i32> @llvm.[[TARGET]].quad.read.lane.at.v3i32(<3 x i32> %[[#]], i32 %[[#]])
+// CHECK: ret <3 x i32> %[[RET]]
+int3 test_int3(int3 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<4 x i32> @llvm.[[TARGET]].quad.read.lane.at.v4i32(<4 x i32> %[[#]], i32 %[[#]])
+// CHECK: ret <4 x i32> %[[RET]]
+int4 test_int4(int4 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]i32 @llvm.[[TARGET]].quad.read.lane.at.i32(i32 %[[#]], i32 %[[#]])
+// CHECK: ret i32 %[[RET]]
+uint test_uint(uint expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<2 x i32> @llvm.[[TARGET]].quad.read.lane.at.v2i32(<2 x i32> %[[#]], i32 %[[#]])
+// CHECK: ret <2 x i32> %[[RET]]
+uint2 test_uint2(uint2 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<3 x i32> @llvm.[[TARGET]].quad.read.lane.at.v3i32(<3 x i32> %[[#]], i32 %[[#]])
+// CHECK: ret <3 x i32> %[[RET]]
+uint3 test_uint3(uint3 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<4 x i32> @llvm.[[TARGET]].quad.read.lane.at.v4i32(<4 x i32> %[[#]], i32 %[[#]])
+// CHECK: ret <4 x i32> %[[RET]]
+uint4 test_uint4(uint4 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]i64 @llvm.[[TARGET]].quad.read.lane.at.i64(i64 %[[#]], i32 %[[#]])
+// CHECK: ret i64 %[[RET]]
+int64_t test_int64_t(int64_t expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<2 x i64> @llvm.[[TARGET]].quad.read.lane.at.v2i64(<2 x i64> %[[#]], i32 %[[#]])
+// CHECK: ret <2 x i64> %[[RET]]
+int64_t2 test_int64_t2(int64_t2 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<3 x i64> @llvm.[[TARGET]].quad.read.lane.at.v3i64(<3 x i64> %[[#]], i32 %[[#]])
+// CHECK: ret <3 x i64> %[[RET]]
+int64_t3 test_int64_t3(int64_t3 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<4 x i64> @llvm.[[TARGET]].quad.read.lane.at.v4i64(<4 x i64> %[[#]], i32 %[[#]])
+// CHECK: ret <4 x i64> %[[RET]]
+int64_t4 test_int64_t4(int64_t4 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]i64 @llvm.[[TARGET]].quad.read.lane.at.i64(i64 %[[#]], i32 %[[#]])
+// CHECK: ret i64 %[[RET]]
+uint64_t test_uint64_t(uint64_t expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<2 x i64> @llvm.[[TARGET]].quad.read.lane.at.v2i64(<2 x i64> %[[#]], i32 %[[#]])
+// CHECK: ret <2 x i64> %[[RET]]
+uint64_t2 test_uint64_t2(uint64_t2 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<3 x i64> @llvm.[[TARGET]].quad.read.lane.at.v3i64(<3 x i64> %[[#]], i32 %[[#]])
+// CHECK: ret <3 x i64> %[[RET]]
+uint64_t3 test_uint64_t3(uint64_t3 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call [[CC]]<4 x i64> @llvm.[[TARGET]].quad.read.lane.at.v4i64(<4 x i64> %[[#]], i32 %[[#]])
+// CHECK: ret <4 x i64> %[[RET]]
+uint64_t4 test_uint64_t4(uint64_t4 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]float @llvm.[[TARGET]].quad.read.lane.at.f32(float %[[#]], i32 %[[#]])
+// CHECK: ret float %[[RET]]
+float test_float(float expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]<2 x float> @llvm.[[TARGET]].quad.read.lane.at.v2f32(<2 x float> %[[#]], i32 %[[#]])
+// CHECK: ret <2 x float> %[[RET]]
+float2 test_float2(float2 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]<3 x float> @llvm.[[TARGET]].quad.read.lane.at.v3f32(<3 x float> %[[#]], i32 %[[#]])
+// CHECK: ret <3 x float> %[[RET]]
+float3 test_float3(float3 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]<4 x float> @llvm.[[TARGET]].quad.read.lane.at.v4f32(<4 x float> %[[#]], i32 %[[#]])
+// CHECK: ret <4 x float> %[[RET]]
+float4 test_float4(float4 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]double @llvm.[[TARGET]].quad.read.lane.at.f64(double %[[#]], i32 %[[#]])
+// CHECK: ret double %[[RET]]
+double test_double(double expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]<2 x double> @llvm.[[TARGET]].quad.read.lane.at.v2f64(<2 x double> %[[#]], i32 %[[#]])
+// CHECK: ret <2 x double> %[[RET]]
+double2 test_double2(double2 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]<3 x double> @llvm.[[TARGET]].quad.read.lane.at.v3f64(<3 x double> %[[#]], i32 %[[#]])
+// CHECK: ret <3 x double> %[[RET]]
+double3 test_double3(double3 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]<4 x double> @llvm.[[TARGET]].quad.read.lane.at.v4f64(<4 x double> %[[#]], i32 %[[#]])
+// CHECK: ret <4 x double> %[[RET]]
+double4 test_double4(double4 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK-NATIVE_HALF: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]half @llvm.[[TARGET]].quad.read.lane.at.f16(half %[[#]], i32 %[[#]])
+// CHECK-NATIVE_HALF: ret half %[[RET]]
+// CHECK-NO_HALF: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]float @llvm.[[TARGET]].quad.read.lane.at.f32(float %[[#]], i32 %[[#]])
+// CHECK-NO_HALF: ret float %[[RET]]
+half test_half(half expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK-NATIVE_HALF: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]<2 x half> @llvm.[[TARGET]].quad.read.lane.at.v2f16(<2 x half> %[[#]], i32 %[[#]])
+// CHECK-NATIVE_HALF: ret <2 x half> %[[RET]]
+// CHECK-NO_HALF: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]<2 x float> @llvm.[[TARGET]].quad.read.lane.at.v2f32(<2 x float> %[[#]], i32 %[[#]])
+// CHECK-NO_HALF: ret <2 x float> %[[RET]]
+half2 test_half2(half2 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK-NATIVE_HALF: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]<3 x half> @llvm.[[TARGET]].quad.read.lane.at.v3f16(<3 x half> %[[#]], i32 %[[#]])
+// CHECK-NATIVE_HALF: ret <3 x half> %[[RET]]
+// CHECK-NO_HALF: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]<3 x float> @llvm.[[TARGET]].quad.read.lane.at.v3f32(<3 x float> %[[#]], i32 %[[#]])
+// CHECK-NO_HALF: ret <3 x float> %[[RET]]
+half3 test_half3(half3 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK-NATIVE_HALF: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]<4 x half> @llvm.[[TARGET]].quad.read.lane.at.v4f16(<4 x half> %[[#]], i32 %[[#]])
+// CHECK-NATIVE_HALF: ret <4 x half> %[[RET]]
+// CHECK-NO_HALF: %[[RET:.*]] = call reassoc nnan ninf nsz arcp afn [[CC]]<4 x float> @llvm.[[TARGET]].quad.read.lane.at.v4f32(<4 x float> %[[#]], i32 %[[#]])
+// CHECK-NO_HALF: ret <4 x float> %[[RET]]
+half4 test_half4(half4 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+#ifdef __HLSL_ENABLE_16_BIT
+// CHECK-NATIVE_HALF: %[[RET:.*]] = call [[CC]]i16 @llvm.[[TARGET]].quad.read.lane.at.i16(i16 %[[#]], i32 %[[#]])
+// CHECK-NATIVE_HALF: ret i16 %[[RET]]
+int16_t test_int16_t(int16_t expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK-NATIVE_HALF: %[[RET:.*]] = call [[CC]]<2 x i16> @llvm.[[TARGET]].quad.read.lane.at.v2i16(<2 x i16> %[[#]], i32 %[[#]])
+// CHECK-NATIVE_HALF: ret <2 x i16> %[[RET]]
+int16_t2 test_int16_t2(int16_t2 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK-NATIVE_HALF: %[[RET:.*]] = call [[CC]]<3 x i16> @llvm.[[TARGET]].quad.read.lane.at.v3i16(<3 x i16> %[[#]], i32 %[[#]])
+// CHECK-NATIVE_HALF: ret <3 x i16> %[[RET]]
+int16_t3 test_int16_t3(int16_t3 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK-NATIVE_HALF: %[[RET:.*]] = call [[CC]]<4 x i16> @llvm.[[TARGET]].quad.read.lane.at.v4i16(<4 x i16> %[[#]], i32 %[[#]])
+// CHECK-NATIVE_HALF: ret <4 x i16> %[[RET]]
+int16_t4 test_int16_t4(int16_t4 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK-NATIVE_HALF: %[[RET:.*]] = call [[CC]]i16 @llvm.[[TARGET]].quad.read.lane.at.i16(i16 %[[#]], i32 %[[#]])
+// CHECK-NATIVE_HALF: ret i16 %[[RET]]
+uint16_t test_uint16_t(uint16_t expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK-NATIVE_HALF: %[[RET:.*]] = call [[CC]]<2 x i16> @llvm.[[TARGET]].quad.read.lane.at.v2i16(<2 x i16> %[[#]], i32 %[[#]])
+// CHECK-NATIVE_HALF: ret <2 x i16> %[[RET]]
+uint16_t2 test_uint16_t2(uint16_t2 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK-NATIVE_HALF: %[[RET:.*]] = call [[CC]]<3 x i16> @llvm.[[TARGET]].quad.read.lane.at.v3i16(<3 x i16> %[[#]], i32 %[[#]])
+// CHECK-NATIVE_HALF: ret <3 x i16> %[[RET]]
+uint16_t3 test_uint16_t3(uint16_t3 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+
+// CHECK-NATIVE_HALF: %[[RET:.*]] = call [[CC]]<4 x i16> @llvm.[[TARGET]].quad.read.lane.at.v4i16(<4 x i16> %[[#]], i32 %[[#]])
+// CHECK-NATIVE_HALF: ret <4 x i16> %[[RET]]
+uint16_t4 test_uint16_t4(uint16_t4 expr, uint idx) { return QuadReadLaneAt(expr, idx); }
+#endif
diff --git a/clang/test/SemaHLSL/BuiltIns/QuadReadLaneAt-errors.hlsl b/clang/test/SemaHLSL/BuiltIns/QuadReadLaneAt-errors.hlsl
new file mode 100644
index 0000000000000..38d22b1f44772
--- /dev/null
+++ b/clang/test/SemaHLSL/BuiltIns/QuadReadLaneAt-errors.hlsl
@@ -0,0 +1,38 @@
+// RUN: %clang_cc1 -finclude-default-header -triple dxil-pc-shadermodel6.6-library %s -verify
+
+bool test_too_few_arg() {
+  return __builtin_hlsl_quad_read_lane_at();
+  // expected-error@-1 {{too few arguments to function call, expected 2, have 0}}
+}
+
+float2 test_too_few_arg_1(float2 p0) {
+  return __builtin_hlsl_quad_read_lane_at(p0);
+  // expected-error@-1 {{too few arguments to function call, expected 2, have 1}}
+}
+
+float2 test_too_many_arg(float2 p0) {
+  return __builtin_hlsl_quad_read_lane_at(p0, p0, p0);
+  // expected-error@-1 {{too many arguments to function call, expected 2, have 3}}
+}
+
+float3 test_index_double_type_check(float3 p0, double idx) {
+  return __builtin_hlsl_quad_read_lane_at(p0, idx);
+  // expected-error@-1 {{passing 'double' to parameter of incompatible type 'unsigned int'}}
+}
+
+float3 test_index_int3_type_check(float3 p0, int3 idxs) {
+  return __builtin_hlsl_quad_read_lane_at(p0, idxs);
+  // expected-error@-1 {{passing 'int3' (aka 'vector<int, 3>') to parameter of incompatible type 'unsigned int'}}
+}
+
+struct S { float f; };
+
+float3 test_index_S_type_check(float3 p0, S idx) {
+  return __builtin_hlsl_quad_read_lane_at(p0, idx);
+  // expected-error@-1 {{passing 'S' to parameter of incompatible type 'unsigned int'}}
+}
+
+S test_expr_struct_type_check(S p0, int idx) {
+  return __builtin_hlsl_quad_read_lane_at(p0, idx);
+  // expected-error@-1 {{invalid operand of type 'S' where a scalar or vector is required}}
+}
diff --git a/llvm/include/llvm/IR/IntrinsicsDirectX.td b/llvm/include/llvm/IR/IntrinsicsDirectX.td
index af360dfc78965..cbcce3bd6b6bc 100644
--- a/llvm/include/llvm/IR/IntrinsicsDirectX.td
+++ b/llvm/include/llvm/IR/IntrinsicsDirectX.td
@@ -277,6 +277,7 @@ def int_dx_wave_prefix_sum : DefaultAttrsIntrinsic<[llvm_any_ty], [LLVMMatchType
 def int_dx_wave_prefix_usum : DefaultAttrsIntrinsic<[llvm_anyint_ty], [LLVMMatchType<0>], [IntrConvergent, IntrNoMem, IntrTriviallyScalarizable]>;
 def int_dx_wave_prefix_product : DefaultAttrsIntrinsic<[llvm_any_ty], [LLVMMatchType<0>], [IntrConvergent, IntrNoMem, IntrTriviallyScalarizable]>;
 def int_dx_wave_prefix_uproduct : DefaultAttrsIntrinsic<[llvm_anyint_ty], [LLVMMatchType<0>], [IntrConvergent, IntrNoMem, IntrTriviallyScalarizable]>;
+def int_dx_quad_read_lane_at : DefaultAttrsIntrinsic<[llvm_any_ty], [LLVMMatchType<0>, llvm_i32_ty], [IntrConvergent, IntrNoMem, IntrTriviallyScalarizable]>;
 def int_dx_quad_read_across_x : DefaultAttrsIntrinsic<[llvm_any_ty], [LLVMMatchType<0>], [IntrConvergent, IntrNoMem, IntrTriviallyScalarizable]>;
 def int_dx_quad_read_across_y : DefaultAttrsIntrinsic<[llvm_any_ty], [LLVMMatchType<0>], [IntrConvergent, IntrNoMem, IntrTriviallyScalarizable]>;
 def int_dx_quad_read_across_diagonal : DefaultAttrsIntrinsic<[llvm_any_ty], [LLVMMatchType<0>], [IntrConvergent, IntrNoMem, IntrTriviallyScalarizable]>;
diff --git a/llvm/include/llvm/IR/IntrinsicsSPIRV.td b/llvm/include/llvm/IR/IntrinsicsSPIRV.td
index 6e4cf8f7e72dc..8c76ef3f9e6c7 100644
--- a/llvm/include/llvm/IR/IntrinsicsSPIRV.td
+++ b/llvm/include/llvm/IR/IntrinsicsSPIRV.td
@@ -166,6 +166,7 @@ def int_spv_rsqrt : DefaultAttrsIntrinsic<[LLVMMatchType<0>], [llvm_anyfloat_ty]
       : DefaultAttrsIntrinsic<[llvm_i32_ty], [], [IntrConvergent]>;
   def int_spv_wave_prefix_sum : DefaultAttrsIntrinsic<[llvm_any_ty], [LLVMMatchType<0>], [IntrConvergent, IntrNoMem]>;
   def int_spv_wave_prefix_product : DefaultAttrsIntrinsic<[llvm_any_ty], [LLVMMatchType<0>], [IntrConvergent, IntrNoMem]>;
+  def int_spv_quad_read_lane_at : DefaultAttrsIntrinsic<[llvm_any_ty], [LLVMMatchType<0>, llvm_i32_ty], [IntrConvergent, IntrNoMem]>;
   def int_spv_quad_read_across_x : DefaultAttrsIntrinsic<[llvm_any_ty], [LLVMMatchType<0>], [IntrConvergent, IntrNoMem]>;
   def int_spv_quad_read_across_y : DefaultAttrsIntrinsic<[llvm_any_ty], [LLVMMatchType<0>], [IntrConvergent, IntrNoMem]>;
   def int_spv_quad_read_across_diagonal : DefaultAttrsIntrinsic<[llvm_any_ty], [LLVMMatchType<0>], [IntrConvergent, IntrNoMem]>;
diff --git a/llvm/lib/Target/Direc...
[truncated]

@kcloudy0717

kcloudy0717 commented Jun 25, 2026

Copy link
Copy Markdown
Contributor Author

Pinging @bob80905, @farzonl, @s-perron for review.

@kcloudy0717 kcloudy0717 force-pushed the kcloudy0717/QuadReadLaneAt branch from 528f7ea to de9cdb6 Compare June 25, 2026 08:35
@github-actions

github-actions Bot commented Jun 25, 2026 •
edited
Loading

Copy link
Copy Markdown

🐧 Linux x64 Test Results

  • 206342 tests passed
  • 6762 tests skipped

✅ The build succeeded and all tests passed.

bob80905
bob80905 reviewed Jun 25, 2026
View reviewed changes
Comment thread llvm/lib/Target/DirectX/DirectXTargetTransformInfo.cpp
Intrinsic::ID ID, unsigned ScalarOpdIdx) const {
switch (ID) {
case Intrinsic::dx_wave_readlane:
case Intrinsic::dx_quad_read_lane_at:

@bob80905 bob80905 Jun 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this needed / called anywhere?

@kcloudy0717 kcloudy0717 Jun 26, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest, I'm not sure, I only put it there because I saw WaveReadLaneAt was there.

Comment thread llvm/lib/Target/SPIRV/SPIRVInstructionSelector.cpp
case Intrinsic::spv_wave_prefix_product:
return selectWaveExclusiveScanProduct(ResVReg, ResType, I);
case Intrinsic::spv_quad_read_lane_at:
return selectWaveOpInst(ResVReg, ResType, I,

@bob80905 bob80905 Jun 25, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not a wave* instruction technically, so I wonder if this should be renamed, or could selectQuadSwap be modified?

@kcloudy0717 kcloudy0717 Jun 26, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can modify selectQuadSwap here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bob80905 bob80905 bob80905 left review comments

Assignees

No one assigned

Labels

backend:DirectX backend:SPIR-V clang:codegen IR generation bugs: mangling, exceptions, etc. clang:frontend Language frontend issues, e.g. anything involving "Sema" HLSL HLSL Language Support llvm:ir

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Implement the QuadReadLaneAt HLSL Function

3 participants

@kcloudy0717 @bob80905 @bogner