[VK] Add ray tracing pipeline, SBT, and DispatchRays bring-up

First per-backend bring-up in the PSO raytracing series (#1268). Adds
the API surface (ComputeEncoder::dispatchRays, Device::createPipelineRT,
Device::createShaderBindingTable, RayTracingPipelineCreateDesc) plus the
Vulkan implementation behind it. D3D12 and Metal stub the new methods
with not-yet-supported errors; their bring-up lands in follow-up PRs.

The pre-existing YAML schema struct from PR #1270 is renamed
ShaderBindingTable -> ShaderBindingTableDesc so the bare name is free
for the runtime resource class (parallel to BLASDesc / TLASDesc vs
AccelerationStructure). A new include/API/ShaderBindingTable.h holds
the abstract runtime base; concrete backend SBT classes derive from it
with LLVM-style classof / cast<>.

The VulkanDevice's prior `RaytracingFunctions RT` lumped AS and RT
pipeline entry points together. They split into two structs —
`ASFunctions AS` and `RTPipelineFunctions RT` — matching the actual
feature-gate split (AS+ray-query is a complete configuration on its
own, RT pipeline is layered on top). `HasRayTracingSupport` renames
to `HasASSupport`, and a separate `HasRTPipelineSupport` tracks the
new VK_KHR_ray_tracing_pipeline extension.

Vulkan bring-up:
  - Extension: VK_KHR_ray_tracing_pipeline is requested when reported,
    with VkPhysicalDeviceRayTracingPipelineFeaturesKHR chained into the
    pre-create feature query. After the query the gating
    rayTracingPipeline bool is checked; capture-replay / trace-rays-
    indirect / traversal-primitive-culling sub-features are cleared
    since the tests don't exercise them.
  - Function pointers: vkCreateRayTracingPipelinesKHR,
    vkGetRayTracingShaderGroupHandlesKHR, vkCmdTraceRaysKHR.
  - Properties: VkPhysicalDeviceRayTracingPipelinePropertiesKHR is
    cached at device-create time for SBT handle size / alignment /
    base-alignment.
  - VKRayTracingPipelineState derives from VulkanPipelineState; an
    IsRayTracing flag on the base lets the existing Vulkan cast<>
    path stay polymorphic without adding a new GPUAPI value.
    classof tests both the API and the flag. The derived class also
    carries a StringMap<uint32_t> resolving each shader EntryPoint or
    HitGroup Name to its index in the pipeline's group array, plus
    per-bucket counts so the SBT builder can slice the contiguous
    handle blob into raygen / miss / hit / callable regions.
  - createPipelineRT builds a single VkShaderModule (the DXIL library
    compiles to one SPIR-V module with multiple OpEntryPoints), then
    one VkPipelineShaderStageCreateInfo per Shader entry and one
    VkRayTracingShaderGroupCreateInfoKHR per general shader / hit
    group. Pipeline layout is shared with the compute path via
    createPipelineLayout, gated on all six RT stage flags so any
    binding can be consumed from any RT shader.
  - createShaderBindingTable allocates a host-visible coherent buffer
    big enough for four regions and lays out each entry as
    [handle bytes][localRootData bytes][padding-to-stride]. Per-region
    stride = align(handleSize + max-local-root-data-in-region,
    handleAlignment); per-region size = align(count * stride,
    baseAlignment). LocalRootData support comes free from the PR1 SBT
    schema; the test doesn't exercise it yet. Each region's
    VkStridedDeviceAddressRegionKHR derives from the buffer's
    vkGetBufferDeviceAddress.
  - dispatchRays binds the pipeline at
    VK_PIPELINE_BIND_POINT_RAY_TRACING_KHR, emits a pre-barrier with
    AS_READ + SHADER_READ/WRITE dst access into
    RAY_TRACING_SHADER_BIT_KHR, then calls vkCmdTraceRaysKHR with the
    SBT's four region structs.
  - createCommands picks the new bind point for RT pipelines so
    vkCmdBindDescriptorSets binds to the right point. executeProgram's
    isRayTracing branch builds a RayTracingPipelineCreateDesc from the
    YAML, calls createPipelineRT then createShaderBindingTable, and
    keeps both on InvocationState for the dispatch.

raygen-roundtrip.test now expects DirectX/Metal/Clang to XFAIL; on a
DXC + Vulkan combo with VK_KHR_ray_tracing_pipeline supported the test
should PASS via this implementation. On the user's Linux + clang-dxc
loop the test still XFAILs because clang-dxc doesn't yet lower
[shader("raygeneration")] entry points to SPIR-V, so the Clang XFAIL
token catches the compile failure. CI on a working DXC install will
exercise the runtime path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: MarijnS95

include/API/Device.h

@@ -20,6 +20,7 @@
 #include "API/Capabilities.h"
 #include "API/CommandBuffer.h"
 #include "API/RenderPass.h"
+#include "API/ShaderBindingTable.h"
 #include "API/Texture.h"
 
 #include "Support/Pipeline.h"
@@ -83,6 +84,21 @@ struct ShaderContainer {
   llvm::SmallVector<SpecializationConstant> SpecializationConstants;
 };
 
+struct RayTracingShader {
+  Stages Stage;
+  std::string EntryPoint;
+};
+
+struct RayTracingPipelineCreateDesc {
+  // All RT shaders are compiled into a single DXIL library; every entry in
+  // `Shaders` references this same blob via the backend's library-loading
+  // path.
+  const llvm::MemoryBuffer *Library = nullptr;
+  llvm::SmallVector<RayTracingShader> Shaders;
+  llvm::SmallVector<HitGroup> HitGroups;
+  RayTracingPipelineConfig Config;
+};
+
 struct TraditionalRasterPipelineCreateDesc {
   llvm::SmallVector<InputLayoutDesc> InputLayout;
   llvm::SmallVector<Format> RTFormats;
@@ -215,6 +231,14 @@ class Device {
       llvm::StringRef Name, const BindingsDesc &BindingsDesc,
       const TraditionalRasterPipelineCreateDesc &Desc) = 0;
 
+  virtual llvm::Expected<std::unique_ptr<PipelineState>>
+  createPipelineRT(llvm::StringRef Name, const BindingsDesc &BindingsDesc,
+                   const RayTracingPipelineCreateDesc &Desc) = 0;
+
+  virtual llvm::Expected<std::unique_ptr<ShaderBindingTable>>
+  createShaderBindingTable(const PipelineState &PSO,
+                           const ShaderBindingTableDesc &Desc) = 0;
+
   virtual llvm::Expected<std::unique_ptr<Fence>>
   createFence(llvm::StringRef Name) = 0;
 

include/API/Encoder.h

@@ -24,6 +24,7 @@ namespace offloadtest {
 class Buffer;
 class PipelineState;
 class AccelerationStructure;
+class ShaderBindingTable;
 struct BLASBuildRequest;
 struct TLASBuildRequest;
 
@@ -105,6 +106,17 @@ class ComputeEncoder : public CommandEncoder {
   /// Metal). A barrier covering AS-build writes is implicitly emitted before
   /// any subsequent command that reads from the freshly-built structures.
   virtual llvm::Error batchBuildAS(llvm::ArrayRef<ASBuildItem> Items) = 0;
+
+  /// Trace rays from a RayTracing pipeline. \p PSO must have been created via
+  /// Device::createPipelineRT and \p SBT via Device::createShaderBindingTable
+  /// on that same PSO. \p Width, \p Height, \p Depth are the dispatch
+  /// dimensions passed through to the backend's DispatchRays equivalent
+  /// (D3D12 DispatchRays, Vulkan vkCmdTraceRaysKHR, Metal compute dispatch
+  /// after metal_irconverter lowering).
+  virtual llvm::Error dispatchRays(const PipelineState &PSO,
+                                   const ShaderBindingTable &SBT,
+                                   uint32_t Width, uint32_t Height,
+                                   uint32_t Depth) = 0;
 };
 
 struct Viewport {

include/API/ShaderBindingTable.h

@@ -0,0 +1,36 @@
+//===- ShaderBindingTable.h - Offload RT Shader Binding Table -------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef OFFLOADTEST_API_SHADERBINDINGTABLE_H
+#define OFFLOADTEST_API_SHADERBINDINGTABLE_H
+
+#include "API/API.h"
+
+namespace offloadtest {
+
+/// Runtime shader binding table built from a RayTracing PipelineState plus a
+/// ShaderBindingTableDesc. Concrete subclasses (one per backend) hold the
+/// device-side records and any address ranges needed by the backend's
+/// DispatchRays call.
+class ShaderBindingTable {
+  GPUAPI API;
+
+public:
+  virtual ~ShaderBindingTable();
+  ShaderBindingTable(const ShaderBindingTable &) = delete;
+  ShaderBindingTable &operator=(const ShaderBindingTable &) = delete;
+
+  GPUAPI getAPI() const { return API; }
+
+protected:
+  explicit ShaderBindingTable(GPUAPI API) : API(API) {}
+};
+
+} // namespace offloadtest
+
+#endif // OFFLOADTEST_API_SHADERBINDINGTABLE_H

include/Support/Pipeline.h

@@ -592,7 +592,7 @@ struct SBTEntry {
   llvm::SmallVector<uint8_t> LocalRootData;
 };
 
-struct ShaderBindingTable {
+struct ShaderBindingTableDesc {
   SBTEntry RayGen;
   llvm::SmallVector<SBTEntry> Miss;
   llvm::SmallVector<SBTEntry> HitGroup;
@@ -614,7 +614,7 @@ struct Pipeline {
   AccelerationStructureDescs AccelStructs;
   std::optional<RayTracingPipelineConfig> RTConfig;
   llvm::SmallVector<HitGroup> HitGroups;
-  std::optional<ShaderBindingTable> SBT;
+  std::optional<ShaderBindingTableDesc> SBT;
 
   uint32_t getVertexCount() const {
     if (DispatchParameters.VertexCount)
@@ -825,8 +825,8 @@ template <> struct MappingTraits<offloadtest::SBTEntry> {
   static void mapping(IO &I, offloadtest::SBTEntry &E);
 };
 
-template <> struct MappingTraits<offloadtest::ShaderBindingTable> {
-  static void mapping(IO &I, offloadtest::ShaderBindingTable &S);
+template <> struct MappingTraits<offloadtest::ShaderBindingTableDesc> {
+  static void mapping(IO &I, offloadtest::ShaderBindingTableDesc &S);
 };
 
 template <> struct ScalarEnumerationTraits<offloadtest::Rule> {

lib/API/DX/Device.cpp

@@ -730,6 +730,12 @@ class DXComputeEncoder : public offloadtest::ComputeEncoder {
   // ID3D12Device5 entry point and helper allocators.
   llvm::Error batchBuildAS(llvm::ArrayRef<ASBuildItem> Items) override;
 
+  llvm::Error dispatchRays(const PipelineState &, const ShaderBindingTable &,
+                           uint32_t, uint32_t, uint32_t) override {
+    return llvm::createStringError(
+        "RayTracing dispatchRays not yet supported on DirectX");
+  }
+
   void endEncodingImpl() override { popDebugGroup(); }
 };
 
@@ -1356,6 +1362,20 @@ class DXDevice : public offloadtest::Device {
     return std::make_unique<DXPipelineState>(Name, RootSig, PSO, std::nullopt);
   }
 
+  llvm::Expected<std::unique_ptr<PipelineState>>
+  createPipelineRT(llvm::StringRef, const BindingsDesc &,
+                   const RayTracingPipelineCreateDesc &) override {
+    return llvm::createStringError(
+        "RayTracing pipeline state not yet supported on DirectX");
+  }
+
+  llvm::Expected<std::unique_ptr<ShaderBindingTable>>
+  createShaderBindingTable(const PipelineState &,
+                           const ShaderBindingTableDesc &) override {
+    return llvm::createStringError(
+        "RayTracing shader binding table not yet supported on DirectX");
+  }
+
   llvm::Expected<std::unique_ptr<offloadtest::Fence>>
   createFence(llvm::StringRef Name) override {
     return DXFence::create(Device.Get(), Name);

lib/API/Device.cpp

@@ -12,6 +12,7 @@
 #include "API/Device.h"
 #include "API/Encoder.h"
 #include "API/FormatConversion.h"
+#include "API/ShaderBindingTable.h"
 
 #include "Config.h"
 
@@ -39,6 +40,8 @@ RenderPass::~RenderPass() {}
 
 AccelerationStructure::~AccelerationStructure() {}
 
+ShaderBindingTable::~ShaderBindingTable() {}
+
 Device::~Device() {}
 
 llvm::Expected<llvm::SmallVector<std::unique_ptr<Device>>>

lib/API/MTL/MTLDevice.cpp

@@ -602,6 +602,12 @@ class MTLComputeEncoder : public offloadtest::ComputeEncoder {
   // MTL::Device handle (used to allocate scratch and instance buffers).
   llvm::Error batchBuildAS(llvm::ArrayRef<ASBuildItem> Items) override;
 
+  llvm::Error dispatchRays(const PipelineState &, const ShaderBindingTable &,
+                           uint32_t, uint32_t, uint32_t) override {
+    return llvm::createStringError(
+        "RayTracing dispatchRays not yet supported on Metal");
+  }
+
   /// Lazily transition into an AccelerationStructureCommandEncoder; mirrors
   /// the existing compute↔blit lazy switch.
   llvm::Error ensureASEncoder() {
@@ -1662,6 +1668,20 @@ class MTLDevice : public offloadtest::Device {
 
   Queue &getGraphicsQueue() override { return GraphicsQueue; }
 
+  llvm::Expected<std::unique_ptr<PipelineState>>
+  createPipelineRT(llvm::StringRef, const BindingsDesc &,
+                   const RayTracingPipelineCreateDesc &) override {
+    return llvm::createStringError(
+        "RayTracing pipeline state not yet supported on Metal");
+  }
+
+  llvm::Expected<std::unique_ptr<ShaderBindingTable>>
+  createShaderBindingTable(const PipelineState &,
+                           const ShaderBindingTableDesc &) override {
+    return llvm::createStringError(
+        "RayTracing shader binding table not yet supported on Metal");
+  }
+
   llvm::Expected<std::unique_ptr<offloadtest::Fence>>
   createFence(llvm::StringRef Name) override {
     return MTLFence::create(Device, Name);