[VK] Add ray tracing pipeline, SBT, and DispatchRays bring-up

First per-backend bring-up in the PSO raytracing series (#1268). Adds
the API surface (ComputeEncoder::dispatchRays, Device::createPipelineRT,
Device::createShaderBindingTable, RayTracingPipelineCreateDesc) plus the
Vulkan implementation behind it. D3D12 and Metal stub the new methods
with not-yet-supported errors; their bring-up lands in follow-up PRs.

The pre-existing YAML schema struct from PR #1270 is renamed
ShaderBindingTable -> ShaderBindingTableDesc so the bare name is free
for the runtime resource class (parallel to BLASDesc / TLASDesc vs
AccelerationStructure). A new include/API/ShaderBindingTable.h holds
the abstract runtime base; concrete backend SBT classes derive from it
with LLVM-style classof / cast<>.

The VulkanDevice's prior `RaytracingFunctions RT` lumped AS and RT
pipeline entry points together. They split into two structs —
`ASFunctions AS` and `RTPipelineFunctions RT` — matching the actual
feature-gate split (AS+ray-query is a complete configuration on its
own, RT pipeline is layered on top). `HasRayTracingSupport` renames
to `HasASSupport`, and a separate `HasRTPipelineSupport` tracks the
new VK_KHR_ray_tracing_pipeline extension.

Vulkan bring-up:
  - Extension: VK_KHR_ray_tracing_pipeline is requested when reported,
    with VkPhysicalDeviceRayTracingPipelineFeaturesKHR chained into the
    pre-create feature query. After the query the gating
    rayTracingPipeline bool is checked; capture-replay / trace-rays-
    indirect / traversal-primitive-culling sub-features are cleared
    since the tests don't exercise them.
  - Function pointers: vkCreateRayTracingPipelinesKHR,
    vkGetRayTracingShaderGroupHandlesKHR, vkCmdTraceRaysKHR.
  - Properties: VkPhysicalDeviceRayTracingPipelinePropertiesKHR is
    cached at device-create time for SBT handle size / alignment /
    base-alignment.
  - VKRayTracingPipelineState derives from VulkanPipelineState; an
    IsRayTracing flag on the base lets the existing Vulkan cast<>
    path stay polymorphic without adding a new GPUAPI value.
    classof tests both the API and the flag. The derived class also
    carries a StringMap<uint32_t> resolving each shader EntryPoint or
    HitGroup Name to its index in the pipeline's group array, plus
    per-bucket counts so the SBT builder can slice the contiguous
    handle blob into raygen / miss / hit / callable regions.
  - createPipelineRT builds a single VkShaderModule (the DXIL library
    compiles to one SPIR-V module with multiple OpEntryPoints), then
    one VkPipelineShaderStageCreateInfo per Shader entry and one
    VkRayTracingShaderGroupCreateInfoKHR per general shader / hit
    group. Pipeline layout is shared with the compute path via
    createPipelineLayout, gated on all six RT stage flags so any
    binding can be consumed from any RT shader.
  - createShaderBindingTable allocates a host-visible coherent buffer
    big enough for four regions and lays out each entry as
    [handle bytes][localRootData bytes][padding-to-stride]. Per-region
    stride = align(handleSize + max-local-root-data-in-region,
    handleAlignment); per-region size = align(count * stride,
    baseAlignment). LocalRootData support comes free from the PR1 SBT
    schema; the test doesn't exercise it yet. Each region's
    VkStridedDeviceAddressRegionKHR derives from the buffer's
    vkGetBufferDeviceAddress.
  - dispatchRays binds the pipeline at
    VK_PIPELINE_BIND_POINT_RAY_TRACING_KHR, emits a pre-barrier with
    AS_READ + SHADER_READ/WRITE dst access into
    RAY_TRACING_SHADER_BIT_KHR, then calls vkCmdTraceRaysKHR with the
    SBT's four region structs.
  - createCommands picks the new bind point for RT pipelines so
    vkCmdBindDescriptorSets binds to the right point. executeProgram's
    isRayTracing branch builds a RayTracingPipelineCreateDesc from the
    YAML, calls createPipelineRT then createShaderBindingTable, and
    keeps both on InvocationState for the dispatch.

raygen-roundtrip.test now expects DirectX/Metal/Clang to XFAIL; on a
DXC + Vulkan combo with VK_KHR_ray_tracing_pipeline supported the test
should PASS via this implementation. On the user's Linux + clang-dxc
loop the test still XFAILs because clang-dxc doesn't yet lower
[shader("raygeneration")] entry points to SPIR-V, so the Clang XFAIL
token catches the compile failure. CI on a working DXC install will
exercise the runtime path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: 2 people

include/API/Device.h

@@ -20,6 +20,7 @@
 #include "API/Capabilities.h"
 #include "API/CommandBuffer.h"
 #include "API/RenderPass.h"
+#include "API/ShaderBindingTable.h"
 #include "API/Texture.h"
 
 #include "Support/Pipeline.h"
@@ -84,6 +85,21 @@ struct ShaderContainer {
   llvm::SmallVector<SpecializationConstant> SpecializationConstants;
 };
 
+struct RayTracingShader {
+  Stages Stage;
+  std::string EntryPoint;
+};
+
+struct RayTracingPipelineCreateDesc {
+  // All RT shaders are compiled into a single DXIL library; every entry in
+  // `Shaders` references this same blob via the backend's library-loading
+  // path.
+  const llvm::MemoryBuffer *Library = nullptr;
+  llvm::SmallVector<RayTracingShader> Shaders;
+  llvm::SmallVector<HitGroup> HitGroups;
+  RayTracingPipelineConfig Config;
+};
+
 struct TraditionalRasterPipelineCreateDesc {
   llvm::SmallVector<InputLayoutDesc> InputLayout;
   llvm::SmallVector<Format> RTFormats;
@@ -259,6 +275,14 @@ class Device {
       llvm::StringRef Name, const BindingsDesc &BindingsDesc,
       const MeshShaderRasterPipelineCreateDesc &Desc) = 0;
 
+  virtual llvm::Expected<std::unique_ptr<PipelineState>>
+  createPipelineRT(llvm::StringRef Name, const BindingsDesc &BindingsDesc,
+                   const RayTracingPipelineCreateDesc &Desc) = 0;
+
+  virtual llvm::Expected<std::unique_ptr<ShaderBindingTable>>
+  createShaderBindingTable(const PipelineState &PSO,
+                           const ShaderBindingTableDesc &Desc) = 0;
+
   virtual llvm::Expected<std::unique_ptr<Fence>>
   createFence(llvm::StringRef Name) = 0;
 

include/API/Encoder.h

@@ -25,6 +25,7 @@ class Buffer;
 class Texture;
 class PipelineState;
 class AccelerationStructure;
+class ShaderBindingTable;
 struct BLASBuildRequest;
 struct TLASBuildRequest;
 
@@ -110,6 +111,17 @@ class ComputeEncoder : public CommandEncoder {
   /// Metal). A barrier covering AS-build writes is implicitly emitted before
   /// any subsequent command that reads from the freshly-built structures.
   virtual llvm::Error batchBuildAS(llvm::ArrayRef<ASBuildItem> Items) = 0;
+
+  /// Trace rays from a RayTracing pipeline. \p PSO must have been created via
+  /// Device::createPipelineRT and \p SBT via Device::createShaderBindingTable
+  /// on that same PSO. \p Width, \p Height, \p Depth are the dispatch
+  /// dimensions passed through to the backend's DispatchRays equivalent
+  /// (D3D12 DispatchRays, Vulkan vkCmdTraceRaysKHR, Metal compute dispatch
+  /// after metal_irconverter lowering).
+  virtual llvm::Error dispatchRays(const PipelineState &PSO,
+                                   const ShaderBindingTable &SBT,
+                                   uint32_t Width, uint32_t Height,
+                                   uint32_t Depth) = 0;
 };
 
 struct Viewport {

include/API/ShaderBindingTable.h

@@ -0,0 +1,78 @@
+//===- ShaderBindingTable.h - Offload RT Shader Binding Table -------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef OFFLOADTEST_API_SHADERBINDINGTABLE_H
+#define OFFLOADTEST_API_SHADERBINDINGTABLE_H
+
+#include "API/API.h"
+
+#include <cstdint>
+
+namespace offloadtest {
+
+struct ShaderBindingTableDesc;
+
+/// Runtime shader binding table built from a RayTracing PipelineState plus a
+/// ShaderBindingTableDesc. Concrete subclasses (one per backend) hold the
+/// device-side records and any address ranges needed by the backend's
+/// DispatchRays call.
+class ShaderBindingTable {
+  GPUAPI API;
+
+public:
+  virtual ~ShaderBindingTable();
+  ShaderBindingTable(const ShaderBindingTable &) = delete;
+  ShaderBindingTable &operator=(const ShaderBindingTable &) = delete;
+
+  GPUAPI getAPI() const { return API; }
+
+protected:
+  explicit ShaderBindingTable(GPUAPI API) : API(API) {}
+};
+
+/// Per-region SBT layout numbers.
+///
+/// Every backend lays an SBT out as four concatenated regions (raygen, miss,
+/// hit-group, callable). Within a region every record is
+/// `[shader-identifier][LocalRootData][padding-to-stride]`, where stride is
+/// `align(identifierSize + max-LocalRootData-in-region, RecordAlign)` and
+/// the region itself is aligned to `BaseAlign`. The numbers match the
+/// alignment rules of both D3D12
+/// (`D3D12_SHADER_IDENTIFIER_SIZE_IN_BYTES` / `…_RECORD_BYTE_ALIGNMENT` /
+/// `…_TABLE_BYTE_ALIGNMENT`) and Vulkan
+/// (`shaderGroupHandleSize` / `shaderGroupHandleAlignment` /
+/// `shaderGroupBaseAlignment`); backend-specific constants are passed in.
+struct SBTRegionLayout {
+  uint32_t Stride = 0;
+  uint32_t Size = 0;
+  uint32_t Offset = 0; // byte offset from the start of the SBT buffer
+};
+
+struct SBTLayout {
+  SBTRegionLayout RayGen;
+  SBTRegionLayout Miss;
+  SBTRegionLayout HitGroup;
+  SBTRegionLayout Callable;
+  uint32_t TotalSize = 0;
+};
+
+/// Compute the per-region layout for an SBT description.
+///
+/// \p IdentifierSize is the size of one shader identifier (32 bytes on
+/// D3D12; `shaderGroupHandleSize` on Vulkan).
+/// \p RecordAlign is the per-record alignment (D3D12: 32 bytes; Vulkan:
+/// `shaderGroupHandleAlignment`).
+/// \p BaseAlign is the per-region alignment (D3D12: 64 bytes; Vulkan:
+/// `shaderGroupBaseAlignment`).
+SBTLayout computeSBTLayout(uint32_t IdentifierSize, uint32_t RecordAlign,
+                           uint32_t BaseAlign,
+                           const ShaderBindingTableDesc &Desc);
+
+} // namespace offloadtest
+
+#endif // OFFLOADTEST_API_SHADERBINDINGTABLE_H

include/Support/Pipeline.h

@@ -593,7 +593,7 @@ struct SBTEntry {
   llvm::SmallVector<uint8_t> LocalRootData;
 };
 
-struct ShaderBindingTable {
+struct ShaderBindingTableDesc {
   SBTEntry RayGen;
   llvm::SmallVector<SBTEntry> Miss;
   llvm::SmallVector<SBTEntry> HitGroup;
@@ -615,7 +615,7 @@ struct Pipeline {
   AccelerationStructureDescs AccelStructs;
   std::optional<RayTracingPipelineConfig> RTConfig;
   llvm::SmallVector<HitGroup> HitGroups;
-  std::optional<ShaderBindingTable> SBT;
+  std::optional<ShaderBindingTableDesc> SBT;
 
   uint32_t getVertexCount() const {
     if (DispatchParameters.VertexCount)
@@ -826,8 +826,8 @@ template <> struct MappingTraits<offloadtest::SBTEntry> {
   static void mapping(IO &I, offloadtest::SBTEntry &E);
 };
 
-template <> struct MappingTraits<offloadtest::ShaderBindingTable> {
-  static void mapping(IO &I, offloadtest::ShaderBindingTable &S);
+template <> struct MappingTraits<offloadtest::ShaderBindingTableDesc> {
+  static void mapping(IO &I, offloadtest::ShaderBindingTableDesc &S);
 };
 
 template <> struct ScalarEnumerationTraits<offloadtest::Rule> {

lib/API/DX/Device.cpp

@@ -846,6 +846,12 @@ class DXComputeEncoder : public offloadtest::ComputeEncoder {
   // ID3D12Device5 entry point and helper allocators.
   llvm::Error batchBuildAS(llvm::ArrayRef<ASBuildItem> Items) override;
 
+  llvm::Error dispatchRays(const PipelineState &, const ShaderBindingTable &,
+                           uint32_t, uint32_t, uint32_t) override {
+    return llvm::createStringError(
+        "RayTracing dispatchRays not yet supported on DirectX");
+  }
+
   void endEncodingImpl() override { popDebugGroup(); }
 };
 
@@ -1532,6 +1538,20 @@ class DXDevice : public offloadtest::Device {
     return std::make_unique<DXPipelineState>(Name, RootSig, PSO, std::nullopt);
   }
 
+  llvm::Expected<std::unique_ptr<PipelineState>>
+  createPipelineRT(llvm::StringRef, const BindingsDesc &,
+                   const RayTracingPipelineCreateDesc &) override {
+    return llvm::createStringError(
+        "RayTracing pipeline state not yet supported on DirectX");
+  }
+
+  llvm::Expected<std::unique_ptr<ShaderBindingTable>>
+  createShaderBindingTable(const PipelineState &,
+                           const ShaderBindingTableDesc &) override {
+    return llvm::createStringError(
+        "RayTracing shader binding table not yet supported on DirectX");
+  }
+
   llvm::Expected<std::unique_ptr<offloadtest::Fence>>
   createFence(llvm::StringRef Name) override {
     return DXFence::create(Device.Get(), Name);

lib/API/Device.cpp

@@ -12,6 +12,7 @@
 #include "API/Device.h"
 #include "API/Encoder.h"
 #include "API/FormatConversion.h"
+#include "API/ShaderBindingTable.h"
 
 #include "Config.h"
 
@@ -39,6 +40,56 @@ RenderPass::~RenderPass() {}
 
 AccelerationStructure::~AccelerationStructure() {}
 
+ShaderBindingTable::~ShaderBindingTable() {}
+
+static uint32_t alignUp(uint32_t Value, uint32_t Alignment) {
+  return (Value + Alignment - 1) & ~(Alignment - 1);
+}
+
+SBTLayout offloadtest::computeSBTLayout(uint32_t IdentifierSize,
+                                        uint32_t RecordAlign,
+                                        uint32_t BaseAlign,
+                                        const ShaderBindingTableDesc &Desc) {
+  auto StrideFor = [&](llvm::ArrayRef<SBTEntry> Entries) {
+    size_t MaxLocal = 0;
+    for (const auto &E : Entries)
+      MaxLocal = std::max<size_t>(MaxLocal, E.LocalRootData.size());
+    return alignUp(IdentifierSize + static_cast<uint32_t>(MaxLocal),
+                   RecordAlign);
+  };
+  auto RegionSize = [&](uint32_t Count, uint32_t Stride) {
+    return Count == 0 ? 0u : alignUp(Count * Stride, BaseAlign);
+  };
+
+  // Vulkan dispatches exactly one raygen per vkCmdTraceRaysKHR and D3D12's
+  // RayGenerationShaderRecord field is a single record; the descriptor only
+  // carries one raygen entry.
+  const llvm::ArrayRef<SBTEntry> RGEntries(&Desc.RayGen, 1);
+
+  SBTLayout L;
+  L.RayGen.Stride = StrideFor(RGEntries);
+  L.RayGen.Size = RegionSize(1, L.RayGen.Stride);
+  L.RayGen.Offset = 0;
+
+  L.Miss.Stride = StrideFor(Desc.Miss);
+  L.Miss.Size =
+      RegionSize(static_cast<uint32_t>(Desc.Miss.size()), L.Miss.Stride);
+  L.Miss.Offset = L.RayGen.Offset + L.RayGen.Size;
+
+  L.HitGroup.Stride = StrideFor(Desc.HitGroup);
+  L.HitGroup.Size = RegionSize(static_cast<uint32_t>(Desc.HitGroup.size()),
+                               L.HitGroup.Stride);
+  L.HitGroup.Offset = L.Miss.Offset + L.Miss.Size;
+
+  L.Callable.Stride = StrideFor(Desc.Callable);
+  L.Callable.Size = RegionSize(static_cast<uint32_t>(Desc.Callable.size()),
+                               L.Callable.Stride);
+  L.Callable.Offset = L.HitGroup.Offset + L.HitGroup.Size;
+
+  L.TotalSize = L.Callable.Offset + L.Callable.Size;
+  return L;
+}
+
 Device::~Device() {}
 
 llvm::Expected<llvm::SmallVector<std::unique_ptr<Device>>>

lib/API/MTL/MTLDevice.cpp

@@ -625,6 +625,12 @@ class MTLComputeEncoder : public offloadtest::ComputeEncoder {
   // MTL::Device handle (used to allocate scratch and instance buffers).
   llvm::Error batchBuildAS(llvm::ArrayRef<ASBuildItem> Items) override;
 
+  llvm::Error dispatchRays(const PipelineState &, const ShaderBindingTable &,
+                           uint32_t, uint32_t, uint32_t) override {
+    return llvm::createStringError(
+        "RayTracing dispatchRays not yet supported on Metal");
+  }
+
   /// Lazily transition into an AccelerationStructureCommandEncoder; mirrors
   /// the existing compute↔blit lazy switch.
   llvm::Error ensureASEncoder() {
@@ -1711,6 +1717,20 @@ class MTLDevice : public offloadtest::Device {
 
   Queue &getGraphicsQueue() override { return GraphicsQueue; }
 
+  llvm::Expected<std::unique_ptr<PipelineState>>
+  createPipelineRT(llvm::StringRef, const BindingsDesc &,
+                   const RayTracingPipelineCreateDesc &) override {
+    return llvm::createStringError(
+        "RayTracing pipeline state not yet supported on Metal");
+  }
+
+  llvm::Expected<std::unique_ptr<ShaderBindingTable>>
+  createShaderBindingTable(const PipelineState &,
+                           const ShaderBindingTableDesc &) override {
+    return llvm::createStringError(
+        "RayTracing shader binding table not yet supported on Metal");
+  }
+
   llvm::Expected<std::unique_ptr<offloadtest::Fence>>
   createFence(llvm::StringRef Name) override {
     return MTLFence::create(Device, Name);