|
| 1 | +# Tutorial30 - Hello visionOS |
| 2 | + |
| 3 | +This tutorial demonstrates how to run Diligent Engine on Apple visionOS. A single app hosts two |
| 4 | +renderers that share the same scene: a 2D **Preview** window and a fully immersive space driven by |
| 5 | +Apple **CompositorServices**. |
| 6 | + |
| 7 | + |
| 8 | +## Prerequisites |
| 9 | + |
| 10 | +- Xcode 16+ with the visionOS SDK (`xrsimulator` / `xros`). |
| 11 | +- CMake 3.28+ (required for the `visionOS` system name). |
| 12 | +- Apple Vision Pro device or the visionOS simulator. |
| 13 | + |
| 14 | +Only the Metal backend is supported on visionOS. This tutorial is self-contained and does not use |
| 15 | +`SampleBase` / `NativeAppBase`, because neither fits the SwiftUI + CompositorServices entry point |
| 16 | +that visionOS requires. |
| 17 | + |
| 18 | + |
| 19 | +## Building |
| 20 | + |
| 21 | +Configure and build for the simulator: |
| 22 | + |
| 23 | +```bash |
| 24 | +cmake -S . -B build/visionOS -G Xcode \ |
| 25 | + -DCMAKE_SYSTEM_NAME=visionOS \ |
| 26 | + -DCMAKE_OSX_SYSROOT=xrsimulator \ |
| 27 | + -DCMAKE_BUILD_TYPE=Debug |
| 28 | +cmake --build build/visionOS --config Debug --target Tutorial30_HelloVisionOS \ |
| 29 | + -- CODE_SIGN_IDENTITY="" CODE_SIGNING_ALLOWED=NO |
| 30 | +``` |
| 31 | + |
| 32 | +For a device build, use `-DCMAKE_OSX_SYSROOT=xros` and supply a valid code signing identity. |
| 33 | + |
| 34 | + |
| 35 | +## Application Structure |
| 36 | + |
| 37 | +On visionOS the application entry point is a SwiftUI `App`. Because Swift can't call C++ directly, |
| 38 | +the tutorial is split into three layers: |
| 39 | + |
| 40 | +- **Swift / SwiftUI** (`src/visionOS/`) — declares the launcher window and the `ImmersiveSpace`, |
| 41 | + owns a `UIView` with a `CAMetalLayer` for preview and hands layers off to the C++ side. |
| 42 | +- **Objective-C++ bridge** (`src/visionOS/VisionOSAppBridge.{h,mm}`) — owns the C++ renderer |
| 43 | + objects and exposes them to Swift. |
| 44 | +- **C++ renderers** (`src/Tutorial30_{Preview,Immersive,Scene}.{hpp,cpp,mm}`) — Diligent Engine |
| 45 | + code. `Tutorial30_Scene` is shared between the two renderers. |
| 46 | + |
| 47 | +Each renderer creates its own `IRenderDevice` and `IDeviceContext`; the only thing they share is |
| 48 | +the `Tutorial30_Scene` C++ class. |
| 49 | + |
| 50 | + |
| 51 | +## Diligent Engine Integration |
| 52 | + |
| 53 | +### Preview renderer |
| 54 | + |
| 55 | +`Tutorial30_Preview` renders into a plain `CAMetalLayer`-backed `UIView`. This is the standard |
| 56 | +Metal swap chain path — identical to any iOS/macOS Diligent application — so it serves as a |
| 57 | +familiar starting point before the immersive renderer: |
| 58 | + |
| 59 | +```cpp |
| 60 | +IEngineFactoryMtl* pFactoryMtl = GetEngineFactoryMtl(); |
| 61 | +pFactoryMtl->CreateDeviceAndContextsMtl(EngineCI, &m_pDevice, &m_pImmediateContext); |
| 62 | + |
| 63 | +SwapChainDesc SCDesc; |
| 64 | +SCDesc.ColorBufferFormat = TEX_FORMAT_BGRA8_UNORM_SRGB; |
| 65 | +SCDesc.DepthBufferFormat = TEX_FORMAT_D32_FLOAT; |
| 66 | +SCDesc.Width = WidthPx; |
| 67 | +SCDesc.Height = HeightPx; |
| 68 | + |
| 69 | +NativeWindow Window{pCAMetalLayer}; |
| 70 | +pFactoryMtl->CreateSwapChainMtl(m_pDevice, m_pImmediateContext, SCDesc, Window, &m_pSwapChain); |
| 71 | +``` |
| 72 | +
|
| 73 | +Rendering is driven by a `CADisplayLink` on the main thread. The preview uses a simple orbit |
| 74 | +camera with right-handed view and projection matrices (see below). |
| 75 | +
|
| 76 | +
|
| 77 | +### Immersive renderer |
| 78 | +
|
| 79 | +`Tutorial30_Immersive` does **not** create a swap chain. CompositorServices owns the textures and |
| 80 | +hands them out per frame, so the renderer only needs a Diligent device and context: |
| 81 | +
|
| 82 | +```cpp |
| 83 | +IEngineFactoryMtl* pFactoryMtl = GetEngineFactoryMtl(); |
| 84 | +pFactoryMtl->CreateDeviceAndContextsMtl(EngineCI, &m_pDevice, &m_pImmediateContext); |
| 85 | +
|
| 86 | +m_Scene = std::make_unique<Tutorial30_Scene>(m_pDevice); |
| 87 | +m_Session = std::make_unique<CompositorServicesSession>(LayerRenderer, m_pDevice, m_pImmediateContext); |
| 88 | +``` |
| 89 | + |
| 90 | +All of the CompositorServices frame pacing (timing query, world anchor update, pose prediction, |
| 91 | +drawable acquisition and present) is encapsulated in `CompositorServicesSession` in |
| 92 | +`Diligent-GraphicsTools`. The tutorial only provides two callbacks: |
| 93 | + |
| 94 | +```cpp |
| 95 | +void Tutorial30_Immersive::RenderNewFrame() |
| 96 | +{ |
| 97 | + m_Session->RenderFrame( |
| 98 | + [this] { m_Scene->Update(); }, |
| 99 | + [this](cp_drawable_t Drawable) { RenderDrawable(Drawable); }); |
| 100 | +} |
| 101 | +``` |
| 102 | + |
| 103 | +Each drawable contains one or two views (one per eye). For each view the renderer obtains the |
| 104 | +color and depth textures from `CompositorServicesSession`, computes view/projection matrices |
| 105 | +and submits the scene: |
| 106 | + |
| 107 | +```cpp |
| 108 | +RefCntAutoPtr<ITexture> pColor = m_Session->GetColorSwapchainImage(Drawable, ViewIdx); |
| 109 | +RefCntAutoPtr<ITexture> pDepth = m_Session->GetDepthSwapchainImage(Drawable, ViewIdx); |
| 110 | + |
| 111 | +const float4x4 ViewProj = |
| 112 | + m_Session->GetViewMatrix(Drawable, ViewIdx) * |
| 113 | + m_Session->GetProjectionMatrix(Drawable, ViewIdx, NearZ, FarZ); |
| 114 | + |
| 115 | +m_Scene->Render(m_pImmediateContext, pRTV, pDSV, ViewProj); |
| 116 | +m_Session->PresentDrawable(Drawable); |
| 117 | +m_pImmediateContext->Flush(); |
| 118 | +m_pImmediateContext->FinishFrame(); |
| 119 | +m_pDevice->ReleaseStaleResources(); |
| 120 | +``` |
| 121 | +
|
| 122 | +The immersive render loop runs on a dedicated serial `dispatch_queue` with `USER_INTERACTIVE` |
| 123 | +QoS, so it never blocks the main thread. |
| 124 | +
|
| 125 | +
|
| 126 | +### Shared scene |
| 127 | +
|
| 128 | +`Tutorial30_Scene` is format-agnostic. The PSO is created lazily on the first `Render()` call, |
| 129 | +using the RTV and DSV formats of the targets the scene is rendered to: |
| 130 | +
|
| 131 | +```cpp |
| 132 | +void Tutorial30_Scene::Render(IDeviceContext* pContext, |
| 133 | + ITextureView* pRTV, |
| 134 | + ITextureView* pDSV, |
| 135 | + const float4x4& ViewProj) |
| 136 | +{ |
| 137 | + EnsurePipelineState(pRTV->GetTexture()->GetDesc().Format, |
| 138 | + pDSV->GetTexture()->GetDesc().Format); |
| 139 | + ... |
| 140 | +} |
| 141 | +``` |
| 142 | + |
| 143 | +This lets the same scene object serve both the preview swap chain and the CompositorServices |
| 144 | +drawables without hard-coding any format constants. |
| 145 | + |
| 146 | + |
| 147 | +### Conventions: right-handed, reverse-Z |
| 148 | + |
| 149 | +Both renderers use **right-handed view and projection matrices**: |
| 150 | + |
| 151 | +- Immersive uses `cp_drawable_compute_projection(..., right_up_back, ...)` and a plain |
| 152 | + `simd_inverse` of the anchor-relative view transform. |
| 153 | +- Preview builds RH matrices explicitly in `Tutorial30_Preview.mm`. |
| 154 | + |
| 155 | +Both also use **reverse-Z**: the depth target is cleared to `0`, the PSO uses |
| 156 | +`COMPARISON_FUNC_GREATER_EQUAL`, and the immersive drawable is configured with |
| 157 | +`cp_drawable_set_depth_range(FarZ, NearZ)`. |
| 158 | + |
| 159 | +Because the RH convention flips NDC winding relative to the default D3D convention, the PSO sets |
| 160 | +`FrontCounterClockwise = True` with `CULL_MODE_BACK`. This matches the canonical VR pattern used |
| 161 | +by [Tutorial28_HelloOpenXR](../Tutorial28_HelloOpenXR/readme.md). |
| 162 | + |
| 163 | + |
| 164 | +## Notes |
| 165 | + |
| 166 | +- SwiftUI is the only supported entry point on visionOS; a UIKit-only app works in the simulator |
| 167 | + but is rejected by the real device. |
| 168 | +- Assets in `assets/` are copied into the app bundle `Resources` directory by CMake, and |
| 169 | + `AppleFileSystem::OpenFile` resolves them via `CFBundleCopyResourceURL`, so no `chdir()` at |
| 170 | + startup is required. |
| 171 | +- The `SIGTERM` / `SIGKILL` seen in the debugger on window close are normal simulator shutdown |
| 172 | + signals, not a crash. Add `process handle SIGTERM --stop false --pass true --notify true` to |
| 173 | + your `~/.lldbinit` if you want lldb to ignore them. |
0 commit comments