|
| 1 | +# Tutorial30 - Hello visionOS |
| 2 | + |
| 3 | +This tutorial demonstrates how to run Diligent Engine on Apple visionOS. A single app hosts two |
| 4 | +renderers that share the same scene: a 2D **Preview** window and a fully immersive space driven by |
| 5 | +Apple **CompositorServices**. |
| 6 | + |
| 7 | + |
| 8 | + |
| 9 | +## Prerequisites |
| 10 | + |
| 11 | +- Xcode 16+ with the visionOS SDK (`xrsimulator` / `xros`). |
| 12 | +- CMake 3.28+ (required for the `visionOS` system name). |
| 13 | +- Apple Vision Pro device or the visionOS simulator. |
| 14 | + |
| 15 | +Only the Metal backend is supported on visionOS. This tutorial is self-contained and does not use |
| 16 | +`SampleBase` / `NativeAppBase`, because neither fits the SwiftUI + CompositorServices entry point |
| 17 | +that visionOS requires. |
| 18 | + |
| 19 | + |
| 20 | +## Building |
| 21 | + |
| 22 | +Configure and build for the simulator: |
| 23 | + |
| 24 | +```bash |
| 25 | +cmake -S . -B build/visionOS -G Xcode \ |
| 26 | + -DCMAKE_SYSTEM_NAME=visionOS \ |
| 27 | + -DCMAKE_OSX_SYSROOT=xrsimulator \ |
| 28 | + -DCMAKE_BUILD_TYPE=Debug |
| 29 | +cmake --build build/visionOS --config Debug --target Tutorial30_HelloVisionOS \ |
| 30 | + -- CODE_SIGN_IDENTITY="" CODE_SIGNING_ALLOWED=NO |
| 31 | +``` |
| 32 | + |
| 33 | +For a device build, use `-DCMAKE_OSX_SYSROOT=xros` and supply a valid code signing identity. |
| 34 | + |
| 35 | + |
| 36 | +## Application Structure |
| 37 | + |
| 38 | +On visionOS the application entry point is a SwiftUI `App`. Because Swift can't call C++ directly, |
| 39 | +the tutorial is split into three layers: |
| 40 | + |
| 41 | +- **Swift / SwiftUI** (`src/visionOS/`) — declares the launcher window and the `ImmersiveSpace`, |
| 42 | + owns a `UIView` with a `CAMetalLayer` for preview and hands layers off to the C++ side. |
| 43 | +- **Objective-C++ bridge** (`src/visionOS/VisionOSAppBridge.{h,mm}`) — owns the C++ preview / |
| 44 | + immersive driver objects and exposes them to Swift. |
| 45 | +- **C++ renderers** (`src/Tutorial30_{RenderEngine,Preview,Immersive,Scene}.{hpp,cpp}`) — |
| 46 | + Diligent Engine code. `Tutorial30_RenderEngine` owns the process-wide Metal device and |
| 47 | + immediate context; `Tutorial30_Scene` is shared by the preview and immersive paths. |
| 48 | + |
| 49 | +The preview is paused while an immersive space is opening or active, so the shared immediate |
| 50 | +context and scene are not rendered from both SwiftUI paths at the same time. As an additional |
| 51 | +safeguard, the Objective-C++ bridge submits all preview and immersive rendering work to one shared |
| 52 | +serial render queue. |
| 53 | + |
| 54 | + |
| 55 | +## Diligent Engine Integration |
| 56 | + |
| 57 | +### Preview renderer |
| 58 | + |
| 59 | +`Tutorial30_Preview` renders into a plain `CAMetalLayer`-backed `UIView`. This is the standard |
| 60 | +Metal swap chain path — identical to any iOS/macOS Diligent application — so it serves as a |
| 61 | +familiar starting point before the immersive renderer: |
| 62 | + |
| 63 | +```cpp |
| 64 | +Tutorial30_RenderEngine& Engine = Tutorial30_RenderEngine::Get(); |
| 65 | +IEngineFactoryMtl* pFactoryMtl = static_cast<IEngineFactoryMtl*>(Engine.GetEngineFactory()); |
| 66 | + |
| 67 | +SwapChainDesc SCDesc; |
| 68 | +SCDesc.ColorBufferFormat = TEX_FORMAT_BGRA8_UNORM_SRGB; |
| 69 | +SCDesc.DepthBufferFormat = TEX_FORMAT_D32_FLOAT; |
| 70 | +SCDesc.Width = WidthPx; |
| 71 | +SCDesc.Height = HeightPx; |
| 72 | + |
| 73 | +NativeWindow Window{pCAMetalLayer}; |
| 74 | +pFactoryMtl->CreateSwapChainMtl(Engine.GetDevice(), Engine.GetImmediateContext(), |
| 75 | + SCDesc, Window, &m_pSwapChain); |
| 76 | +``` |
| 77 | +
|
| 78 | +Rendering is scheduled by a `CADisplayLink` on the main thread and executed on the shared serial |
| 79 | +render queue. The preview uses a simple orbit camera with right-handed view and projection matrices |
| 80 | +(see below). |
| 81 | +
|
| 82 | +
|
| 83 | +### Immersive renderer |
| 84 | +
|
| 85 | +`Tutorial30_Immersive` does **not** create a swap chain. CompositorServices owns the textures and |
| 86 | +hands them out per frame, so the renderer only attaches a `CompositorServicesSession` to the shared |
| 87 | +Diligent device and context: |
| 88 | +
|
| 89 | +```cpp |
| 90 | +Tutorial30_RenderEngine& Engine = Tutorial30_RenderEngine::Get(); |
| 91 | +
|
| 92 | +m_Session = std::make_unique<CompositorServicesSession>(LayerRenderer, |
| 93 | + Engine.GetDevice(), |
| 94 | + Engine.GetImmediateContext()); |
| 95 | +``` |
| 96 | + |
| 97 | +All of the CompositorServices frame pacing (timing query, world anchor update, pose prediction, |
| 98 | +drawable acquisition and present) is encapsulated in `CompositorServicesSession` in |
| 99 | +`Diligent-GraphicsTools`. The tutorial only provides two callbacks: |
| 100 | + |
| 101 | +```cpp |
| 102 | +void Tutorial30_Immersive::RenderFrame() |
| 103 | +{ |
| 104 | + Tutorial30_Scene& Scene = Tutorial30_RenderEngine::Get().GetScene(); |
| 105 | + |
| 106 | + m_Session->RenderFrame( |
| 107 | + [&Scene] { Scene.Update(); }, |
| 108 | + [this](void* pDrawable) { RenderDrawable(pDrawable); }); |
| 109 | +} |
| 110 | +``` |
| 111 | + |
| 112 | +Each drawable contains one or two views (one per eye). For each view the renderer obtains the |
| 113 | +color and depth textures from `CompositorServicesSession`, computes view/projection matrices |
| 114 | +and submits the scene: |
| 115 | + |
| 116 | +```cpp |
| 117 | +RefCntAutoPtr<ITexture> pColor = m_Session->GetColorSwapchainImage(pDrawable, ViewIdx); |
| 118 | +RefCntAutoPtr<ITexture> pDepth = m_Session->GetDepthSwapchainImage(pDrawable, ViewIdx); |
| 119 | + |
| 120 | +const float4x4 ViewProj = |
| 121 | + m_Session->GetViewMatrix(pDrawable, ViewIdx) * |
| 122 | + m_Session->GetProjectionMatrix(pDrawable, ViewIdx, NearZ, FarZ); |
| 123 | + |
| 124 | +Scene.Render(pImmediateContext, |
| 125 | + pColor->GetDefaultView(TEXTURE_VIEW_RENDER_TARGET), |
| 126 | + pDepth->GetDefaultView(TEXTURE_VIEW_DEPTH_STENCIL), |
| 127 | + ViewProj); |
| 128 | + |
| 129 | +m_Session->PresentDrawable(pDrawable); |
| 130 | +pImmediateContext->Flush(); |
| 131 | +pImmediateContext->FinishFrame(); |
| 132 | +pDevice->ReleaseStaleResources(); |
| 133 | +``` |
| 134 | +
|
| 135 | +The immersive render loop runs on the shared serial render queue with `USER_INTERACTIVE` QoS, so it |
| 136 | +never blocks the main thread and cannot overlap the preview renderer. |
| 137 | +
|
| 138 | +
|
| 139 | +### Shared scene |
| 140 | +
|
| 141 | +`Tutorial30_Scene` is format-agnostic. The PSO is created lazily on the first `Render()` call, |
| 142 | +using the RTV and DSV formats of the targets the scene is rendered to: |
| 143 | +
|
| 144 | +```cpp |
| 145 | +void Tutorial30_Scene::Render(IDeviceContext* pContext, |
| 146 | + ITextureView* pRTV, |
| 147 | + ITextureView* pDSV, |
| 148 | + const float4x4& ViewProj) |
| 149 | +{ |
| 150 | + EnsurePipelineState(pRTV->GetTexture()->GetDesc().Format, |
| 151 | + pDSV->GetTexture()->GetDesc().Format); |
| 152 | + ... |
| 153 | +} |
| 154 | +``` |
| 155 | + |
| 156 | +This lets the same scene object serve both the preview swap chain and the CompositorServices |
| 157 | +drawables without hard-coding any format constants. |
| 158 | + |
| 159 | + |
| 160 | +### Conventions: right-handed, reverse-Z |
| 161 | + |
| 162 | +Both renderers use **right-handed view and projection matrices**: |
| 163 | + |
| 164 | +- Immersive uses `cp_drawable_compute_projection(..., right_up_back, ...)` and a plain |
| 165 | + `simd_inverse` of the anchor-relative view transform. |
| 166 | +- Preview builds RH matrices explicitly in `Tutorial30_Preview.cpp`. |
| 167 | + |
| 168 | +Both also use **reverse-Z**: the depth target is cleared to `0`, the PSO uses |
| 169 | +`COMPARISON_FUNC_GREATER_EQUAL`, and the immersive drawable is configured with |
| 170 | +`cp_drawable_set_depth_range(FarZ, NearZ)`. |
| 171 | + |
| 172 | +Because the RH convention flips NDC winding relative to the default D3D convention, the PSO sets |
| 173 | +`FrontCounterClockwise = True` with `CULL_MODE_BACK`. This matches the canonical VR pattern used |
| 174 | +by [Tutorial28_HelloOpenXR](../Tutorial28_HelloOpenXR/readme.md). |
| 175 | + |
0 commit comments