Merge pull request #18 from Uralstech/unstable

UXR.QuestCamrea v4.2.1

Author: Uralstech

Documentation/DocSource/QuickStart.md

@@ -1,7 +1,7 @@
 # Quick Start
 
 The example code provided in this quick start guide is for educational and demonstration purposes only. It may not represent best practices for production use.
-This quick start was last updated for **UXR.QuestCamera v4.1.0**.
+This quick start was last updated for **UXR.QuestCamera v4.2.1**.
 
 ## Breaking Changes Notice
 
@@ -246,7 +246,7 @@ Note that an awaiter of `TakePicture` also has to wait for the camera device and
 
 ### Vulkan-specific Capture Sessions (Experimental)
 
-If your app uses the Vulkan Graphics API and targets API Level 33 (Android 13) or higher, you can use
+If your app uses the Vulkan Graphics API and Linear color space, and targets API Level 33 (Android 13) or higher, you can use
 `VulkanContinuousCaptureSession` and `VulkanOnDemandCaptureSession`, in the `Uralstech.UXR.QuestCamera.Vulkan` namespace,
 instead of `ContinuousCaptureSession` and `OnDemandCaptureSession`. It can improve memory usage and
 performance as it uses a native rendering plugin to perform YUV-to-RGBA conversion
@@ -456,10 +456,11 @@ This sample also uses the old input system. There is no code that references it,
 
 ### YUV Conversion Accuracy
 
-OpenGL's `EXT_YUV_target`[<sup>[1]</sup>](#yuvconversionaccuracy_ref1) and Vulkan's `VK_KHR_sampler_ycbcr_conversion`[<sup>[2]</sup>](#yuvconversionaccuracy_ref2) may produce non-linear R'G'B' values when sampling YUV images. This package applies only rudimentary transfer-function corrections to convert these values to linear RGB, which may result in inaccurate colors.
-
-Both the compute-shader-based YUV converter and the OpenGL converter are potentially inaccurate because they currently assume
-full-range BT.601 YUV data and do not account for the actual color range or standard of the source image.
+OpenGL's `EXT_YUV_target`[<sup>[1]</sup>](#yuvconversionaccuracy_ref1) and Vulkan's `VK_KHR_sampler_ycbcr_conversion`[<sup>[2]</sup>](#yuvconversionaccuracy_ref2) may produce non-linear R'G'B'
+values when sampling YUV images. Starting with UXR.QuestCamera v4.2.1, the package's Vulkan capture sessions use the SMPTE 170M[<sup>[3]</sup>](#yuvconversionaccuracy_ref3) inverse transfer
+function to convert sampled colors to linear RGB. The included compute-shader-based converter and OpenGL session currently assume full-range BT.601 YUV data and treat sampled colors as
+if they are in gamma space.
 
 <a id="yuvconversionaccuracy_ref1"></a> <sup>[1]</sup> <https://registry.khronos.org/OpenGL/extensions/EXT/EXT_YUV_target.txt> (Issues, 9. Should sampling return samples converted in to linear space or raw samples?)<br/>
-<a id="yuvconversionaccuracy_ref2"></a> <sup>[2]</sup> <https://github.com/KhronosGroup/Vulkan-Docs/issues/2356>
+<a id="yuvconversionaccuracy_ref2"></a> <sup>[2]</sup> <https://github.com/KhronosGroup/Vulkan-Docs/issues/2356><br/>
+<a id="yuvconversionaccuracy_ref3"></a> <sup>[3]</sup> <https://developer.android.com/reference/android/hardware/DataSpace#TRANSFER_SMPTE_170M>

Documentation/DocSource/V3Migration.md

@@ -7,6 +7,4 @@ items:
 - name: Quick Start
   href: QuickStart.md
 - name: Advanced Samples
-  href: AdvancedSamples.md
-- name: V3 Migration (Archived)
-  href: V3Migration.md
+  href: AdvancedSamples.md

Documentation/DocSource/toc.yml

@@ -5,10 +5,26 @@ layout(set = 0, binding = 0) uniform sampler2D src;
 layout(location = 0) in vec2 uv;
 layout(location = 0) out vec4 outColor;
 
+// Inverse transfer function for SMPTE 170M, formulae from:
+// https://www.kernel.org/doc/Documentation/media/uapi/v4l/colorspaces-details.rst
+// Adjusted for 0-1 input range.
+
+const float inverseExponent = 1.0 / 0.45;
+
+float smpte170mInverseTransfer(float val) {
+
+    return val >= 0.081
+        ? pow((val + 0.099) / 1.099, inverseExponent)
+        : val / 4.5;
+}
+
 void main() {
     vec4 color = texture(src, uv);
 
-    // TODO: implement conversions from
-    // https://developer.android.com/reference/android/hardware/DataSpace#TRANSFER_GAMMA2_2
-    outColor = vec4(pow(color.rgb, vec3(2.2)), color.a);
+    outColor = vec4(
+        smpte170mInverseTransfer(color.r),
+        smpte170mInverseTransfer(color.g),
+        smpte170mInverseTransfer(color.b),
+        1.0
+    );
 }

THIRD-PARTY-LICENSES.md

@@ -42,6 +42,7 @@ public sealed class Proxy : ProxyBase { }
         public bool HasNewFrame => _lastUpdateFrame == Time.frameCount;
 
         /// <summary>The output texture with converted frames.</summary>
+        // TODO: Change to RenderTexture and update conversion to use same transfer as the Vulkan shader.
         public readonly Texture2D Texture;
 
         /// <summary>The capture timestamp of the last processed frame.</summary>

UCamera/VulkanPlugin/Shaders/Render.frag

@@ -3,7 +3,7 @@
   "displayName": "UXR.QuestCamera",
   "description": "A Unity package to use the new Meta Quest Passthrough Camera API.\n\nUXR.QuestCamera v4.0.0 is a rewrite of the package with lots of breaking changes. Please see the documentation, reference manual and updated sample for changes.",
   "keywords": [],
-  "version": "4.2.0",
+  "version": "4.2.1",
   "unity": "2022.3",
   "hideInEditor": false,
   "documentationUrl": "https://uralstech.github.io/UXR.QuestCamera/",