|
| 1 | +# 3DGS 4x4 Cognitive-Shader SoA Plan — ndarray |
| 2 | + |
| 3 | +## Goal |
| 4 | + |
| 5 | +Promote the 3DGS math carrier from a narrow 3x3-only view into a 4x4 cognitive-shader-friendly SoA carrier while preserving the correct 3DGS covariance spine. |
| 6 | + |
| 7 | +Important distinction: |
| 8 | + |
| 9 | +```text |
| 10 | +3x3 SPD covariance |
| 11 | + remains the mathematically correct spatial covariance for each 3D Gaussian |
| 12 | +
|
| 13 | +4x4 carrier |
| 14 | + becomes the homogeneous / temporal / semantic / shader-lane envelope around that covariance |
| 15 | +``` |
| 16 | + |
| 17 | +Do not destroy the 3x3 SPD invariant. Lift it into a 4x4 block where needed. |
| 18 | + |
| 19 | +## Why 4x4 |
| 20 | + |
| 21 | +The 4x4 shape maps better to: |
| 22 | + |
| 23 | +```text |
| 24 | +homogeneous transforms |
| 25 | +SIMD lane groups |
| 26 | +cognitive-shader-driver BindSpace columns |
| 27 | +Mat4 camera/world transforms |
| 28 | +quaternion / covariance / opacity packing |
| 29 | +(4x4)^4 tensor-block fanout |
| 30 | +``` |
| 31 | + |
| 32 | +It also gives a natural bridge between rendering, registration, ultrasound frame fusion, genetics-like transition matrices, and neuronal adjacency blocks. |
| 33 | + |
| 34 | +## Core representation |
| 35 | + |
| 36 | +Add or stabilize a Mat4 carrier under the existing linalg path: |
| 37 | + |
| 38 | +```text |
| 39 | +src/hpc/linalg/ |
| 40 | + mat4.rs |
| 41 | + spd4.rs |
| 42 | + block4.rs |
| 43 | +``` |
| 44 | + |
| 45 | +Candidate types: |
| 46 | + |
| 47 | +```rust |
| 48 | +pub struct Mat4x4<T> { |
| 49 | + pub m: [[T; 4]; 4], |
| 50 | +} |
| 51 | + |
| 52 | +pub struct Sym4<T> { |
| 53 | + pub a00: T, pub a01: T, pub a02: T, pub a03: T, |
| 54 | + pub a11: T, pub a12: T, pub a13: T, |
| 55 | + pub a22: T, pub a23: T, |
| 56 | + pub a33: T, |
| 57 | +} |
| 58 | + |
| 59 | +pub struct Splat4Carrier<T> { |
| 60 | + pub spatial_sigma_3x3: [T; 6], |
| 61 | + pub lane4: [T; 4], |
| 62 | + pub transform4x4: Mat4x4<T>, |
| 63 | +} |
| 64 | +``` |
| 65 | + |
| 66 | +## 3x3 to 4x4 lift |
| 67 | + |
| 68 | +Use explicit lifts instead of pretending 3x3 and 4x4 mean the same thing. |
| 69 | + |
| 70 | +```text |
| 71 | +3DGS spatial covariance Sigma3 |
| 72 | + -> |
| 73 | +4x4 homogeneous carrier Sigma4 |
| 74 | + -> |
| 75 | +projection / temporal / semantic / shader lane operations |
| 76 | + -> |
| 77 | +extract spatial 3x3 or image-space 2x2 when needed |
| 78 | +``` |
| 79 | + |
| 80 | +Example lift: |
| 81 | + |
| 82 | +```rust |
| 83 | +pub fn lift_spd3_to_sym4(sigma3: [f32; 6], w_lane: [f32; 4]) -> Sym4<f32>; |
| 84 | +pub fn extract_spd3_from_sym4(sym4: Sym4<f32>) -> [f32; 6]; |
| 85 | +``` |
| 86 | + |
| 87 | +## Cognitive-shader SoA layout |
| 88 | + |
| 89 | +Prefer 4-lane grouped SoA columns: |
| 90 | + |
| 91 | +```text |
| 92 | +BindSpace4 |
| 93 | + lane0: position / nucleotide / neuron source / feature id shard |
| 94 | + lane1: covariance / transition / edge weight / local statistic |
| 95 | + lane2: opacity / expression / activation / confidence |
| 96 | + lane3: time / phase / semantic role / provenance |
| 97 | +``` |
| 98 | + |
| 99 | +Concrete splat columns: |
| 100 | + |
| 101 | +```text |
| 102 | +pos_xyzw[] |
| 103 | +scale_xyz_opacity[] |
| 104 | +quat_xyzw[] |
| 105 | +color_rgba[] |
| 106 | +feature_id_time[] |
| 107 | +certificate_confidence[] |
| 108 | +``` |
| 109 | + |
| 110 | +## (4x4)^4 tensor-block fanout |
| 111 | + |
| 112 | +Interpret `(4x4)^4` as a four-level block grammar, not as one giant dense matrix. |
| 113 | + |
| 114 | +```text |
| 115 | +level 0: Mat4 local carrier |
| 116 | +level 1: 4x4 block of carriers |
| 117 | +level 2: 4x4 block of blocks |
| 118 | +level 3: 4x4 graph/tile/neural field super-block |
| 119 | +``` |
| 120 | + |
| 121 | +This gives hierarchical locality: |
| 122 | + |
| 123 | +```text |
| 124 | +splat -> block -> tile -> region -> graph domain |
| 125 | +``` |
| 126 | + |
| 127 | +## Domain-neutral kernels |
| 128 | + |
| 129 | +Create kernels that do not care whether the 4x4 block is geospatial, ultrasound, genetic, or neuronal. |
| 130 | + |
| 131 | +```rust |
| 132 | +pub trait Block4Kernel<T> { |
| 133 | + fn score_block(&self, block: &Block4<T>) -> f32; |
| 134 | + fn contract(&self, parent: &Block4<T>, child: &Block4<T>) -> f32; |
| 135 | + fn certify(&self, block: &Block4<T>) -> Block4Certificate; |
| 136 | +} |
| 137 | +``` |
| 138 | + |
| 139 | +## 3DGS compatibility rules |
| 140 | + |
| 141 | +- Keep `Spd3` as the authoritative spatial covariance. |
| 142 | +- Use `Mat4x4` for transforms and carriers. |
| 143 | +- Use `Sym4` only when the extra lane has defined meaning. |
| 144 | +- Image-space EWA still extracts the proper spatial covariance path. |
| 145 | +- Pillar tests must assert that 4x4 lifts do not break 3x3 invariants. |
| 146 | + |
| 147 | +## Pillar additions |
| 148 | + |
| 149 | +Add substrate probes: |
| 150 | + |
| 151 | +```text |
| 152 | +Pillar-18: 3x3-to-4x4 lift preserves spatial SPD invariants |
| 153 | +Pillar-19: Block4 contraction under HHTL hierarchy |
| 154 | +Pillar-20: 4-lane SoA equivalence against scalar AoS reference |
| 155 | +``` |
| 156 | + |
| 157 | +## Acceptance criteria |
| 158 | + |
| 159 | +- 4x4 carrier compiles behind `linalg` or a specific feature gate. |
| 160 | +- `splat3d` can use 4x4 transforms without losing 3x3 covariance correctness. |
| 161 | +- SoA 4-lane layout has scalar reference tests. |
| 162 | +- HHTL cascade can score 4x4 blocks. |
| 163 | +- Pillar tests prove lift/extract invariants. |
| 164 | + |
| 165 | +## Cross-repo link |
| 166 | + |
| 167 | +`lance-graph` should use this as the numeric substrate for the 4x4 cross-domain fanout plans. |
0 commit comments