[SPZ] Regression Large Splats Now Fail to Load

[lukeyreyno]

Context

Previously, large spz were able to be decoded just fine. It seems like the cause may be due to the shipped WASM binary in @adobe/spz being limited to 2GB of memory growth.

• Though, blindly increasing this the max memory to 4GB (for 32 bit WASM) isn't quite full proof and will still fail on larger splats.

Depending on the priorities of this tool, it could be worth reverting changes and going with a non-WASM decode.

• These larger splats decode fine using the spz v4 changes from Support SPZ Version 4 #226.

The alternative I suppose is to wait for the https://github.com/nianticlabs/spz / https://www.npmjs.com/package/@adobe/spz to limit the memory use in WASM decoding. Regardless, I figured it would be good to have this on your radar.

• The spz WASM module could consider using chunking to send data to V8 memory and/or store the splat attributes in column to prevent JS arrays from gettings too large.
• @azadbal for vis
• Edit: [WASM] Stop materializing the full unpacked cloud in the WASM Heap nianticlabs/spz#86

Examples

Splat Transform

Before

splat-transform v2.0.4 (b40e8f7)
▸ [1/1] Input /Users/lucas/Assets/Models/RnD-360pipeline.spz
  ▸ Reading
    ▸ decoding [####################] 5.738s
  · 10.5M gaussians · 3 SH bands · 2.31GB
# Summary

**Row Count:** 10512453
...

After

splat-transform v2.1.0 (e1c8619)
▸ [1/1] Input /Users/lucas/Assets/Models/RnD-360pipeline.spz
  ▸ Reading
[SPZ] deserializePackedGaussians: version=3, numPoints=10512453, shDegree=3, fractionalBits=12, antialiased=0, hasExtensions=0
Aborted()
    ✗ RuntimeError: Aborted(). Build with -sASSERTIONS for more info.
    at abort (file:///Users/lucas/repo/splat-transform/dist/spz-R7y_wx37.js:1:1590)
    at __abort_js (file:///Users/lucas/repo/splat-transform/dist/spz-R7y_wx37.js:1:751700)
    at wasm://wasm/002c81ba:wasm-function[770]:0x6674f
    at wasm://wasm/002c81ba:wasm-function[62]:0x3f07
    at wasm://wasm/002c81ba:wasm-function[40]:0x1226
    at wasm://wasm/002c81ba:wasm-function[129]:0x8c0c
    at wasm://wasm/002c81ba:wasm-function[840]:0x9b779
    at wasm://wasm/002c81ba:wasm-function[843]:0x9d5d4
    at dynCall (file:///Users/lucas/repo/splat-transform/dist/spz-R7y_wx37.js:1:768922)
    at file:///Users/lucas/repo/splat-transform/dist/spz-R7y_wx37.js:1:769065
    failed in 5.901s
  failed in 5.901s
✗ failed in 5.903s  [peak 3098.44GB]

Super Splat

Before	After

[slimbuck]

Yeah that's a pity. Could you perhaps summarise the versions we'd ideally support for read and write?

I suppose opus could easily generate this code in JS.

[lukeyreyno]

Yeah that's a pity. Could you perhaps summarise the versions we'd ideally support for read and write?

I suppose opus could easily generate this code in JS.

Ideally, splat-transform would have the ability to read / write (version 2, 3, and 4) of the spz format. The reference library in the spz repo demonstrates this. Also, I'm sure there are plenty of older spz files floating around, and the lift to support v2/v3 is relatively low.
As for the spz export UI in supersplat, Niantic Spatial would want to promote adoption of spz v4. Granted, it's also understandable if spz writes for older versions are available during this "transition" phase for spz v4.

Happy to help with the TS code for spz, if needbe. Though, agreed opus should be able to adapt the reference spz library fairly well.