feat: improve pointcloud logging

[ypang-neuraco]

Features

• Update pointcloud logging from base64 json encoding to storing in binary (metadata still in json)
• Update synced_recording to cache the binary files

Benchmark against old encoding

Case	Points	RGB	Legacy encode	New encode	Encode speedup	Legacy decode	New decode	Decode speedup	Legacy frame size	New frame size	Frame size ratio	Legacy disk (20 frames)	New disk (20 frames)	Disk size ratio
small_xyz	4,096	No	0.100 ms	0.011 ms	9.2x	0.076 ms	0.016 ms	4.6x	33.0 KB	24.9 KB	1.3x	660.0 KB	501.8 KB	1.3x
medium_xyz_rgb	65,536	Yes	2.130 ms	0.028 ms	77.0x	1.549 ms	0.057 ms	27.2x	786.7 KB	590.1 KB	1.3x	15.73 MB	11.81 MB	1.3x
large_xyz_rgb	307,200	Yes	11.897 ms	0.292 ms	40.7x	7.915 ms	0.575 ms	13.8x	3.69 MB	2.77 MB	1.3x	73.73 MB	55.31 MB	1.3x

Items

• NCRL-639

Related PRs

• feat: add utils for pointcloud logging neuracore_types#95

[github-actions]

Consider updating changelogs/pending-changelog.md with a summary of this change for the release notes. This is optional and non-blocking.

[ypang-neuraco]

@aditya-neuraco I did a benchmark on with and without zstd, the file size only shrunk by ~10% but the encode/decode time increased a lot. I don't think it is worth it at this stage. Maybe we can find a better compression method down the line.

RealSense results on a single frame (307K pts + RGB)
Disk: 2.77 MB → 2.43–2.56 MB (~1.1x at L3, ~1.14x at L6)
Encode: 0.14 ms (v1) → 5.2 ms (zstd L3)
Decode: 0.41 ms (v1) → 2.2 ms (zstd)

[aditya-neuraco]

@aditya-neuraco I did a benchmark on with and without zstd, the file size only shrunk by ~10% but the encode/decode time increased a lot. I don't think it is worth it at this stage. Maybe we can find a better compression method down the line.

RealSense results on a single frame (307K pts + RGB)

Disk: 2.77 MB → 2.43–2.56 MB (~1.1x at L3, ~1.14x at L6)

Encode: 0.14 ms (v1) → 5.2 ms (zstd L3)

Decode: 0.41 ms (v1) → 2.2 ms (zstd)

Agreed, doesn't make sense to add it.

[aditya-neuraco]

LGTM w / comments