Skip to content

cockpit/torso: real BodyParts3D anatomy as a gaussian splat (splat3d CPU + three.js orbit)#52

Merged
AdaWorldAPI merged 2 commits into
mainfrom
claude/fma-torso-splat
Jun 24, 2026
Merged

cockpit/torso: real BodyParts3D anatomy as a gaussian splat (splat3d CPU + three.js orbit)#52
AdaWorldAPI merged 2 commits into
mainfrom
claude/fma-torso-splat

Conversation

@AdaWorldAPI

@AdaWorldAPI AdaWorldAPI commented Jun 24, 2026

Copy link
Copy Markdown
Owner

What

Two new cockpit pages rendering a torso as a gaussian splat from real anatomical geometry — the follow-on to the /fma heart slice (#51), which had to synthesize its layout because FMA itself has zero geometry.

The fix is BodyParts3D (DBCLS): it realizes FMA concepts as 3D OBJ meshes in one shared whole-body frame and keys each mesh directly on its FMA concept id. So the torso needs no synthesized positions — these are real anatomical coordinates. Per the BodyParts3D mereotopology (NAR 2009, Table 1: A part_of B ⟺ A° ⊂ B°, coveredBy), the FMA part_of relation is the spatial containment of those meshes — so the FMA partonomy selects, nests, and colors the real geometry.

The two pages (one shared asset)

  • /torso — splat3d CPU render (the "splat"): a turntable of frames rendered bake-side by ndarray::hpc::splat3d (CPU-SIMD EWA, no GPU) over torso.splat. Drag to scrub, click to pause. Rendered under ndarray's own pinned 1.95 toolchain via a scratchpad driver (SPL1 → Gaussian3D → splat3d), so q2's workspace stays free of the ndarray dependency.
  • /torso-live — three.js live orbit (the "live orbit"): the same torso.splat, orbitable in WebGL.

Both read one baked asset, so geometry is identical across pages.

Data (measured)

  • Root FMA7181 trunk178 concepts → 577 OBJ meshes → ~694K verts (thoracic segment 548, body wall 81, abdominal 24, perineum 4). The heart (osint/fma: hydrate the bake from the real FMA heart subtree, not a fixture #51) nests inside it.
  • Baked to 231,320 gaussians, 102 structures each with its own muted-pastel hue (golden-angle, S=0.34 V=0.78).
  • cockpit/public/torso.splat (3.7 MB, SPL1 binary) + torso.manifest.json + 20 JPEG frames (~0.7 MB).

Pipeline

BodyParts3D (FMA partof tree + FJ OBJ meshes, shared frame)
  → tools/bake_torso_splat.py  (BFS FMA7181 → meshes → verts, deepest-owner colour,
                                recenter/normalize → SPL1)
  → cockpit/public/torso.splat
  → /torso-live  (three.js orbit, SPL1 decoder)
  → scratchpad torso-render → ndarray::hpc::splat3d → 20 frames → /torso  (turntable)

Licence / attribution

Geometry: BodyParts3D, © The Database Center for Life Science, CC-BY 4.0. Attribution is shown in-view on both pages (required). The 62 MB OBJ archive + ~1.4 MB mapping files are external (download URLs documented in the bake tool); only the derived splat + frames are committed.

Verification

  • Render correctness confirmed by viewing the rendered frames (upright torso, 102 structures in soft pastels, correct anatomical proportions).
  • tsc clean. Browser render of the live three.js page not exercised in this environment (the decoder is standard; geometry verified via the CPU frames).

🤖 Generated with Claude Code

https://claude.ai/code/session_01TzqvDqbFRzyx17EkLKBoZF


Generated by Claude Code

Summary by CodeRabbit

  • New Features
    • Added a new torso visualization experience with two views: an interactive live orbit view and a frame-based render view.
    • Added automatic playback plus drag-to-scrub controls for the torso frame viewer.
    • Added routing so the new torso views are accessible from the app.

claude added 2 commits June 24, 2026 06:20
FMA has no geometry, so the /fma heart slice (PR #51) synthesizes its layout
from the 8:8 HHTL address. This adds a TORSO rendered from REAL anatomical
meshes: BodyParts3D (DBCLS) keys 3D OBJ meshes directly on FMA concept IDs in
one shared whole-body frame, so there are no synthesized positions. Per the
BodyParts3D NAR 2009 mereotopology, the FMA part_of relation IS spatial
containment of those meshes (child coveredBy parent) — so the FMA partonomy
selects and colours the real geometry.

- tools/bake_torso_splat.py: BFS the FMA part_of subtree at FMA7181 (trunk) over
  BodyParts3D's partof tree -> 178 concepts -> 577 OBJ meshes -> ~694K verts.
  Colours each mesh by its DEEPEST owning concept (compound concepts list all
  descendant elements, so claim deepest-first) -> 102 structures, distinct hues.
  Recenter + normalize (max half-extent = 1) -> SPL1 gaussian asset (231K
  gaussians, deterministic stride-3 downsample).
- cockpit/public/torso.splat (3.7 MB) + torso.manifest.json (attribution + legend).
- cockpit/src/TorsoSplat.tsx (/torso-live): SPL1 decoder + three.js Points (soft
  gaussian sprite, per-structure colour, OrbitControls auto-rotate), oriented
  upright (BodyParts3D superior-inferior = model +Z -> world +Y).
- main.tsx: /torso-live route.

Source geometry: BodyParts3D 4.0 part-of OBJ (decimated 99%), CC-BY 4.0,
(c) The Database Center for Life Science. Attribution is shown in-view (required
by the licence). The 62 MB OBJ archive + the ~1.4 MB mapping txt files are
external (download URLs documented in the bake tool); only the derived splat is
committed.

This is the "live orbit" of the two requested torso pages; the CPU splat3d
render (/torso) via ndarray::hpc::splat3d is the next step. tsc clean; the
browser render was not exercised in this environment.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TzqvDqbFRzyx17EkLKBoZF
Adds the second torso page (/torso, the "splat") and softens the colours on
both pages per request.

- /torso (TorsoRender.tsx): a turntable of frames rendered bake-side by
  ndarray::hpc::splat3d (CPU-SIMD EWA, no GPU) over the same torso.splat the
  live page orbits. Drag to scrub, click to pause. A scratchpad torso-render
  driver reads SPL1 -> Gaussian3D -> splat3d turntable (no Inria .ply needed),
  under ndarray's own pinned 1.95 toolchain so q2's workspace stays free of the
  ndarray dependency. 20 JPEG frames (720x960) -> cockpit/public/torso-frames/.
- Muted palette: per-structure golden-angle hue at S=0.34 V=0.78 (was the neon
  S=0.62 V=0.96). torso.splat re-baked; the live three.js page (/torso-live)
  picks it up directly, the splat3d frames re-rendered.
- main.tsx: /torso route; TorsoSplat nav cross-links splat <-> live orbit.

Render verified by viewing the frames: real BodyParts3D torso anatomy, upright,
102 structures in soft pastels. tsc clean. Geometry: BodyParts3D, CC-BY 4.0,
(c) The Database Center for Life Science (attribution shown in-view).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TzqvDqbFRzyx17EkLKBoZF
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@coderabbitai

coderabbitai Bot commented Jun 24, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 9d64903b-eb2a-4ed4-b0be-f51f903d00b5

📥 Commits

Reviewing files that changed from the base of the PR and between 81a0d3d and f197cab.

⛔ Files ignored due to path filters (20)
  • cockpit/public/torso-frames/torso_000.jpg is excluded by !**/*.jpg
  • cockpit/public/torso-frames/torso_001.jpg is excluded by !**/*.jpg
  • cockpit/public/torso-frames/torso_002.jpg is excluded by !**/*.jpg
  • cockpit/public/torso-frames/torso_003.jpg is excluded by !**/*.jpg
  • cockpit/public/torso-frames/torso_004.jpg is excluded by !**/*.jpg
  • cockpit/public/torso-frames/torso_005.jpg is excluded by !**/*.jpg
  • cockpit/public/torso-frames/torso_006.jpg is excluded by !**/*.jpg
  • cockpit/public/torso-frames/torso_007.jpg is excluded by !**/*.jpg
  • cockpit/public/torso-frames/torso_008.jpg is excluded by !**/*.jpg
  • cockpit/public/torso-frames/torso_009.jpg is excluded by !**/*.jpg
  • cockpit/public/torso-frames/torso_010.jpg is excluded by !**/*.jpg
  • cockpit/public/torso-frames/torso_011.jpg is excluded by !**/*.jpg
  • cockpit/public/torso-frames/torso_012.jpg is excluded by !**/*.jpg
  • cockpit/public/torso-frames/torso_013.jpg is excluded by !**/*.jpg
  • cockpit/public/torso-frames/torso_014.jpg is excluded by !**/*.jpg
  • cockpit/public/torso-frames/torso_015.jpg is excluded by !**/*.jpg
  • cockpit/public/torso-frames/torso_016.jpg is excluded by !**/*.jpg
  • cockpit/public/torso-frames/torso_017.jpg is excluded by !**/*.jpg
  • cockpit/public/torso-frames/torso_018.jpg is excluded by !**/*.jpg
  • cockpit/public/torso-frames/torso_019.jpg is excluded by !**/*.jpg
📒 Files selected for processing (7)
  • claude-notes/plans/2026-06-24-fma-torso-bodyparts3d-splat.md
  • cockpit/public/torso.manifest.json
  • cockpit/public/torso.splat
  • cockpit/src/TorsoRender.tsx
  • cockpit/src/TorsoSplat.tsx
  • cockpit/src/main.tsx
  • crates/osint-bake/tools/bake_torso_splat.py

📝 Walkthrough

Walkthrough

Adds an end-to-end FMA torso Gaussian splat feature: a Python bake script ingests BodyParts3D OBJ meshes, emits a binary SPL1 asset and torso.manifest.json, and two new React routes serve the result—/torso-live as an interactive Three.js WebGL orbit viewer and /torso as a CPU-rendered turntable frame scrubber.

Changes

FMA Torso Gaussian Splat Pipeline

Layer / File(s) Summary
SPL1 bake script and asset generation
crates/osint-bake/tools/bake_torso_splat.py
Script loads BodyParts3D hierarchy via load_tree, traverses the FMA subtree with bfs, assigns pastel colors per concept via concept_color, reads OBJ vertices with read_obj_vertices, downsamples and normalizes coordinates in main, then serializes the SPL1 binary and writes torso.manifest.json with counts, bbox, and region mappings.
SPL1 binary format contract and committed manifest
cockpit/public/torso.manifest.json, cockpit/src/TorsoSplat.tsx (lines 1–84)
Committed manifest carries source/license/attribution, dataset stats, bbox, and region definitions. TorsoSplat.tsx defines the Spl1 typed interface, decodeSpl1 validates the magic and unpacks header plus per-gaussian float32 positions and uint8 colors, and gaussianSprite() generates the canvas-based radial-gradient point texture.
Three.js WebGL live orbit viewer
cockpit/src/TorsoSplat.tsx (lines 85–220)
mount() builds the Three.js scene with Points geometry from decoded typed arrays, OrbitControls with damping and auto-rotate, ResizeObserver resize handling, and full disposal on unmount. The TorsoSplat React component fetches and decodes /torso.splat and the manifest, mounts the WebGL view, and renders a status overlay with gaussian/mesh/concept counts and CC-BY attribution.
CPU-rendered turntable frame viewer
cockpit/src/TorsoRender.tsx
TorsoRender preloads all /torso-frames/torso_XXX.jpg images, fetches the manifest, auto-advances frames at ~95 ms intervals when playing, and supports pointer-drag scrubbing. Renders a fixed full-screen image with loading status, playback instructions, manifest metadata, and a navigation link to /torso-live.
Router wiring and plan document
cockpit/src/main.tsx, claude-notes/plans/2026-06-24-fma-torso-bodyparts3d-splat.md
main.tsx imports TorsoSplat and TorsoRender and registers /torso-live and /torso routes. The plan document records the FMA/BodyParts3D conceptual mapping, pipeline stages, licensing, scale figures, and a completion checklist.

Sequence Diagram

sequenceDiagram
  participant User
  participant ReactRouter
  participant TorsoSplat
  participant ServerAssets as Static Assets
  participant decodeSpl1
  participant ThreeJS as Three.js Scene

  User->>ReactRouter: navigate to /torso-live
  ReactRouter->>TorsoSplat: render TorsoSplat
  TorsoSplat->>ServerAssets: fetch /torso.splat
  TorsoSplat->>ServerAssets: fetch /torso.manifest.json
  ServerAssets-->>TorsoSplat: SPL1 ArrayBuffer
  ServerAssets-->>TorsoSplat: Manifest JSON
  TorsoSplat->>decodeSpl1: decode(ArrayBuffer)
  decodeSpl1-->>TorsoSplat: Spl1 { positions, colors, count }
  TorsoSplat->>ThreeJS: mount(container, splat)
  ThreeJS-->>User: WebGL orbit view with auto-rotate
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 A torso of gaussians, baked with care,
SPL1 bits floating through air,
Three.js orbits, frames on a reel,
FMA ontology mapped with zeal!
The rabbit hops through anatomies bright,
Every vertex a colorful sight. 🫁

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants