TriSplat: Simulation-Ready Feed-Forward 3D Scene Reconstruction

Weijie Wang^1,*   Zimu Li^1,*   Jinchuan Shi¹   Zeyu Zhang¹   Botao Ye^2,3
Marc Pollefeys^2,4   Donny Y. Chen⁵   Bohan Zhuang¹  

¹Zhejiang University     ²ETH Zurich     ³ETH AI Center     ⁴Microsoft     ⁵Monash University

TriSplat is a feed-forward 3D reconstruction model that predicts simulation-ready triangle meshes from sparse, unposed images. Unlike Gaussian-splatting pipelines that require post-hoc mesh extraction, TriSplat directly predicts oriented triangle primitives, camera poses, point maps, and appearance attributes in one forward pass. We train on RealEstate10K and DL3DV, and evaluate zero-shot generalization on ScanNet with RE10K-trained models.

Method

Given sparse input views, TriSplat predicts dense local point maps, triangle attributes, camera poses, and optional intrinsics. Point-map geometry anchors triangle orientation through geometry normals, a learned normal refiner, and a monocular-normal bootstrap. A differentiable triangle rasterizer renders RGB, depth, and normals, while mesh export only needs opacity filtering, winding correction, and duplicate-vertex merging.

Installation

Create the environment:

conda create -y -n trisplat python=3.10
conda activate trisplat
pip install --upgrade pip

Install PyTorch and Python dependencies:

pip install torch==2.1.2 torchvision==0.16.2 torchaudio==2.1.2 \
  --index-url https://download.pytorch.org/whl/cu118
pip install -r requirements.txt --no-build-isolation

Build CUDA extensions:

bash scripts/env/rebuild_extensions.sh

Download initialization weights used by the model:

mkdir -p pretrained_weights
wget -O pretrained_weights/pi3.safetensors \
  https://huggingface.co/yyfz233/Pi3/resolve/main/model.safetensors
wget -O pretrained_weights/omnidata_dpt_normal_v2.ckpt \
  'https://zenodo.org/records/10447888/files/omnidata_dpt_normal_v2.ckpt?download=1'

Models

Download released TriSplat checkpoints from lhmd/TriSplat:

mkdir -p checkpoints
wget -O checkpoints/re10k_trisplat.ckpt \
  https://huggingface.co/lhmd/TriSplat/resolve/main/re10k_trisplat.ckpt
wget -O checkpoints/dl3dv_trisplat.ckpt \
  https://huggingface.co/lhmd/TriSplat/resolve/main/dl3dv_trisplat.ckpt

Datasets

Packed .torch datasets default to:

data/re10k
data/dl3dv

You can also set:

export RE10K_ROOT="$PWD/data/re10k"
export DL3DV_ROOT="$PWD/data/dl3dv"

See data/README.md for dataset layout and conversion notes.

Training

Train on RealEstate10K:

bash scripts/train/train_re10k.sh --gpus 0,1,2,3,4,5,6,7 --wandb-mode offline

Train on DL3DV:

bash scripts/train/train_dl3dv.sh --gpus 0,1,2,3,4,5,6,7 --wandb-mode offline

Extra arguments after -- are passed to Hydra. Use --ckpt to resume or initialize from a checkpoint.

Evaluation

Evaluate and render RealEstate10K meshes:

bash scripts/eval/eval_re10k_mesh.sh \
  --ckpt checkpoints/re10k_trisplat.ckpt \
  --data-root "$RE10K_ROOT"

Evaluate and render DL3DV meshes:

bash scripts/eval/eval_dl3dv_mesh.sh \
  --ckpt checkpoints/dl3dv_trisplat.ckpt \
  --data-root "$DL3DV_ROOT"

Custom Image Inference

For raw custom images, use the plain torch inference script. It does not use the Lightning Trainer or dataset DataModule; it loads images from a folder, runs the pose-free encoder directly, and exports a direct triangle mesh plus predicted camera poses:

python -m src.scripts.infer_custom_mesh \
  --image-dir /path/to/images \
  --ckpt checkpoints/re10k_trisplat.ckpt \
  --out-dir outputs/custom_mesh/demo \
  --num-views 2 \
  --force

By default the script uses the lightweight RE10K 2-view experiment and selects views uniformly from the folder. Use --view-indices 0,5 to choose frames explicitly. If camera intrinsics are unavailable, the script creates a normalized pinhole camera from a 60 degree horizontal FOV; override this with --fov-deg or --intrinsics-json.

The main outputs are:

outputs/custom_mesh/demo/mesh/DIRECT_triangle_mesh.ply
outputs/custom_mesh/demo/mesh/DIRECT_triangle_mesh_post.ply
outputs/custom_mesh/demo/predicted_c2w.json
outputs/custom_mesh/demo/inference_summary.json

Pass extra Hydra options with repeated --override flags, for example --override mesh.tsdf_gs2d.direct_post_process=false to save only the raw direct mesh.

Simulation

TriSplat exports ordinary triangle meshes, so the output can be opened directly by common graphics and simulation tools. The evaluation scripts above write per-scene meshes under:

outputs/<eval_root>/<run_name>/<scene>/mesh/DIRECT_triangle_mesh.ply
outputs/<eval_root>/<run_name>/<scene>/mesh/DIRECT_triangle_mesh.off
outputs/<eval_root>/<run_name>/<scene>/mesh/DIRECT_triangle_mesh_post.ply
outputs/<eval_root>/<run_name>/<scene>/mesh/DIRECT_triangle_mesh_post.off

The _post mesh is the default rendering and simulation output. Direct triangle meshes use quantile geometry cleanup to remove non-finite, degenerate, very large, or distant triangle outliers before compacting referenced vertices. TSDF meshes still use connected-component cleanup. For example, after running scripts/eval/eval_re10k_mesh.sh, use:

ls outputs/re10k_mesh_eval/re10k_mesh_eval/*/mesh/DIRECT_triangle_mesh_post.ply

The exported _post.ply mesh is vertex-colored and can be imported into Blender, Open3D, Isaac Sim, Unity, or PyBullet as a static triangle mesh. For simulation, use the .ply mesh for visual geometry and generate a collision mesh in your simulator if needed; for example, simplify or convex-decompose it before rigid-body simulation when the raw mesh is too dense.

Citation

If you find this repository useful, please cite:

@article{wang2026trisplat,
  title={TriSplat: Simulation-Ready Feed-Forward 3D Scene Reconstruction},
  author={Wang, Weijie and Li, Zimu and Shi, Jinchuan and Zhang, Zeyu and Ye, Botao and Pollefeys, Marc and Chen, Donny Y. and Zhuang, Bohan},
  journal={arXiv preprint arXiv:2605.26115},
  year={2026}
}

Acknowledgements

This codebase builds on open-source work including YoNoSplat, MVSplat, pixelSplat, CroCo, DINOv2, Omnidata, 3D Gaussian Splatting, and Triangle Splatting.