Quality of scene level consistency

[physics-constrained-Real2Sim]

Hi, thank you for open-sourcing the great work. I really enjoyed reading it.

I have a question regarding scene-level consistency under occlusion. In the demo figures in your paper, many objects appear to be cleanly visible. However, in real-world scenarios, occlusions are quite common. For instance, in the example with the chair heavily occluded by the toy bear, it is not entirely clear how well the method handles such cases. (picture is from SAM3D)

I was wondering whether this limitation might be related to the underlying assumptions in methods like VGGT and TRELLIS, since they do not explicitly address amodal segmentation (i.e., reasoning about the full extent of partially occluded objects).

I’m not raising this as criticism. Instead, I’m genuinely interested in understanding the current best practices. Specifically:

1. What are some recommended approaches for multi-view 3D reconstruction with strong scene-level consistency under occlusion?
2. Do you think recent works like multi-view SAM3D (e.g., the arXiv version) paper move in this direction, or is this still an open challenge?

I would really appreciate any insights or pointers you could share. Thanks again for your great work!

[Jiahao620]

Sorry I've been a bit busy lately, which has caused the delay in answering questions.

1. For cases with severe occlusion, I believe a strategy similar to ReconViaGen should be used to extend SAM3D to a sparse multi-view setting. We are currently developing this, so please stay tuned.
2. Multi-view SAM3D is a great work, but that method is training-free, so the results may not be ideal.