fix(sim): register mesh assets on runtime add_object(shape="mesh") injection#1003
Conversation
…jection The incremental scene-edit path (inject_object_into_scene -> SpecBuilder.add_object) never registered the mesh asset a mesh geom names, so add_object(shape="mesh", mesh_path=...) at runtime always failed to recompile - even for a valid mesh file - and reported the opaque "spec recompile refused". Only the full-scene build() registered meshes, in a separate pass. Register the mesh in the incremental path (leaving build() untouched to avoid a double-registration collision), roll the asset back with the body when a recompile is refused, and delete it on eject so the object name stays reusable across remove/re-add cycles. The facade now rejects shape="mesh" without a mesh_path up front with an actionable message instead of an opaque recompile failure. Adds SpecBuilder.remove_mesh and a regression suite (4/5 fail pre-fix).
yinsong1986
left a comment
There was a problem hiding this comment.
Summary
This PR fixes runtime mesh injection in the MuJoCo backend. Simulation.add_object(shape="mesh", mesh_path=...) previously always failed with the opaque spec recompile refused error because the incremental scene-edit path (inject_object_into_scene -> SpecBuilder.add_object) never registered the mesh_<name> asset that the geom references, while the full-scene SpecBuilder.build did. The fix registers the mesh via spec.add_mesh(...) before adding the body, rolls back both the body and the mesh asset on a refused recompile (so the scene stays compilable and the name reusable), deletes the asset in eject_body_from_scene for clean remove/re-add cycles, adds a fail-fast on shape="mesh" without mesh_path, and introduces SpecBuilder.remove_mesh. Description, code, and tests are consistent.
What's good
- Rollback symmetry is correct: the mesh asset is removed in both the
add_objectexception branch and the recompile-refused branch, andremove_meshis a safe no-op for primitives (guardsspec.mesh(name)withexcept (KeyError, ValueError)and aNonecheck). This satisfies the AGENTS.md #86 "Resource Cleanup on Partial Failure" contract. - All failure paths return structured error dicts rather than raising, and the fail-fast missing-
mesh_pathmessage is actionable and ASCII-only (no emoji, per project rule). build()is deliberately left untouched so the two paths never double-register the same mesh name.- No change to any success-path signature, return shape, wire format, or persisted schema, so there is no one-way-door surface here.
- Tests pin each behavioral guarantee (valid add, fail-fast, bad-mesh rollback + name reuse, remove/re-add, no-brick-after-bad-mesh), and 4/5 fail on pre-fix code.
Verification suggestions
Standard CI (ruff, mypy, the sim suites under MUJOCO_GL=egl) is sufficient. To spot-check the rollback guarantee directly: run tests/simulation/mujoco/test_runtime_mesh_injection.py::TestRuntimeMeshInjection::test_bad_mesh_rolls_back_asset_and_name_is_reusable under MUJOCO_GL=egl and confirm mesh_widget is absent from spec.meshes after the failed add.
Problem
Simulation.add_object(shape="mesh", mesh_path=...)always failed at runtime, even with a valid mesh file, reporting the opaque errorFailed to inject '<name>': spec recompile refused.A mesh geom names a mesh asset (
mesh_<name>) that must be registered on theMjSpecbefore the geom can compile.SpecBuilder.build(full-scene build) registers those assets in a dedicated pass, but the incremental scene-edit path (inject_object_into_scene->SpecBuilder.add_object) never did. So the geom referenced an asset that did not exist and every recompile was refused. Runtime mesh injection was effectively dead despitemesh_pathbeing a documented, supportedadd_objectparameter.Change
inject_object_into_scenenow registers the mesh asset (spec.add_mesh(name=f"mesh_{obj.name}", file=...)) before adding the body.build()is left untouched so the two paths do not double-register the same mesh name.eject_body_from_scenedeletes themesh_<name>asset too, so remove/re-add cycles under the same name work.add_objectnow rejectsshape="mesh"without amesh_pathup front with an actionable message (add_object: shape='mesh' requires mesh_path ...) instead of letting the recompile fail opaquely.SpecBuilder.remove_meshhelper.No change to any success-path signature, return shape, wire format, or persisted schema.
Tests
tests/simulation/mujoco/test_runtime_mesh_injection.py(5 tests): valid mesh add succeeds and registers the asset; missingmesh_pathfails fast; an unloadable mesh rolls back and the name is reusable; remove-then-re-add under the same name works; a failed mesh add does not brick later primitive adds. 4 of the 5 fail on pre-fix code (the 5th is a guardrail that legitimately stayed green). Full local gate is clean:ruff check,ruff format --check,mypy strands_robots tests tests_integ(no issues in 646 files), and the affected sim suites (183 tests) pass underMUJOCO_GL=egl.Visual
Three mesh objects (plus a primitive box) injected into a live scene at runtime with an SO-100 arm, rendered headless in MuJoCo (
MUJOCO_GL=egl). This scene could not be constructed before the fix.Docs
docs/simulation/world-building.mdgains a "Mesh objects" section documenting runtime mesh injection, themesh_pathrequirement, and the rollback guarantee; theadd_objectdocstring error list is updated.