Skip to content

fix(sim): register mesh assets on runtime add_object(shape="mesh") injection#1003

Merged
cagataycali merged 1 commit into
strands-labs:mainfrom
cagataycali:fix/runtime-mesh-object-injection
Jul 2, 2026
Merged

fix(sim): register mesh assets on runtime add_object(shape="mesh") injection#1003
cagataycali merged 1 commit into
strands-labs:mainfrom
cagataycali:fix/runtime-mesh-object-injection

Conversation

@cagataycali

Copy link
Copy Markdown
Member

Problem

Simulation.add_object(shape="mesh", mesh_path=...) always failed at runtime, even with a valid mesh file, reporting the opaque error Failed to inject '<name>': spec recompile refused.

A mesh geom names a mesh asset (mesh_<name>) that must be registered on the MjSpec before the geom can compile. SpecBuilder.build (full-scene build) registers those assets in a dedicated pass, but the incremental scene-edit path (inject_object_into_scene -> SpecBuilder.add_object) never did. So the geom referenced an asset that did not exist and every recompile was refused. Runtime mesh injection was effectively dead despite mesh_path being a documented, supported add_object parameter.

Change

  • inject_object_into_scene now registers the mesh asset (spec.add_mesh(name=f"mesh_{obj.name}", file=...)) before adding the body. build() is left untouched so the two paths do not double-register the same mesh name.
  • On a refused recompile (e.g. an unloadable mesh file), the body and its mesh asset are rolled back, so the scene returns to its last compilable state and the object name stays reusable. A leaked asset would otherwise collide on a duplicate mesh name at the next recompile and brick further edits.
  • eject_body_from_scene deletes the mesh_<name> asset too, so remove/re-add cycles under the same name work.
  • The facade add_object now rejects shape="mesh" without a mesh_path up front with an actionable message (add_object: shape='mesh' requires mesh_path ...) instead of letting the recompile fail opaquely.
  • New SpecBuilder.remove_mesh helper.

No change to any success-path signature, return shape, wire format, or persisted schema.

Tests

tests/simulation/mujoco/test_runtime_mesh_injection.py (5 tests): valid mesh add succeeds and registers the asset; missing mesh_path fails fast; an unloadable mesh rolls back and the name is reusable; remove-then-re-add under the same name works; a failed mesh add does not brick later primitive adds. 4 of the 5 fail on pre-fix code (the 5th is a guardrail that legitimately stayed green). Full local gate is clean: ruff check, ruff format --check, mypy strands_robots tests tests_integ (no issues in 646 files), and the affected sim suites (183 tests) pass under MUJOCO_GL=egl.

Visual

Three mesh objects (plus a primitive box) injected into a live scene at runtime with an SO-100 arm, rendered headless in MuJoCo (MUJOCO_GL=egl). This scene could not be constructed before the fix.

runtime mesh injection

Docs

docs/simulation/world-building.md gains a "Mesh objects" section documenting runtime mesh injection, the mesh_path requirement, and the rollback guarantee; the add_object docstring error list is updated.

…jection

The incremental scene-edit path (inject_object_into_scene -> SpecBuilder.add_object)
never registered the mesh asset a mesh geom names, so add_object(shape="mesh",
mesh_path=...) at runtime always failed to recompile - even for a valid mesh file -
and reported the opaque "spec recompile refused". Only the full-scene build()
registered meshes, in a separate pass.

Register the mesh in the incremental path (leaving build() untouched to avoid a
double-registration collision), roll the asset back with the body when a recompile
is refused, and delete it on eject so the object name stays reusable across
remove/re-add cycles. The facade now rejects shape="mesh" without a mesh_path
up front with an actionable message instead of an opaque recompile failure.

Adds SpecBuilder.remove_mesh and a regression suite (4/5 fail pre-fix).

@yinsong1986 yinsong1986 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

This PR fixes runtime mesh injection in the MuJoCo backend. Simulation.add_object(shape="mesh", mesh_path=...) previously always failed with the opaque spec recompile refused error because the incremental scene-edit path (inject_object_into_scene -> SpecBuilder.add_object) never registered the mesh_<name> asset that the geom references, while the full-scene SpecBuilder.build did. The fix registers the mesh via spec.add_mesh(...) before adding the body, rolls back both the body and the mesh asset on a refused recompile (so the scene stays compilable and the name reusable), deletes the asset in eject_body_from_scene for clean remove/re-add cycles, adds a fail-fast on shape="mesh" without mesh_path, and introduces SpecBuilder.remove_mesh. Description, code, and tests are consistent.

What's good

  • Rollback symmetry is correct: the mesh asset is removed in both the add_object exception branch and the recompile-refused branch, and remove_mesh is a safe no-op for primitives (guards spec.mesh(name) with except (KeyError, ValueError) and a None check). This satisfies the AGENTS.md #86 "Resource Cleanup on Partial Failure" contract.
  • All failure paths return structured error dicts rather than raising, and the fail-fast missing-mesh_path message is actionable and ASCII-only (no emoji, per project rule).
  • build() is deliberately left untouched so the two paths never double-register the same mesh name.
  • No change to any success-path signature, return shape, wire format, or persisted schema, so there is no one-way-door surface here.
  • Tests pin each behavioral guarantee (valid add, fail-fast, bad-mesh rollback + name reuse, remove/re-add, no-brick-after-bad-mesh), and 4/5 fail on pre-fix code.

Verification suggestions

Standard CI (ruff, mypy, the sim suites under MUJOCO_GL=egl) is sufficient. To spot-check the rollback guarantee directly: run tests/simulation/mujoco/test_runtime_mesh_injection.py::TestRuntimeMeshInjection::test_bad_mesh_rolls_back_asset_and_name_is_reusable under MUJOCO_GL=egl and confirm mesh_widget is absent from spec.meshes after the failed add.

@cagataycali
cagataycali merged commit 0cfc801 into strands-labs:main Jul 2, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants