Skip to content

Add 11 RoboLab task specs reusing existing scenes#890

Open
qianl-nv wants to merge 4 commits into
mainfrom
qianl/dev/robolab_batch3
Open

Add 11 RoboLab task specs reusing existing scenes#890
qianl-nv wants to merge 4 commits into
mainfrom
qianl/dev/robolab_batch3

Conversation

@qianl-nv

@qianl-nv qianl-nv commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

Add 11 RoboLab task specs reusing existing scenes

Detailed description

  • Add 7 atomic pick-and-place task specs (canned_food_in_bin, food_packing_1_cans, marker_in_mug, mustard_in_right_bin, small_pumpkin_in_bin, smaller_object_butter_in_bin, smartphone_in_bin) on top of already-split scene YAMLs.
  • Add 4 multi-subtask task specs: parallel (rubiks_cube_and_banana, hammers_in_left_bin) and sequential (rubiks_cube_then_banana, food_packing_boxes_then_cans).
  • Fill the Arena task-spec column for these tasks in task_catalog.md and drop the redundant RoboLab prompt text from those rows.

List of new scenes:

  • canned_food_in_bin (verified)
  • food_packing_1_cans
  • food_backing_boxes_then_cans
  • hammers_in_left_bin
  • marker_in_mug
  • mustard_in_right_bin
  • rubiks_cube_and_banana
  • rubiks_cube_then_banan
  • small_pumpkin_in_bin
  • smaller_object_butter_in_bin
  • smartphone_in_bin

qianl-nv added 3 commits July 9, 2026 22:42
- Add recursive `yaml:` include merging to env-graph spec loading so a task
  spec can pull in a shared scene spec (deep dict merge, cycle detection)
- Split each robolab/*_linked.yaml into robolab/scenes/<scene>.yaml (scene
  only) and robolab/tasks/<task>.yaml (task overlay including its scene)
- Drop the `_linked` suffix and use RoboLab task snake_case names for tasks
- Update task_catalog.md, agentic env-gen docs, and the OSMO submit script to
  the new tasks/ layout; add yaml-include loader tests

Signed-off-by: Qian Lin <qianl@nvidia.com>
- Add 8 task YAMLs reusing existing robolab/scenes/ specs via `yaml:` include
  (butter, mustard, tomato can, canned tuna, smartphone, marker, small pumpkin,
  rubiks cube), covering RoboLab tasks that had a matching scene but no spec
- List each new task in task_catalog.md with its task and scene YAML links

Signed-off-by: Qian Lin <qianl@nvidia.com>
- Add rubiks_cube_and_banana and rubiks_cube_then_banana on the rubiks_cube_banana_bowl scene.
- Add hammers_in_left_bin on the tools_container scene and food_packing_boxes_then_cans on the foodpacking_1bin_1box_1can scene.
- Fill in the Arena task-spec column for these tasks in task_catalog.md.

Signed-off-by: Qian Lin <qianl@nvidia.com>
@qianl-nv qianl-nv changed the base branch from main to qianl/dev/spec_split July 10, 2026 12:37
Remove the max_separation override from marker_in_mug,
rubiks_cube_and_banana, and rubiks_cube_then_banana so each new task
matches the existing task sharing its scene (mouse_on_keyboard and
banana_then_rubiks_cube), which rely on the PickAndPlaceTask default.

Signed-off-by: Qian Lin <qianl@nvidia.com>
@qianl-nv qianl-nv marked this pull request as ready for review July 10, 2026 13:54
pick_up_object: marker
destination_location: ceramic_mug
background_scene: maple_table_robolab
episode_length_s: 20.0

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 marker_in_mug places the marker in the ceramic_mug (a container), yet it's the only in-container task in this batch that omits max_separation. Without it, success collapses to the object_on_destination contact check, which fires on any low-velocity contact with the mug — including the marker merely resting against the outside — so success can be over-counted (this is exactly the tighter-container-placement gap the objects_in_proximity TODO in pick_and_place_task.py calls out). The commit matched this to mouse_on_keyboard, but that peer is a surface placement (mouse on keyboard), not a container one; every other in-bin/in-container task in this PR sets [0.1, 0.1, 0.1]. Intended, or should this match the bin tasks?

Suggested change
episode_length_s: 20.0
episode_length_s: 20.0
max_separation: [0.1, 0.1, 0.1]

@arena-review-bot

Copy link
Copy Markdown
Contributor

🤖 Isaac Lab-Arena Review Bot

Summary

Purely additive, data-only PR adding 11 declarative RoboLab task specs (7 atomic pick-and-place, 2 parallel, 2 sequential) that reuse already-split scene YAMLs via external_yamls, plus filling the Arena task-spec column in task_catalog.md. I cross-checked every pick_up_object / destination_location against its referenced scene: all resolve to real object ids, the left/right-bin mappings are correct (container_f24 = left / bin_b04 = right; grey_bin_left / grey_bin_right), compositions match the descriptions, small_pumpkin/hammers match their existing scene peers, and the new files carry the correct single-year 2026 copyright. This lines up well with the intended scene-reuse composition pattern.

Findings

🟡 Warning: tasks/marker_in_mug.yaml:18 — Only in-container task in the batch that omits max_separation. Success then reduces to the object_on_destination contact check, which can fire on the marker merely touching the mug's exterior — looser than every other in-bin task here ([0.1, 0.1, 0.1]) and the exact case the objects_in_proximity TODO flags. The commit matched mouse_on_keyboard, but that peer is a surface placement, not a container one. Raised as a question — see the inline comment. (The rubiks_cube_*bowl tasks also omit it, but there it is consistent with the existing banana_then_rubiks_cube bowl peer, so no concern.)

Test Coverage

No per-task test is expected — the repo validates task-YAML loading with a single representative case (test_robolab_task_yaml_loads_scene_includebanana_in_bowl.yaml) rather than parametrizing over every spec, and these files follow that convention. CI (pre-commit, tests, GR00T E2E) is green.

Verdict

Minor fixes needed — resolve the marker_in_mug max_separation question; everything else is ship-ready.

@greptile-apps

greptile-apps Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds 11 new RoboLab task spec YAML files covering 7 atomic pick-and-place tasks and 4 multi-subtask tasks (2 parallel, 2 sequential), and updates task_catalog.md to fill in previously-blank Arena task-spec columns for those rows.

  • All new task YAMLs reference pre-existing scene YAMLs via external_yamls, and every pick_up_object and destination_location identifier was verified against its respective scene definition.
  • The max_separation convention is followed correctly: present for bin/container destinations, omitted for bowl/mug destinations, consistent with the existing codebase pattern.
  • The two new parallel composition tasks (rubiks_cube_and_banana, hammers_in_left_bin) introduce the first use of that composition type in the task directory.

Confidence Score: 5/5

Safe to merge — all object and destination IDs were cross-checked against their scene YAMLs and every reference resolves correctly.

All 11 new task specs reuse well-established scene definitions without modifying them, and every pick_up_object / destination_location pair was verified to exist in the referenced scene YAML. The max_separation convention is applied correctly across all new files, and the two new parallel-composition tasks follow the same structure as existing sequential ones. The only note is a minor phrasing inconsistency in smaller_object_butter_in_bin.yaml with no functional impact.

smaller_object_butter_in_bin.yaml has a minor description phrasing inconsistency (trailing period, "butter box" vs object id butter); all other files look clean.

Important Files Changed

Filename Overview
isaaclab_arena_environments/robolab/task_catalog.md Fills in previously-blank Arena task-spec columns for 11 rows and adds one new FoodPackingBoxesThenCansTask row; all links and task names match the newly added YAML files.
isaaclab_arena_environments/robolab/tasks/smaller_object_butter_in_bin.yaml Atomic pick-and-place task; description phrasing ("butter box") differs from the actual pick_up_object id (butter), and the description ends with a trailing period unlike every other task YAML in this PR.
isaaclab_arena_environments/robolab/tasks/hammers_in_left_bin.yaml Parallel two-subtask spec placing hammer_1 and hammer_2 into container_f24 (the left bin); all ids verified correct in tools_container.yaml.
isaaclab_arena_environments/robolab/tasks/rubiks_cube_and_banana.yaml Parallel composition placing cube and banana into bowl; no max_separation, consistent with the existing rubiks_cube.yaml and banana_then_rubiks_cube.yaml convention for bowl destinations.
isaaclab_arena_environments/robolab/tasks/food_packing_boxes_then_cans.yaml Sequential two-subtask spec packing cheez_it_box then tomato_soup_can into bin_a06; both objects and destination verified in scene YAML.
isaaclab_arena_environments/robolab/tasks/rubiks_cube_then_banana.yaml Sequential counterpart to existing banana_then_rubiks_cube.yaml; description format and structure match established patterns.
isaaclab_arena_environments/robolab/tasks/canned_food_in_bin.yaml Atomic pick-and-place task placing canned_tuna into grey_bin; both ids verified present in bin_condiments.yaml scene.
isaaclab_arena_environments/robolab/tasks/food_packing_1_cans.yaml Atomic task placing tomato_soup_can into bin_a06; mirrors the structure of the existing food_packing_1_boxes.yaml correctly.
isaaclab_arena_environments/robolab/tasks/marker_in_mug.yaml Atomic task placing marker into ceramic_mug; both objects verified in workdesk.yaml, no max_separation consistent with mug/bowl destination convention.
isaaclab_arena_environments/robolab/tasks/mustard_in_right_bin.yaml Atomic task; correctly uses grey_bin_right to mirror the existing mustard_in_left_bin.yaml which uses grey_bin_left.
isaaclab_arena_environments/robolab/tasks/small_pumpkin_in_bin.yaml Atomic task placing pumpkin_small into container_f24; matches the existing big_pumpkin_in_bin.yaml pattern for the same scene.
isaaclab_arena_environments/robolab/tasks/smartphone_in_bin.yaml Atomic task placing smartphone into grey_bin; both ids verified in workdesk_bin.yaml scene.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Scene YAML<br/>external_yamls] --> B{composition}
    B -->|atomic| C[Single PickAndPlaceTask]
    B -->|sequential| D[Ordered PickAndPlaceTask list<br/>task 1 → task 2]
    B -->|parallel| E[Unordered PickAndPlaceTask list<br/>task 1 ∥ task 2]

    C --> F[pick_up_object<br/>destination_location<br/>background_scene<br/>episode_length_s<br/>max_separation?]
    D --> F
    E --> F

    subgraph New atomic tasks
        G[canned_food_in_bin<br/>marker_in_mug<br/>mustard_in_right_bin<br/>food_packing_1_cans<br/>small_pumpkin_in_bin<br/>smaller_object_butter_in_bin<br/>smartphone_in_bin]
    end

    subgraph New parallel tasks
        H[rubiks_cube_and_banana<br/>hammers_in_left_bin]
    end

    subgraph New sequential tasks
        I[rubiks_cube_then_banana<br/>food_packing_boxes_then_cans]
    end

    B -->|atomic| G
    B -->|parallel| H
    B -->|sequential| I
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[Scene YAML<br/>external_yamls] --> B{composition}
    B -->|atomic| C[Single PickAndPlaceTask]
    B -->|sequential| D[Ordered PickAndPlaceTask list<br/>task 1 → task 2]
    B -->|parallel| E[Unordered PickAndPlaceTask list<br/>task 1 ∥ task 2]

    C --> F[pick_up_object<br/>destination_location<br/>background_scene<br/>episode_length_s<br/>max_separation?]
    D --> F
    E --> F

    subgraph New atomic tasks
        G[canned_food_in_bin<br/>marker_in_mug<br/>mustard_in_right_bin<br/>food_packing_1_cans<br/>small_pumpkin_in_bin<br/>smaller_object_butter_in_bin<br/>smartphone_in_bin]
    end

    subgraph New parallel tasks
        H[rubiks_cube_and_banana<br/>hammers_in_left_bin]
    end

    subgraph New sequential tasks
        I[rubiks_cube_then_banana<br/>food_packing_boxes_then_cans]
    end

    B -->|atomic| G
    B -->|parallel| H
    B -->|sequential| I
Loading

Reviews (1): Last reviewed commit: "Drop max_separation where scene peers om..." | Re-trigger Greptile

- ../scenes/butter_raisin_box_grey_bin.yaml
task:
composition: atomic
description: Pick up the butter box and place it into the grey bin on the maple table.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The description ends with a trailing period (unlike every other task YAML in this PR and the existing ones) and refers to the object as "butter box" while the pick_up_object id is butter. Keeping both consistent with the codebase phrasing avoids confusion.

Suggested change
description: Pick up the butter box and place it into the grey bin on the maple table.
description: pick up the butter and place it into the grey bin

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@qianl-nv qianl-nv force-pushed the qianl/dev/spec_split branch from 69978e0 to 2da34c6 Compare July 15, 2026 16:58
Base automatically changed from qianl/dev/spec_split to main July 16, 2026 00:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant