OMPE-88188: Add frame stacking support for explicit temporal info

[jmart-nv]

Description

This enables frame stacking for newton+warp by default in the cartpole camera presets task.

Newton Frame Stacking: Provides explicit temporal data to newton.

RTX uses DLSS anti-aliasing by default, which provides implicit temporal data. The newton_renderer does not provide temporal data. However, newton's energy-conserving physics solver requires temporal velocity data in order to compensate for the lack of damping, which causes convergence problems when paired with newton_renderer.

This commit provides explicit temporal information via 2-frame stacking by default for cartpole-camera when using newton+newton_renderer. This allows newton to provide the damping it needs to converge at the same rate as physx. This adds 36% GPU memory overhead, but the wall clock overhead is negligible.

The default for all other physics/renderer backends is still stack size = 1 (disabled) since physx has implicit damping built-in via its TGS solver, and RTX provides temporal data implicitly.

Implementation:

A new FrameStackBuffer helper class implements a ring buffer of the last N rendered frames which can supply explicit temporal observations to a policy. Tasks can then instantiate this buffer on startup and update it each frame.

Added new unit tests for the frame stack buffer and cartpole integration, and updated the documentation with a note about newton's dependency on temporal data.

Note: This is a task-local re-implementation of the closed PR #5232

Type of change

• Bug fix (non-breaking change which fixes an issue)

Checklist

• I have read and understood the contribution guidelines
• I have run the pre-commit checks with ./isaaclab.sh --format
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• I have updated the changelog and the corresponding version in the extension's config/extension.toml file
• I have added my name to the CONTRIBUTORS.md or my name already exists there

[greptile-apps]

Greptile Summary

This PR introduces a FrameStackBuffer ring buffer and a CartpoleCameraPresetsEnv subclass to provide explicit temporal observations for the Newton + Newton-Warp backend combination, where neither the physics solver nor the renderer supplies implicit temporal data.

• FrameStackBuffer maintains a (frame_stack, num_envs, H, W, C) history tensor and reconstructs the stacked output in-place via narrow + copy_, avoiding per-frame allocations; the init-on-reset path correctly fills all history slots with the first post-reset frame so the policy never sees zero-padded warm-up data.
• CartpoleCameraPresetsEnv wraps CartpoleCameraEnv, auto-detects the Newton+Warp combo via lazy isinstance checks, bumps observation_space before calling super().__init__(), and correctly clones the buffer's shared return value in _get_observations.
• The frame_stack=0 sentinel in the config (meaning "auto-detect") is documented in comments but not enforced by validation, making it easy to misinterpret 0 as "disabled" when setting via CLI overrides.

Confidence Score: 4/5

Safe to merge with minor follow-up; the core ring-buffer logic is correct and well-tested, and the env wiring works for the current usage pattern.

The ring-buffer arithmetic, init-on-reset behavior, and env integration are all verified by a thorough test suite. The main concerns are design-level: cfg.observation_space[-1] is mutated in-place before the parent constructor runs (safe only if configclass always deep-copies), update() returns shared internal storage that callers must clone (current code does so correctly), and the frame_stack=0 sentinel could be misread by users as disabled rather than auto-detect. None of these affect correctness in the current code paths, but they introduce fragility for future callers and config overrides.

cartpole_camera_presets_env.py and cartpole_camera_presets_env_cfg.py deserve a second look — the in-place cfg mutation and the sentinel value design are the most likely sources of hard-to-diagnose bugs if the config machinery or user overrides change.

Important Files Changed

Filename	Overview
source/isaaclab/isaaclab/envs/utils/frame_stack.py	New FrameStackBuffer ring buffer class; ring arithmetic and init-on-reset logic are correct and well-tested, but update() returns shared mutable internal storage that callers must clone.
source/isaaclab_tasks/isaaclab_tasks/direct/cartpole/cartpole_camera_presets_env.py	New CartpoleCameraPresetsEnv subclass wiring FrameStackBuffer; correctly clones the stacked output, but mutates cfg.observation_space in-place before super().init() and passes Sequence[int] where torch.Tensor is declared.
source/isaaclab_tasks/isaaclab_tasks/direct/cartpole/cartpole_camera_presets_env_cfg.py	Adds frame_stack: int = 0 sentinel to BaseCartpoleCameraEnvCfg; sentinel semantics (0 = auto, 1 = disabled) are unintuitive and not enforced by validation.
source/isaaclab/test/envs/test_frame_stack.py	Comprehensive pure-tensor unit tests covering ring ordering, partial env reset, dtype preservation, CUDA device, and long-run stability; no issues found.
source/isaaclab_tasks/test/test_cartpole_camera_presets_frame_stack.py	Integration tests covering the full physics×renderer truth table, obs-space arithmetic for all data-type presets, and end-to-end env construction; good coverage of the sentinel resolution logic.
source/isaaclab_tasks/isaaclab_tasks/direct/cartpole/init.py	Redirects Isaac-Cartpole-Camera-Presets-Direct-v0 entry point to the new CartpoleCameraPresetsEnv; change is minimal and correct.
source/isaaclab/isaaclab/envs/utils/init.py	Exports FrameStackBuffer from the utils package; no issues.
docs/source/overview/core-concepts/renderers.rst	Adds a documentation note explaining Newton Warp renderer's lack of temporal anti-aliasing and recommending task-local frame stacking; clear and accurate.

Sequence Diagram

sequenceDiagram
    participant Env as CartpoleCameraPresetsEnv
    participant Base as CartpoleCameraEnv
    participant Buf as FrameStackBuffer

    Note over Env: __init__
    Env->>Env: resolve_cfg_presets(cfg)
    Env->>Env: _resolve_frame_stack_default() → 1 or 2
    Env->>Env: "bump cfg.observation_space[-1] *= frame_stack"
    Env->>Base: super().__init__(cfg)
    Env->>Buf: FrameStackBuffer(single_shape, frame_stack, device)

    Note over Env: per-step loop
    Env->>Base: super()._get_observations()
    Base-->>Env: obs[policy] shape (N,H,W,C)
    Env->>Buf: update(obs[policy])
    Buf->>Buf: fill history slots for reset envs (init path)
    Buf->>Buf: copy current frame into _history[frame_idx]
    Buf->>Buf: rebuild _stacked via narrow+copy_
    Buf-->>Env: "_stacked (N,H,W,C*K) shared ref"
    Env->>Env: .clone() → obs[policy]

    Note over Env: on episode reset
    Env->>Base: super()._reset_idx(env_ids)
    Env->>Buf: reset(env_ids) marks envs for init
    Note over Buf: next update() fills all history slots with first post-reset frame

Loading

_{Reviews (1): Last reviewed commit: "OMPE-88188: Add frame stacking support f..." | Re-trigger Greptile}

[isaaclab-review-bot]

Code Review: Frame Stacking for Temporal Observations

This PR adds a well-designed FrameStackBuffer utility and integrates it into the cartpole camera presets task. The implementation addresses a real need—providing explicit temporal information for Newton+Warp where implicit temporal data isn't available. Overall structure is clean, but I found a few issues worth addressing.

Summary

Strengths:

• Ring buffer design avoids per-frame allocations
• Auto-detection of Newton+Warp combo is convenient
• Comprehensive unit tests for the buffer itself
• Good documentation of the temporal info requirement

Areas for improvement: See inline comments below.

[kellyguo11]

I think this would be better placed for https://isaac-sim.github.io/IsaacLab/develop/source/overview/core-concepts/renderers.html since it's a general note around the renderer behavior difference.

[kellyguo11]

I think we can avoid a FrameStackBuffer class all together and use the existing CircularBuffer implementation instead.

This can become:

• Add a stacked_image ManagerTermBase in isaaclab/envs/mdp/observations.py so manager-based environments can reference this mdp in the environment config.
• For the direct env cartpole task, use CircularBuffer directly in the task (keeping it task-local), but don't invent a new helper class for the ring buffer.