[CI unblock] Skip viewergl-fully-black test + fix warp intersphinx 404#5538
Conversation
ViewerGL.get_frame returns a fully-black 600x600x3 buffer in CI on the current Isaac Sim image + Newton 1.2.0rc2 + warp-lang 1.13 cohort. The failure is deterministic across two consecutive reruns of the same SHA and reproduces on every recent PR that touches the rendering / camera / sensor / USD stack: isaac-sim#5523 (Newton 1.2.0rc2 bump) isaac-sim#5534 (Camera/Renderer warp rewrite) isaac-sim#5024 (heterogeneous dexsuite) isaac-sim#5495 (usd-core 25.11) isaac-sim#5492 (OVRTX rendering tests) 9 visualizer-job failures across these 5 PRs in the last 100 build.yaml runs, zero failures on PRs outside that scope. Investigation ruled out: * rc1 -> rc2 viewer code diff -- only a 7-line image_logger.clear addition. * wp.RegisteredGLBuffer API -- byte-identical between warp-lang 1.12 and 1.13. * pure CI flakiness -- deterministic across reruns. * the bump cohort alone -- branches on warp-lang 1.12 + Newton @a27277e both pass and fail this test depending on PR content. Strongest remaining hypothesis: a CUDA-OpenGL interop init-order fragility in the PBO + glReadPixels + RegisteredGLBuffer.map readback path that gets tipped to deterministic-fail by any source change perturbing GL/CUDA bring-up. Pinning it down requires py-spy or glReadPixels return-code inspection inside the CI container, which is out of scope for this change. Skip both ['physx', 'newton'] parametrizations until the root cause is identified. Re-enable in a follow-up PR.
Greptile SummaryThis PR skips
Confidence Score: 5/5Safe to merge — a single skip decorator is added to one test function; no production code is touched. The change is limited to adding @pytest.mark.skip above @pytest.mark.parametrize, which is the correct placement to skip all parametrized variants. The empty changelog fragment follows established project convention. No logic, APIs, or production paths are affected. No files require special attention. Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A["test_cartpole_newton_visualizer_viewergl_rgb_motion"] -->|"@pytest.mark.skip"| B["SKIPPED — both parametrizations"]
B --> C["[physx, newton]"]
B --> D["[newton, newton]"]
C -.->|"pending root-cause fix"| E["Re-enable in follow-up PR"]
D -.->|"pending root-cause fix"| E
Reviews (1): Last reviewed commit: "Skip test_cartpole_newton_visualizer_vie..." | Re-trigger Greptile |
https://nvidia.github.io/warp/objects.inv now returns 404; the inventory moved under /stable/ (and /latest/). With warnings treated as errors, the broken intersphinx fetch fails the docs build on every PR. Repro: any recent CI run on develop, e.g. https://github.com/isaac-sim/IsaacLab/actions/runs/25533637651/job/74955918336 WARNING: failed to reach any of the inventories with the following issues: intersphinx inventory 'https://nvidia.github.io/warp/objects.inv' not fetchable due to 404 Client Error: Not Found build finished with problems, 1 warning (with warnings treated as errors).
be1553e to
0de218c
Compare
Summary
Two unrelated CI breakages on develop, bundled here so develop turns green in one PR.
1. Skip the failing viewergl test
test_cartpole_newton_visualizer_viewergl_rgb_motion[physx,newton]started returning all-black frames on develop afternvcr.io/nvidian/isaac-sim:latest-developflipped to a Kit 110.1.1 + USD 25.11 base. The failure has been deterministic across multiple PRs (#5523, #5495, #5408, …).Investigation so far has ruled out:
isaaclab_physx.renderersGL_NO_ERROR)Diagnostic output (PR #5540 v2):
The FBO itself is empty — Newton's pyglet/EGL renderer is not depositing pixels under Kit 110.1.1, even though
tiled_camera_rgb_non_black(Kit RTX path) on the same env passes. Underlying root cause still being chased; this PR ships the skip to unblock develop.2. Fix warp intersphinx 404 in docs build
https://nvidia.github.io/warp/objects.invstarted returning 404 — Warp'sobjects.invonly lives at/stable/and/latest/now. With Sphinx'swarnings_treated_as_errors, the broken intersphinx fetch fails the docs build on every PR. Pinning to/stable/(matches the existing PyTorch/docs/2.11/workaround pattern in the same file).Verified
https://nvidia.github.io/warp/stable/objects.invreturns 200.Test plan
isaaclab_visualizerson this branch — was passing earlier with the skip; will re-verify with the bundled docs fixBuild Latest Docson this branch — must turn green (was failing on every recent PR before this fix)Re-enable plan
Once the underlying viewergl bug is identified and fixed, drop the
@pytest.mark.skipdecorator and remove thejichuanh-disable-viewergl-flaky.skipfragment.