[H4 fix attempt] Add explicit glFinish + wp.synchronize_device before PBO map#5542
Closed
hujc7 wants to merge 2 commits intoisaac-sim:developfrom
Closed
[H4 fix attempt] Add explicit glFinish + wp.synchronize_device before PBO map#5542hujc7 wants to merge 2 commits intoisaac-sim:developfrom
hujc7 wants to merge 2 commits intoisaac-sim:developfrom
Conversation
…O map Adds explicit GL fence + CUDA sync between glReadPixels (PBO async readback) and wp.RegisteredGLBuffer.map (CUDA-side mapped read) in ViewerGL.get_frame, via a test conftest. Tests whether the regression is a missed sync between GL framebuffer-to-PBO transfer and CUDA's mapped read.
This was referenced May 8, 2026
Collaborator
Author
|
Hypothesis disproven. With explicit Combined with the diagnostic in #5540 (CPU FBO readback also zero), GL/CUDA sync is not the cause — the FBO itself is empty, so no amount of post-readback sync can produce non-zero pixels. Closing. |
AntoineRichard
pushed a commit
that referenced
this pull request
May 8, 2026
#5538) ## Summary Two unrelated CI breakages on develop, bundled here so develop turns green in one PR. ### 1. Skip the failing viewergl test `test_cartpole_newton_visualizer_viewergl_rgb_motion[physx,newton]` started returning all-black frames on develop after `nvcr.io/nvidian/isaac-sim:latest-develop` flipped to a Kit 110.1.1 + USD 25.11 base. The failure has been deterministic across multiple PRs (#5523, #5495, #5408, …). Investigation so far has ruled out: - PR #5521 (revert in #5539 still failed) - Newton 1.0 → 1.2.0rc2 viewer code regression (only 7-line addition; ViewerGL alone yields 1.08M nonzero pixels) - warp 1.12 → 1.13 RegisteredGLBuffer ABI (byte-identical) - Module-load side effects of `isaaclab_physx.renderers` - CUDA-GL interop (PR #5540 diagnostic confirms direct CPU FBO readback also returns zeros, with `GL_NO_ERROR`) - GL context-currency (PR #5541 H6 attempt: still fails) - GL/CUDA sync (PR #5542 H4 attempt: still fails) Diagnostic output (PR #5540 v2): ``` [VIZDIAG] fbo=c_uint(8) pbo=None size=600x600 [VIZDIAG] glGetError before: GL_NO_ERROR [VIZDIAG] CPU-readback: nonzero=0/1080000 max=0 err=GL_NO_ERROR [VIZDIAG] PBO-result: nonzero=0/1080000 max=0 ``` The FBO itself is empty — Newton's pyglet/EGL renderer is not depositing pixels under Kit 110.1.1, even though `tiled_camera_rgb_non_black` (Kit RTX path) on the same env passes. Underlying root cause still being chased; this PR ships the skip to unblock develop. ### 2. Fix warp intersphinx 404 in docs build `https://nvidia.github.io/warp/objects.inv` started returning 404 — Warp's `objects.inv` only lives at `/stable/` and `/latest/` now. With Sphinx's `warnings_treated_as_errors`, the broken intersphinx fetch fails the docs build on every PR. Pinning to `/stable/` (matches the existing PyTorch `/docs/2.11/` workaround pattern in the same file). Verified `https://nvidia.github.io/warp/stable/objects.inv` returns 200. ## Test plan - [x] CI `isaaclab_visualizers` on this branch — was passing earlier with the skip; will re-verify with the bundled docs fix - [ ] CI `Build Latest Docs` on this branch — must turn green (was failing on every recent PR before this fix) ## Re-enable plan Once the underlying viewergl bug is identified and fixed, drop the `@pytest.mark.skip` decorator and remove the `jichuanh-disable-viewergl-flaky.skip` fragment.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix-attempt PR for the viewergl-fully-black regression on develop.
Hypothesis (H4): warp 1.13 (or a Kit/Replicator version flip in the daily
nvcr.io/nvidian/isaac-sim:latest-developimage) changed the implicit CUDA-OpenGL sync semantics insidewp.RegisteredGLBuffer.map, so the CUDA read of the GL PBO can race the asynchronousglReadPixelswrite into that same PBO.This patch adds an explicit
glFinish()+wp.synchronize_device()between the PBO write and the CUDA-side map. If the test passes with this, the proper fix is upstream in Newton'sget_frame.Companion of #5540 (diagnostic) and #5541 (H6 _make_current attempt). PR #5521 hypothesis disproven by #5539.
DO NOT MERGE — testing only.