Skip to content

DO NOT MERGE: Experiment — test library/kernel lifetime vs context - #1672

Closed
Andy-Jost wants to merge 2 commits into
NVIDIA:mainfrom
Andy-Jost:test-library-context-lifetime
Closed

DO NOT MERGE: Experiment — test library/kernel lifetime vs context#1672
Andy-Jost wants to merge 2 commits into
NVIDIA:mainfrom
Andy-Jost:test-library-context-lifetime

Conversation

@Andy-Jost

Copy link
Copy Markdown
Contributor

Summary

This is an experiment — do not merge.

Tests whether cuLibraryUnload after context destruction causes segfaults on any CI platform. Currently LibraryBox does not store a ContextHandle, so nothing prevents the context from being destroyed before the library.

Test Scenarios

  1. ObjectCode outlives primary context reset
  2. Kernel outlives primary context reset
  3. ObjectCode outlives primary context reset + re-retain (numba-cuda pattern)
  4. ObjectCode outlives non-primary context destruction
  5. Kernel outlives non-primary context destruction
  6. Kernel outlives ObjectCode AND primary context reset
  7. Kernel outlives ObjectCode AND non-primary context destruction

All 7 pass locally on driver 580.105.08 (L40S). Running CI to see if any platform crashes.

Made with Cursor

@Andy-Jost Andy-Jost self-assigned this Feb 20, 2026
@copy-pr-bot

copy-pr-bot Bot commented Feb 20, 2026

Copy link
Copy Markdown
Contributor

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@Andy-Jost

Copy link
Copy Markdown
Contributor Author

/ok to test 3c9a874c7ec5751e1156243bba5529abda221be4

@Andy-Jost Andy-Jost added this to the cuda.core backlog milestone Feb 20, 2026
@Andy-Jost Andy-Jost added cuda.core Everything related to the cuda.core module experiment Describes an investigation or measurement labels Feb 20, 2026
@github-actions

Copy link
Copy Markdown

Experiment to detect segfaults on CI platforms where cuLibraryUnload
after context destruction may crash. LibraryBox does not currently
store a ContextHandle, so nothing prevents the context from being
destroyed before the library.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Andy-Jost
Andy-Jost force-pushed the test-library-context-lifetime branch from 3c9a874 to 44f812e Compare February 20, 2026 20:38
@Andy-Jost

Copy link
Copy Markdown
Contributor Author

/ok to test 44f812e

@Andy-Jost

Andy-Jost commented Feb 20, 2026

Copy link
Copy Markdown
Contributor Author

Test results with primary context resets + context create/destroy

Platform GPU CUDA Driver SEGV?
linux-64 h100 13.0.2 580.126.09 Yes
linux-64 h100 13.1.1 580.126.09 Yes
linux-64 rtxpro6000 13.0.2 580.126.09 Yes
linux-64 rtxpro6000 13.1.1 580.126.09 Yes
linux-64 l4 13.0.2 580.126.09 Yes
linux-64 l4 13.1.1 580.126.09 Yes
linux-aarch64 a100 12.9.1 580.126.09 Yes
win-64 rtx2080 WDDM 12.9.1 581.15 Yes
win-64 rtxpro6000 TCC 13.0.2 581.15 Yes
win-64 rtxpro6000 TCC 13.1.1 581.15 Yes
win-64 rtx4090 WDDM 13.1.1 581.15 Yes
win-64 a100 TCC 13.0.2 581.15 Yes
win-64 l4 TCC 12.9.1 581.15 Yes
linux-64 v100 12.9.1 580.126.09 No
linux-64 t4 12.9.1 580.126.09 No

Test results with context create/destroy only

All pass

@Andy-Jost

Copy link
Copy Markdown
Contributor Author

/ok to test 7154e73ad7311996c5052935f926f90285f3cf2c

Primary context resets corrupt CuPy's internal state, causing cascading
failures in unrelated tests. Keep only non-primary context destroy tests
which isolate the library/context lifetime issue without side effects.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Andy-Jost
Andy-Jost force-pushed the test-library-context-lifetime branch from 7154e73 to 19f1037 Compare February 20, 2026 22:02
@Andy-Jost

Copy link
Copy Markdown
Contributor Author

/ok to test 19f1037

@Andy-Jost

Copy link
Copy Markdown
Contributor Author

Closing as not intended for merge.

@Andy-Jost Andy-Jost closed this Feb 24, 2026
github-actions Bot pushed a commit that referenced this pull request Feb 25, 2026
Removed preview folders for the following PRs:
- PR #1672
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cuda.core Everything related to the cuda.core module experiment Describes an investigation or measurement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant