docs: clarify cuda.core concurrency and thread-safety policy

Document the concurrency contract that was previously agreed only in review
threads: concurrent reads of a cuda.core object are supported, but concurrent
mutation of the same object requires application-provided synchronization.
Distinct objects can still collide through shared CUDA driver/context state
(e.g. peer device access), which is also the caller's responsibility.

- docs/source/concurrency.rst: new user-facing "Concurrency and Thread Safety"
  page, linked into the docs toctree.
- cuda_core/AGENTS.md: contributor-facing invariants (reads-safe/mutation
  boundary, prefer immutable designs, shared-driver-state collisions, guarding
  internal cached state only for legitimately-shared objects, GIL-held entry
  points, and releasing the GIL before entering the driver from callback/__del__
  paths).

Author: Andy-Jost

cuda_core/AGENTS.md

@@ -63,6 +63,44 @@ This file describes `cuda_core`, the high-level Pythonic CUDA subpackage in the
 - Prefer explicit error propagation over silent fallback paths.
 - If you change public behavior, update tests and docs under `docs/source/`.
 
+## Concurrency and free-threading
+
+`cuda.core` ships free-threaded (no-GIL) wheels and builds with
+`freethreading_compatible=True`. The user-facing policy lives in
+`docs/source/concurrency.rst`; the invariants below are for contributors. Reviewers
+and agents should flag violations.
+
+- **Reads are safe; mutation is the boundary**: concurrent reads of an object are
+  supported, but concurrent mutation of the same public object (e.g., building one
+  graph from two threads, or `close()` racing another call) is the caller's
+  responsibility -- do not add locks to make it safe. Prefer immutable designs to
+  keep the mutable, thread-unsafe surface small. Protecting library-internal state
+  *is* in scope.
+- **Distinct objects can still collide via shared driver state**: operating on
+  separate objects is not automatically safe when they share driver or context
+  state (e.g., changing peer device access while another thread touches affected
+  memory). Synchronizing these cases is the caller's responsibility; do not try to
+  lock around them internally.
+- **Protect internal cached/module-level state**: guard lazily-populated
+  `cdef object` caches and module-level state so concurrent access cannot corrupt
+  interpreter state (CPython reference counts -- a strictly free-threading hazard).
+  Established patterns are `@cython.critical_section` on accessors (#2215), an
+  atomic initialization flag (#2216), and `dict.setdefault` for identity caches
+  (#2217). Guard state only on objects that are legitimately shared between threads;
+  objects that are not meant to be shared (e.g., the thread-local `Device`) do not
+  need such guards (see #2321). Reference-count integrity is guaranteed; cache
+  value-identity/idempotency is not.
+- **Entry points assume the GIL is held**: the helpers in `_cpp/resource_handles.*`
+  are called from Cython with the GIL held and do not re-acquire it. Driver and
+  destructor callbacks run at arbitrary times, so they take the GIL (`with gil`)
+  and probe for interpreter shutdown before touching Python objects.
+- **Lock ordering -- release the GIL before entering the driver**: any CUDA work
+  reachable from a host callback or a retained object's `__del__` must release the
+  GIL before calling the driver, to avoid GIL/driver-lock deadlocks (see the
+  `_py_host_trampoline` path and numba-cuda#321). Objects retained into a graph
+  (kernel arguments, memcpy/memset operands, `dst_owner`/`src_owner`, and
+  host-callback closures) inherit this contract.
+
 ## API design guidelines
 
 These are some API design guidelines we try to follow when adding new APIs to

cuda_core/docs/source/concurrency.rst

@@ -0,0 +1,28 @@
+.. SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+.. SPDX-License-Identifier: Apache-2.0
+
+.. currentmodule:: cuda.core
+
+Concurrency and Thread Safety
+=============================
+
+``cuda.core`` allows concurrent reads of its objects from multiple host
+threads, but concurrent *mutation* of the same object is **not supported** --
+for example, adding nodes to the same graph, or closing a resource while
+another thread is using it. Whenever an object is shared across threads and at
+least one of them may mutate it, the application is responsible for providing
+external synchronization.
+
+The library does protect the integrity of its own internal state (such as
+cached attributes and reference counting), so that concurrent reads of the same
+object, or any kind of concurrent use of *distinct* objects, cannot corrupt the
+interpreter. This is an integrity guarantee only: the ordering and outcome of
+concurrent operations on a shared object are otherwise undefined.
+
+Additional limitations apply because ``cuda.core`` inherits the concurrency
+constraints of the underlying CUDA driver. Distinct ``cuda.core`` objects can
+share driver or context state, so operating on separate objects is not always
+safe. For example, modifying peer device access from one thread while another
+thread accesses device memory affected by that change is unsafe, even though the
+two threads use different objects. The application is responsible for
+synchronizing operations that concurrently read and modify shared driver state.

cuda_core/docs/source/index.rst

@@ -1,4 +1,4 @@
-.. SPDX-FileCopyrightText: Copyright (c) 2024-2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+.. SPDX-FileCopyrightText: Copyright (c) 2024-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 .. SPDX-License-Identifier: Apache-2.0
 
 ``cuda.core``: Pythonic access to CUDA core functionality
@@ -14,6 +14,7 @@ Welcome to the documentation for ``cuda.core``.
    install
    examples
    interoperability
+   concurrency
    api
    api_nvml
    environment_variables