Skip to content

cuda.core: add atomic update methods for graph definition nodes #2352

Description

@Andy-Jost

Background

Part of #1330. Depends on #2350.

GraphNode subclasses currently expose node parameters as read-only properties. The original plan called for property setters, but CUDA updates most node types by replacing one complete parameter structure. Several parameters and their ownership requirements are also coupled—for example, a host callback function with its userData, or a kernel with its arguments.

Add atomic, subclass-specific update() methods rather than independent property setters.

This feature requires CUDA 12.2+ and uses the generic cuGraphNodeSetParams API. Supporting CUDA 12.0/12.1 through the older typed setters is intentionally out of scope.

Proposed API

Properties remain read-only for inspection. Mutation uses one method call and one CUDA parameter update.

kernel.update(config=new_config)
kernel.update(args=new_args)
kernel.update(kernel=new_kernel, args=new_args)
kernel.update(config=new_config, kernel=new_kernel, args=new_args)

memcpy.update(dst=new_dst, src=new_src, size=new_size)
memset.update(dst=new_dst, value=new_value, width=new_width)

host.update(new_callback, user_data=new_user_data)
child.update(new_child_graph)
event_record.update(new_event)
event_wait.update(new_event)

Omitted keyword arguments preserve their current values where doing so is coherent.

Kernel-specific rules:

  • Arguments may be updated while retaining the existing kernel.
  • Changing the kernel requires explicitly supplying its arguments, including args=() for a no-argument kernel.
  • Configuration, kernel, and arguments may be replaced atomically in one call.
  • Cluster dimensions and cooperative-launch state remain outside this API unless they can be represented correctly by the supported graph-node APIs.

Memcpy/memset rules:

  • Multiple fields may be replaced atomically.
  • Buffer operands retain their allocation handles.
  • Raw integer addresses remain caller-owned.
  • Explicit dst_owner and src_owner are accepted only alongside their corresponding raw-address operands.
  • Supplying a new raw address without an owner removes that slot's ownership from the updated definition.

Host callbacks always replace the callback and user-data binding together; updating only one is not supported.

Implementation

Each method combines supplied values with the node's current state to construct one fully initialized, zero-padded CUgraphNodeParams. It sets the appropriate node-type member and invokes cuGraphNodeSetParams exactly once through shared update-and-rollback plumbing.

When ownership changes:

  1. Resolve all inputs and construct an immutable replacement NodeAttachment.
  2. Retain its user object on the definition graph.
  3. Call cuGraphNodeSetParams.
  4. On success, publish the replacement attachment record, release the graph's old user-object reference, and update cached Python state.
  5. On failure, release the replacement and leave parameters, attachments, and cached state unchanged.

Updates that do not change ownership may reuse the existing attachment record.

ChildGraphNode.update() uses the CU_GRAPH_NODE_TYPE_GRAPH member of CUgraphNodeParams. Because CUDA replaces the node's embedded clone, the update must also invalidate handles for the old child hierarchy and import/rekey attachment metadata for the replacement hierarchy.

Supported nodes

Implement update() for:

  • KernelNode
  • MemcpyNode
  • MemsetNode
  • HostCallbackNode
  • ChildGraphNode
  • EventRecordNode
  • EventWaitNode

EmptyNode has no mutable parameters, while CUDA does not support parameter updates for allocation, free, or conditional nodes.

Acceptance criteria

  • Every supported node type can be updated and then instantiated successfully on CUDA 12.2+.
  • Coupled parameters are changed with one cuGraphNodeSetParams call.
  • Multi-field memcpy, memset, and kernel updates are atomic.
  • Existing executable graphs continue using their old parameters and attachments after their source definition is updated.
  • New executable graphs use the replacement parameters and attachments.
  • Invalid updates leave driver state, Python state, and attachment ownership unchanged.
  • Replaced owners are released when no clone or executable graph retains them.
  • Raw addresses and Buffer/explicit-owner inputs follow their documented ownership contracts.
  • Child replacement invalidates old descendant views and preserves replacement hierarchy metadata.
  • Public stubs and graph API documentation describe every update signature and the CUDA 12.2+ requirement.

Non-goals

  • CUDA 12.0/12.1 fallback through typed node setters.
  • Executable-graph node updates.
  • Property setters as mutation syntax.
  • Updating allocation/free or conditional-node structure.
  • Recovering coherent metadata after graph mutation through raw CUDA handles.

Metadata

Metadata

Assignees

Labels

P0High priority - Must do!cuda.coreEverything related to the cuda.core modulefeatureNew feature or request

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions