cuda-core experimental namespace deprecation (attempt v1) (NVIDIA#1377)

* Migrate cuda.core.experimental to cuda.core

This commit migrates all code from cuda.core.experimental to cuda.core,
completing the deprecation of the experimental namespace.

Changes:
- Move all files from cuda/core/experimental/ to cuda/core/
- Move include/ directory to _include/ (with leading underscore for
  implementation detail)
- Update all imports from cuda.core.experimental.* to cuda.core.*
- Update build_hooks.py to use cuda.core instead of cuda.core.experimental
- Update pyproject.toml package-data paths
- Update cuda/core/__init__.py to export all symbols
- Update cuda/core/experimental/__init__.py for backward compatibility
  with deprecation warnings
- Update Cython extern declarations to use _include/ instead of include/
- Fix import paths for _memory_pool (cuda.core._memory._memory_pool)
- Update test files and test helpers to use new import paths

All tests pass (1499 passed, 79 skipped).

* Remove experimental namespace from examples and test files

Update all example files and test Cython files to use cuda.core instead
of cuda.core.experimental:

- Update all example imports from cuda.core.experimental.* to cuda.core.*
- Update example utils imports from cuda.core.experimental.utils to cuda.core.utils
- Update test Cython file imports
- Update build_tests.sh include path from experimental/include to _include

All example files now use the non-experimental import paths.

* Update .spdx-ignore for cuda_core/cuda/core/_include/dlpack.h

* Update documentation to reflect migration from experimental to core

Update all documentation files to reference cuda.core instead of
cuda.core.experimental:

- api.rst: Change module from cuda.core.experimental to cuda.core
- getting-started.rst: Update currentmodule and example imports
- interoperability.rst: Update currentmodule
- api_private.rst: Update currentmodule
- conf.py: Update system import paths and excluded_dirs paths

All documentation now reflects the new non-experimental API paths.
Release notes files are left unchanged as they are historical.

* Update issue templates and wheel merge script for cuda.core migration

Update GitHub issue templates and wheel merge script to reference
cuda.core instead of cuda.core.experimental:

- bug_report.yml: Update example references from cuda.core.experimental
  to cuda.core
- feature_request.yml: Update example references from cuda.core.experimental
  to cuda.core
- merge_cuda_core_wheels.py: Update to merge cuda/core/ instead of
  cuda/core/experimental/, and adopt logic that copies only binaries
  into versioned subdirectories while keeping Python modules in cuda/core/

* chore: bump pixi version

* test: handle deprecated calls

* build: remove unnecessary `local_include_dirs`

* chore: try reverting to the original merge wheel script

* revert: chore: try reverting to the original merge wheel script

This reverts commit e0f784d.

* test: try removing unnecessary utils import

* chore: copy everything

* Fix merge script to selectively copy files to versioned directories

Fix the wheel merge script to copy files selectively instead of copying
everything, which was causing import errors:

- Copy binaries (.so, .pyd, .dll) to versioned directories (version-specific)
- Copy Python files (.py) to versioned directories (needed for imports like utils.py)
- Do NOT copy Cython files (.pyx, .pxd) to versioned directories

The previous "copy everything" approach caused failures when testing CUDA 12.9.1
because CUDA 13 .pyx files (which reference version-specific C functions like
cuMemGetMemPool) were being copied into cu12/ directories, causing import errors.

This selective approach fixes both:
- Original issue: utils.py import failures (now .py files are copied)
- Current issue: cuMemGetMemPool errors (now .pyx files are NOT copied)

* Add debugging output to show wheel directory structures

Add debugging output to show the directory structure of input and output
wheels for troubleshooting. This will help diagnose issues with the
wheel merge process.

- Use Python's zipfile module (standard library) to list wheel contents
- Add debugging output showing cuda/core/ directory structure for:
  * Each input wheel before merging
  * The output merged wheel after merging
- Format output similar to unzip -l for readability
- Filter output to show only cuda/core/ entries

* Fix merge script to match main branch's working approach

Based on analysis of main branch's merge script and CI logs, updated the
merge logic to:

1. Use shutil.copytree() to copy entire directory trees into versioned
   subdirectories (cu12/, cu13/), matching main's approach for experimental/

2. Clean up main cuda/core/ directory to only keep:
   - __init__.py
   - _include/ directory
   - Versioned subdirectories (cu12/, cu13/)

   This ensures Python imports from versioned directories instead of
   main directory, which may contain binaries from a different CUDA version.

3. Add DRY helper function print_wheel_directory_structure() for debugging
   output, refactored from duplicate code.

4. Update docstring to reference cuda/core instead of experimental namespace.

This fixes the ImportError issues where Python was loading CUDA 13 binaries
when running on CUDA 12.9.1, because the main directory still contained
.so files from the base wheel.

* Fix merge script: keep _version.py and __init__.pxd in main directory

The merge script was removing _version.py and __init__.pxd from the main
cuda/core/ directory, but these files are required:
- _version.py: imported by __init__.py at module initialization
- __init__.pxd: kept for Cython compatibility (matches main branch behavior)

Updated the cleanup logic to preserve these files along with __init__.py,
_include/, and versioned subdirectories (cu12/, cu13/).

The code was also manually cleaned up for better readability and
maintainability.

* Rework top-level into in cuda_core/docs/source/api.rst

* Bug fix in ci/tools/merge_cuda_core_wheels.py: replace AI-generated .touch() with os.truncate()

* Add backward compatibility tests for experimental namespace

- Add test_experimental_backward_compat.py from v0 branch
- Remove __getattr__ from experimental/__init__.py per reviewer feedback
  (underscored modules are not public APIs)
- Update test to expect AttributeError when accessing underscored modules
- Document in module docstring that underscored modules are intentionally
  not accessible through experimental namespace
- Move note about underscored modules to module docstring (per pre-commit)

* Remove __init__.pxd from items_to_keep list in ci/tools/merge_cuda_core_wheels.py

* Combine experimental backward compat tests into single function

- Combine all test functions into test_experimental_backward_compatibility()
  to avoid race conditions when tests run in parallel
- Add defensive sys.modules cleanup at start of test function
- Remove test for underscored module access (not needed per reviewer feedback)

* Remove StridedMemoryView from experimental namespace

Remove StridedMemoryView and args_viewable_as_strided_memory from
experimental namespace. These should be imported from cuda.core.utils
instead.

* Specify v1.0.0 as removal target for experimental namespace

Update deprecation warning and docstring to specify that the
experimental namespace will be removed in v1.0.0, rather than
the vague 'future release'. This provides users with a clear
timeline for migration.

* Apply suggestions from code review

* [pre-commit.ci] auto code formatting

* make linter happy

* [pre-commit.ci] auto code formatting

* revert the utils change

---------

Co-authored-by: Phillip Cloud <417981+cpcloud@users.noreply.github.com>
Co-authored-by: Leo Fang <leof@nvidia.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Author: 4 people

.github/ISSUE_TEMPLATE/bug_report.yml

@@ -52,7 +52,7 @@ body:
     attributes:
       label: Describe the bug
       description: A clear and concise description of what problem you are running into.
-      placeholder: "Attempting to compile a program via `cuda.core.experimental.Program.compile` throws a `ValueError`."
+      placeholder: "Attempting to compile a program via `cuda.core.Program.compile` throws a `ValueError`."
     validations:
       required: true
 
@@ -62,7 +62,7 @@ body:
       label: How to Reproduce
       description: Steps used to reproduce the bug.
       placeholder: |
-        0. Construct a `cuda.core.experimental.Program` instance
+        0. Construct a `cuda.core.Program` instance
         1. Call the `.compile(...)` method of the instance
         2. The call throws a `ValueError` with the following:
         ```
@@ -76,7 +76,7 @@ body:
     attributes:
       label: Expected behavior
       description: A clear and concise description of what you expected to happen.
-      placeholder: "Using `cuda.core.experimental.Program.compile(...)` should run successfully and not throw a `ValueError`"
+      placeholder: "Using `cuda.core.Program.compile(...)` should run successfully and not throw a `ValueError`"
     validations:
       required: true
 

.github/ISSUE_TEMPLATE/feature_request.yml

@@ -36,7 +36,7 @@ body:
     attributes:
       label: Is your feature request related to a problem? Please describe.
       description: A clear and concise description of what the problem is, e.g., "I would like to be able to..."
-      placeholder: I would like to be able to use the equivalent of `cuda.core.experimental.Program.compile(...)` to compile my code to PTX.
+      placeholder: I would like to be able to use the equivalent of `cuda.core.Program.compile(...)` to compile my code to PTX.
     validations:
       required: true
 
@@ -46,7 +46,7 @@ body:
       label: Describe the solution you'd like
       description: A clear and concise description of what you want to happen.
       placeholder: |
-        Support a `ptx` target_type in the `cuda.core.experimental.Program.compile(...)` function.
+        Support a `ptx` target_type in the `cuda.core.Program.compile(...)` function.
     validations:
       required: true
 
@@ -57,7 +57,7 @@ body:
       description:
         If applicable, please add a clear and concise description of any alternative solutions or features you've
         considered.
-      placeholder: The alternatives to using `cuda.core.experimental.Program.compile(...)` are unappealing. They usually involve using lower level bindings to something like nvRTC or invoking the nvcc executable.
+      placeholder: The alternatives to using `cuda.core.Program.compile(...)` are unappealing. They usually involve using lower level bindings to something like nvRTC or invoking the nvcc executable.
     validations:
       required: false
 

.spdx-ignore

@@ -9,6 +9,6 @@ requirements*.txt
 cuda_bindings/examples/*
 
 # Vendored
-cuda_core/cuda/core/experimental/include/dlpack.h
+cuda_core/cuda/core/_include/dlpack.h
 
 qa/ctk-next.drawio.svg

ci/tools/merge_cuda_core_wheels.py

@@ -12,8 +12,8 @@
 
 In particular, each wheel contains a CUDA-specific build of the `cuda.core` library
 and the associated bindings. This script merges these directories into a single wheel
-that supports both CUDA versions, i.e., containing both `cuda/core/experimental/cu12`
-and `cuda/core/experimental/cu13`. At runtime, the code in `cuda/core/experimental/__init__.py`
+that supports both CUDA versions, i.e., containing both `cuda/core/cu12`
+and `cuda/core/cu13`. At runtime, the code in `cuda/core/__init__.py`
 is used to import the appropriate CUDA-specific bindings.
 
 This script is based on the one in NVIDIA/CCCL.
@@ -25,6 +25,7 @@
 import subprocess
 import sys
 import tempfile
+import zipfile
 from pathlib import Path
 from typing import List
 
@@ -46,7 +47,38 @@ def run_command(cmd: List[str], cwd: Path = None, env: dict = os.environ) -> sub
     return result
 
 
-def merge_wheels(wheels: List[Path], output_dir: Path) -> Path:
+def print_wheel_directory_structure(wheel_path: Path, filter_prefix: str = "cuda/core/", label: str = None):
+    """Print the directory structure of a wheel file, similar to unzip -l output.
+
+    Args:
+        wheel_path: Path to the wheel file to inspect
+        filter_prefix: Only show files matching this prefix (default: "cuda/core/")
+        label: Optional label to print before the structure (e.g., "Input wheel 1: name.whl")
+    """
+    if label:
+        print(f"\n--- {label} ---", file=sys.stderr)
+    try:
+        with zipfile.ZipFile(wheel_path, "r") as zf:
+            print(f"{'Length':>10}  {'Date':>12}  {'Time':>8}  Name", file=sys.stderr)
+            print("-" * 80, file=sys.stderr)
+            total_size = 0
+            file_count = 0
+            for name in sorted(zf.namelist()):
+                if filter_prefix in name:
+                    info = zf.getinfo(name)
+                    total_size += info.file_size
+                    file_count += 1
+                    date_time = info.date_time
+                    date_str = f"{date_time[0]:04d}-{date_time[1]:02d}-{date_time[2]:02d}"
+                    time_str = f"{date_time[3]:02d}:{date_time[4]:02d}:{date_time[5]:02d}"
+                    print(f"{info.file_size:10d}  {date_str}  {time_str}  {name}", file=sys.stderr)
+            print("-" * 80, file=sys.stderr)
+            print(f"{total_size:10d}                    {file_count} files", file=sys.stderr)
+    except Exception as e:
+        print(f"Warning: Could not list wheel contents: {e}", file=sys.stderr)
+
+
+def merge_wheels(wheels: List[Path], output_dir: Path, show_wheel_contents: bool = True) -> Path:
     """Merge multiple wheels into a single wheel with version-specific binaries."""
     print("\n=== Merging wheels ===", file=sys.stderr)
     print(f"Input wheels: {[w.name for w in wheels]}", file=sys.stderr)
@@ -91,30 +123,50 @@ def merge_wheels(wheels: List[Path], output_dir: Path) -> Path:
 
             extracted_wheels.append(extract_dir)
 
+        if show_wheel_contents:
+            print("\n=== Input wheel directory structures ===", file=sys.stderr)
+            for i, wheel in enumerate(wheels):
+                print_wheel_directory_structure(wheel, label=f"Input wheel {i + 1}: {wheel.name}")
+
         # Use the first wheel as the base and merge binaries from others
         base_wheel = extracted_wheels[0]
 
-        # now copy the version-specific directory from other wheels
-        # into the appropriate place in the base wheel
+        # Copy version-specific directories from each wheel into versioned subdirectories
+        base_dir = Path("cuda") / "core"
+
         for i, wheel_dir in enumerate(extracted_wheels):
             cuda_version = wheels[i].name.split(".cu")[1].split(".")[0]
-            base_dir = Path("cuda") / "core" / "experimental"
-            # Copy from other wheels
-            print(f"  Copying {wheel_dir} to {base_wheel}", file=sys.stderr)
-            shutil.copytree(wheel_dir / base_dir, base_wheel / base_dir / f"cu{cuda_version}")
-
-            # Overwrite the __init__.py in versioned dirs
-            os.truncate(base_wheel / base_dir / f"cu{cuda_version}" / "__init__.py", 0)
-
-        # The base dir should only contain __init__.py, the include dir, and the versioned dirs
-        files_to_remove = os.scandir(base_wheel / base_dir)
-        for f in files_to_remove:
+            versioned_dir = base_wheel / base_dir / f"cu{cuda_version}"
+
+            # Copy entire directory tree from source wheel to versioned directory
+            print(f"  Copying {wheel_dir / base_dir} to {versioned_dir}", file=sys.stderr)
+            shutil.copytree(wheel_dir / base_dir, versioned_dir, dirs_exist_ok=True)
+
+            # Overwrite the __init__.py in versioned dirs to be empty
+            os.truncate(versioned_dir / "__init__.py", 0)
+
+        print("\n=== Removing files from cuda/core/ directory ===", file=sys.stderr)
+        items_to_keep = (
+            "__init__.py",
+            "_version.py",
+            "_include",
+            "cu12",
+            "cu13",
+        )
+        all_items = os.scandir(base_wheel / base_dir)
+        removed_count = 0
+        for f in all_items:
             f_abspath = f.path
-            if f.name not in ("__init__.py", "cu12", "cu13", "include"):
-                if f.is_dir():
-                    shutil.rmtree(f_abspath)
-                else:
-                    os.remove(f_abspath)
+            if f.name in items_to_keep:
+                continue
+            if f.is_dir():
+                print(f"  Removing directory: {f.name}", file=sys.stderr)
+                shutil.rmtree(f_abspath)
+            else:
+                print(f"  Removing file: {f.name}", file=sys.stderr)
+                os.remove(f_abspath)
+            removed_count += 1
+        print(f"Removed {removed_count} items from cuda/core/ directory", file=sys.stderr)
 
         # Repack the merged wheel
         output_dir.mkdir(parents=True, exist_ok=True)
@@ -142,6 +194,11 @@ def merge_wheels(wheels: List[Path], output_dir: Path) -> Path:
 
         merged_wheel = output_wheels[0]
         print(f"Successfully merged wheel: {merged_wheel}", file=sys.stderr)
+
+        if show_wheel_contents:
+            print("\n=== Output wheel directory structure ===", file=sys.stderr)
+            print_wheel_directory_structure(merged_wheel)
+
         return merged_wheel
 
 

cuda_core/build_hooks.py

@@ -68,7 +68,7 @@ def _build_cuda_core():
 
     # It seems setuptools' wildcard support has problems for namespace packages,
     # so we explicitly spell out all Extension instances.
-    root_module = "cuda.core.experimental"
+    root_module = "cuda.core"
     root_path = f"{os.path.sep}".join(root_module.split(".")) + os.path.sep
     ext_files = glob.glob(f"{root_path}/**/*.pyx", recursive=True)
 
@@ -86,6 +86,7 @@ def get_cuda_paths():
         print("CUDA paths:", CUDA_PATH)
         return CUDA_PATH
 
+    all_include_dirs = list(os.path.join(root, "include") for root in get_cuda_paths())
     extra_compile_args = []
     if COMPILE_FOR_COVERAGE:
         # CYTHON_TRACE_NOGIL indicates to trace nogil functions.  It is not
@@ -94,9 +95,9 @@ def get_cuda_paths():
 
     ext_modules = tuple(
         Extension(
-            f"cuda.core.experimental.{mod.replace(os.path.sep, '.')}",
-            sources=[f"cuda/core/experimental/{mod}.pyx"],
-            include_dirs=list(os.path.join(root, "include") for root in get_cuda_paths()),
+            f"cuda.core.{mod.replace(os.path.sep, '.')}",
+            sources=[f"cuda/core/{mod}.pyx"],
+            include_dirs=all_include_dirs,
             language="c++",
             extra_compile_args=extra_compile_args,
         )

cuda_core/cuda/core/__init__.py

@@ -3,3 +3,67 @@
 # SPDX-License-Identifier: Apache-2.0
 
 from cuda.core._version import __version__
+
+try:
+    from cuda import bindings
+except ImportError:
+    raise ImportError("cuda.bindings 12.x or 13.x must be installed") from None
+else:
+    cuda_major, cuda_minor = bindings.__version__.split(".")[:2]
+    if cuda_major not in ("12", "13"):
+        raise ImportError("cuda.bindings 12.x or 13.x must be installed")
+
+import importlib
+
+subdir = f"cu{cuda_major}"
+try:
+    versioned_mod = importlib.import_module(f".{subdir}", __package__)
+    # Import all symbols from the module
+    globals().update(versioned_mod.__dict__)
+except ImportError:
+    # This is not a wheel build, but a conda or local build, do nothing
+    pass
+else:
+    del versioned_mod
+finally:
+    del bindings, importlib, subdir, cuda_major, cuda_minor
+
+from cuda.core import utils  # noqa: E402
+from cuda.core._device import Device  # noqa: E402
+from cuda.core._event import Event, EventOptions  # noqa: E402
+from cuda.core._graph import (  # noqa: E402
+    Graph,
+    GraphBuilder,
+    GraphCompleteOptions,
+    GraphDebugPrintOptions,
+)
+from cuda.core._launch_config import LaunchConfig  # noqa: E402
+from cuda.core._launcher import launch  # noqa: E402
+from cuda.core._layout import _StridedLayout  # noqa: E402
+from cuda.core._linker import Linker, LinkerOptions  # noqa: E402
+from cuda.core._memory import (  # noqa: E402
+    Buffer,
+    DeviceMemoryResource,
+    DeviceMemoryResourceOptions,
+    GraphMemoryResource,
+    LegacyPinnedMemoryResource,
+    ManagedMemoryResource,
+    ManagedMemoryResourceOptions,
+    MemoryResource,
+    PinnedMemoryResource,
+    PinnedMemoryResourceOptions,
+    VirtualMemoryResource,
+    VirtualMemoryResourceOptions,
+)
+from cuda.core._memoryview import (  # noqa: E402
+    StridedMemoryView,  # noqa: E402
+    args_viewable_as_strided_memory,  # noqa: E402
+)
+from cuda.core._module import Kernel, ObjectCode  # noqa: E402
+from cuda.core._program import Program, ProgramOptions  # noqa: E402
+from cuda.core._stream import Stream, StreamOptions  # noqa: E402
+from cuda.core._system import System  # noqa: E402
+
+system = System()
+__import__("sys").modules[__spec__.name + ".system"] = system
+del System

…core/cuda/core/experimental/_context.pyx cuda_core/cuda/core/_context.pyxcuda_core/cuda/core/experimental/_context.pyx renamed to cuda_core/cuda/core/_context.pyx cuda_core/cuda/core/experimental/_context.pyx renamed to cuda_core/cuda/core/_context.pyx

@@ -4,7 +4,7 @@
 
 from dataclasses import dataclass
 
-from cuda.core.experimental._utils.cuda_utils import driver
+from cuda.core._utils.cuda_utils import driver
 
 
 @dataclass

…_core/cuda/core/experimental/_device.pyx cuda_core/cuda/core/_device.pyxcuda_core/cuda/core/experimental/_device.pyx renamed to cuda_core/cuda/core/_device.pyx cuda_core/cuda/core/experimental/_device.pyx renamed to cuda_core/cuda/core/_device.pyx

@@ -6,27 +6,27 @@ cimport cpython
 from libc.stdint cimport uintptr_t
 
 from cuda.bindings cimport cydriver
-from cuda.core.experimental._utils.cuda_utils cimport HANDLE_RETURN
+from cuda.core._utils.cuda_utils cimport HANDLE_RETURN
 
 import threading
 from typing import Optional, TYPE_CHECKING, Union
 
-from cuda.core.experimental._context import Context, ContextOptions
-from cuda.core.experimental._event import Event, EventOptions
-from cuda.core.experimental._graph import GraphBuilder
-from cuda.core.experimental._stream import IsStreamT, Stream, StreamOptions
-from cuda.core.experimental._utils.clear_error_support import assert_type
-from cuda.core.experimental._utils.cuda_utils import (
+from cuda.core._context import Context, ContextOptions
+from cuda.core._event import Event, EventOptions
+from cuda.core._graph import GraphBuilder
+from cuda.core._stream import IsStreamT, Stream, StreamOptions
+from cuda.core._utils.clear_error_support import assert_type
+from cuda.core._utils.cuda_utils import (
     ComputeCapability,
     CUDAError,
     driver,
     handle_return,
     runtime,
 )
-from cuda.core.experimental._stream cimport default_stream
+from cuda.core._stream cimport default_stream
 
 if TYPE_CHECKING:
-    from cuda.core.experimental._memory import Buffer, MemoryResource
+    from cuda.core._memory import Buffer, MemoryResource
 
 # TODO: I prefer to type these as "cdef object" and avoid accessing them from within Python,
 # but it seems it is very convenient to expose them for testing purposes...
@@ -1034,7 +1034,7 @@ class Device:
         tuple of Device
             A tuple containing instances of available devices.
         """
-        from cuda.core.experimental import system
+        from cuda.core import system
         total = system.get_num_devices()
         return tuple(cls(device_id) for device_id in range(total))
 
@@ -1168,17 +1168,17 @@ class Device:
                     )
                 )
             if attr == 1:
-                from cuda.core.experimental._memory import DeviceMemoryResource
+                from cuda.core._memory import DeviceMemoryResource
                 self._memory_resource = DeviceMemoryResource(self._id)
             else:
-                from cuda.core.experimental._memory import _SynchronousMemoryResource
+                from cuda.core._memory import _SynchronousMemoryResource
                 self._memory_resource = _SynchronousMemoryResource(self._id)
 
         return self._memory_resource
 
     @memory_resource.setter
     def memory_resource(self, mr):
-        from cuda.core.experimental._memory import MemoryResource
+        from cuda.core._memory import MemoryResource
         assert_type(mr, MemoryResource)
         self._memory_resource = mr
 
@@ -1237,7 +1237,7 @@ class Device:
         Acts as an entry point of this object. Users always start a code by
         calling this method, e.g.
 
-        >>> from cuda.core.experimental import Device
+        >>> from cuda.core import Device
         >>> dev0 = Device(0)
         >>> dev0.set_current()
         >>> # ... do work on device 0 ...

…_core/cuda/core/experimental/_dlpack.pxd cuda_core/cuda/core/_dlpack.pxdcuda_core/cuda/core/experimental/_dlpack.pxd renamed to cuda_core/cuda/core/_dlpack.pxd cuda_core/cuda/core/experimental/_dlpack.pxd renamed to cuda_core/cuda/core/_dlpack.pxd

@@ -14,7 +14,7 @@ from libc.stdint cimport uint64_t
 from libc.stdint cimport intptr_t
 
 
-cdef extern from "include/dlpack.h" nogil:
+cdef extern from "_include/dlpack.h" nogil:
     """
     #define DLPACK_TENSOR_UNUSED_NAME "dltensor"
     #define DLPACK_VERSIONED_TENSOR_UNUSED_NAME "dltensor_versioned"