Migrate from pynvml to cuda.core.system

[mdboom]

This migrates from pynvml to the new Cython/cybind-based bindings for NVML in cuda.core.system.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[pentschev]

/ok to test 1306c08

[TomAugspurger]

/ok to test 00453df

[TomAugspurger]

Looks nice overall.

IMO, the changes to dask_cuda.utils (removal of unpack_bitmask, change in return types) aren't worth trying to deprecate.

[pentschev]

IMO, the changes to dask_cuda.utils (removal of unpack_bitmask, change in return types) aren't worth trying to deprecate.

Yes, absolutely not worth it, just removing them (as is being proposed) is enough.

@mdboom it seems you're still missing some cuda-core pinning changes:

• dask-cuda/dependencies.yaml

  Lines 257 to 266 in c0f9a3f

  	extra_cu12:
  	common:
  	- output_types: [pyproject]
  	packages:
  	- cuda-core[cu12]>=0.3.2
  	extra_cu13:
  	common:
  	- output_types: [pyproject]
  	packages:
  	- cuda-core[cu13]>=0.3.2

• dask-cuda/conda/recipes/dask-cuda/recipe.yaml

  Line 40 in c0f9a3f

  - cuda-core >=0.3

-And since we're upgrading the version of cuda-core it would be nice to properly handle (i.e., remove) previous compatibility changes:

• dask-cuda/dask_cuda/initialize.py

  Lines 12 to 15 in c0f9a3f

  	# Remove when cuda-core>=0.5
  	import cuda.core.experimental
  	Device = cuda.core.experimental.Device

• dask-cuda/dask_cuda/_compat.py

  Lines 22 to 29 in c0f9a3f

  	@functools.lru_cache(maxsize=None)
  	def get_cuda_core_version() -> packaging.version.Version:
  	return packaging.version.parse(importlib.metadata.version("cuda-core"))
  	@functools.lru_cache(maxsize=None)
  	def CUDA_CORE_0_5_0():
  	return get_cuda_core_version() >= packaging.version.parse("0.5.0")

As well as in the following test files:

• https://github.com/rapidsai/dask-cuda/blob/main/dask_cuda/tests/test_initialize.py
• https://github.com/rapidsai/dask-cuda/blob/main/dask_cuda/tests/test_utils.py

Could you do that?

[mdboom]

/ok to test 97bfb94

[pentschev]

/ok to test 97bfb94

[wence-]

/ok to test 3933543