Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/developers/dynamic-mig.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ This feature will not be implemented without the help of @sailorvii.

## Introduction

The NVIDIA GPU built-in sharing method includes: time-slice, MPS and MIG. The context switch for time slice sharing would waste some time, so MPS and MIG are preferred. The GPU MIG profile is variable, the user could acquire the MIG device in the profile definition, but current implementation only defines the dedicated profile before the user requirement. That limits the usage of MIG. The goal is to develop an automatic slice plugin and create the slice when the user requires it. For the scheduling method, node-level binpack and spread will be supported. Referring to the binpack plugin, the scheduler considers CPU, Mem, GPU memory and other user-defined resources. HAMi is done by using [hami-core](https://github.com/Project-HAMi/HAMi-core), which is a cuda-hacking library. But mig is also widely used across the world. A unified API for dynamic-mig and hami-core is needed.
The NVIDIA GPU built-in sharing method includes: time-slice, MPS and MIG. The context switch for time slice sharing would waste some time, so MPS and MIG are preferred. The GPU MIG profile is variable, the user could acquire the MIG device in the profile definition, but current implementation only defines the dedicated profile before the user requirement. That limits the usage of MIG. The goal is to develop an automatic slice plugin and create the slice when the user requires it. For the scheduling method, node-level binpack and spread will be supported. Referring to the binpack plugin, the scheduler considers CPU, Mem, GPU memory and other user-defined resources. HAMi is done by using [hami-core](https://github.com/Project-HAMi/HAMi-core), which is a cuda-hacking library. But MIG is also widely used across the world. A unified API for dynamic-mig and hami-core is needed.

## Targets

Expand Down
2 changes: 1 addition & 1 deletion docs/developers/protocol.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ hami.io/node-nvidia-register: GPU-00552014-5c87-89ac-b1a6-7b53aa24b0ec,10,32768,

```

In this example, this node has two different AI devices, 2 Nvidia-V100 GPUs, and 2 Cambricon 370-X4 MLUs
In this example, this node has two different AI devices, 2 NVIDIA-V100 GPUs, and 2 Cambricon 370-X4 MLUs

A device node may become unavailable due to hardware or network failure. If a node hasn't registered in the last 5 minutes, the scheduler marks it as 'unavailable'.

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: Assign task to mig instance
title: Assign task to MIG instance
---

This example will allocate `2g.10gb * 2` for A100-40GB-PCIE device or `1g.10gb * 2` for A100-80GB-SXM device.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@ translated: true

### 介绍

NVIDIA GPU 内置的共享方法包括:时间片、MPS 和 MIG。时间片共享的上下文切换会浪费一些时间,所以我们选择了 MPS 和 MIG。GPU MIG 配置是可变的,用户可以在配置定义中获取 MIG 设备,但当前实现仅在用户需求之前定义了专用配置。这限制了 MIG 的使用。我们希望开发一个自动切片插件,并在用户需要时创建切片。对于调度方法,将支持节点级别的 binpack 和 spread。参考 binpack 插件,我们考虑了 CPU、内存、GPU 显存和其他用户定义的资源。HAMi 是通过使用 [hami-core](https://github.com/Project-HAMi/HAMi-core) 完成的,这是一个 cuda-hacking 库。但 mig 在全球范围内也被广泛使用。需要一个用于动态-mig 和 hami-core 的统一 API。
NVIDIA GPU 内置的共享方法包括:时间片、MPS 和 MIG。时间片共享的上下文切换会浪费一些时间,所以我们选择了 MPS 和 MIG。GPU MIG 配置是可变的,用户可以在配置定义中获取 MIG 设备,但当前实现仅在用户需求之前定义了专用配置。这限制了 MIG 的使用。我们希望开发一个自动切片插件,并在用户需要时创建切片。对于调度方法,将支持节点级别的 binpack 和 spread。参考 binpack 插件,我们考虑了 CPU、内存、GPU 显存和其他用户定义的资源。HAMi 是通过使用 [hami-core](https://github.com/Project-HAMi/HAMi-core) 完成的,这是一个 cuda-hacking 库。但 MIG 在全球范围内也被广泛使用。需要一个用于动态-mig 和 hami-core 的统一 API。

## 目标

- CPU、内存和 GPU 组合调度
- GPU 动态切片:Hami-core 和 MIG
- GPU 动态切片:HAMi-core 和 MIG
- 支持通过 GPU 显存、CPU 和显存的节点级别 binpack 和 spread
- 不同虚拟化技术的统一 vGPU 池
- 任务可以选择使用 MIG、使用 HAMi-core 或同时使用两者。
Expand Down Expand Up @@ -101,7 +101,7 @@ data:

## 示例

动态 mighami 任务兼容,如下例所示:只需设置 `nvidia.com/gpu` 和 `nvidia.com/gpumem`。
动态 MIGHAMi 任务兼容,如下例所示:只需设置 `nvidia.com/gpu` 和 `nvidia.com/gpumem`。

```yaml
apiVersion: v1
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@ translated: true

:::note

使用 volcano-vgpu 时,你*不需要*安装 HAMi,只需使用 [Volcano vgpu 设备插件](https://github.com/Project-HAMi/volcano-vgpu-device-plugin) 就足够了。它可以为由 volcano 管理的 NVIDIA 设备提供设备共享机制。
使用 volcano-vgpu 时,你*不需要*安装 HAMi,只需使用 [Volcano vGPU 设备插件](https://github.com/Project-HAMi/volcano-vgpu-device-plugin) 就足够了。它可以为由 volcano 管理的 NVIDIA 设备提供设备共享机制。

这基于 [Nvidia 设备插件](https://github.com/NVIDIA/k8s-device-plugin),使用 [HAMi-core](https://github.com/Project-HAMi/HAMi-core) 支持 GPU 卡的硬隔离。
这基于 [NVIDIA 设备插件](https://github.com/NVIDIA/k8s-device-plugin),使用 [HAMi-core](https://github.com/Project-HAMi/HAMi-core) 支持 GPU 卡的硬隔离。

Volcano vgpu 仅在 volcano > 1.9 中可用
Volcano vGPU 仅在 volcano > 1.9 中可用

:::

Expand Down Expand Up @@ -89,9 +89,9 @@ status:
volcano.sh/gpu-number: "10" # vGPU 资源
```

### 运行 VGPU 作业
### 运行 vGPU 作业

可以通过在 resource.limit 中设置 "volcano.sh/vgpu-number"、"volcano.sh/vgpu-cores" 和 "volcano.sh/vgpu-memory" 来请求 VGPU
可以通过在 resource.limit 中设置 "volcano.sh/vgpu-number"、"volcano.sh/vgpu-cores" 和 "volcano.sh/vgpu-memory" 来请求 vGPU

```shell script
$ cat <<EOF | kubectl apply -f -
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,15 @@ translated: true

## 介绍

HAMi 已经在 NVIDIA 设备上支持了 K8s [DRA](https://kubernetes.io/docs/concepts/scheduling-eviction/dynamic-resource-allocation/)(动态资源分配)功能。通过安装 hami-k8s-dra-driver 你的集群调度器可以发现节点上的 Nvidia GPU 设备。
HAMi 已经在 NVIDIA 设备上支持了 K8s [DRA](https://kubernetes.io/docs/concepts/scheduling-eviction/dynamic-resource-allocation/)(动态资源分配)功能。通过安装 hami-k8s-dra-driver 你的集群调度器可以发现节点上的 NVIDIA GPU 设备。

## 前提条件

- 底层容器运行时(例如 containerd 或 CRI-O)启用 [CDI](https://github.com/cncf-tags/container-device-interface?tab=readme-ov-file#how-to-configure-cdi)

## 安装

Nvidia dra driver 内置在 HAMi 中,无需单独安装,只需要在[安装 HAMi DRA](../../installation/how-to-use-hami-dra) 时指定 `--set hami-dra-webhook.drivers.nvidia.enabled=true` 参数即可。更多信息参考[安装 Nvidia DRA driver](https://github.com/Project-HAMi/HAMi-DRA?tab=readme-ov-file#installation)
NVIDIA DRA driver 内置在 HAMi 中,无需单独安装,只需要在[安装 HAMi DRA](../../installation/how-to-use-hami-dra) 时指定 `--set hami-dra-webhook.drivers.nvidia.enabled=true` 参数即可。更多信息参考[安装 NVIDIA DRA driver](https://github.com/Project-HAMi/HAMi-DRA?tab=readme-ov-file#installation)

## 验证安装

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: 将任务分配给 mig 实例
title: 将任务分配给 MIG 实例
translated: true
---

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: 默认 vgpu 作业
title: 默认 vGPU 作业
sidebar_label: 默认作业
translated: true
---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: Volcano vgpu 设备插件用于 Kubernetes
title: Volcano vGPU 设备插件用于 Kubernetes
sidebar_label: 如何使用 Volcano vGPU
translated: true
---
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ HAMi is done by using [hami-core](https://github.com/Project-HAMi/HAMi-core), wh
## Targets

- CPU, Mem, and GPU combined schedule
- GPU dynamic slice: Hami-core and MIG
- GPU dynamic slice: HAMi-core and MIG
- Support node-level binpack and spread by GPU memory, CPU and Mem
- A unified vGPU Pool different virtualization techniques
- Tasks can choose to use MIG, use HAMi-core, or use both.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ title: Deploy HAMi using Helm

This guide will cover:

- Configure nvidia container runtime in each GPU nodes
- Configure NVIDIA container runtime in each GPU nodes
- Install HAMi using helm
- Launch a vGPU task
- Check if the corresponding device resources are limited inside container
Expand Down
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
---
title: How to use volcano vgpu
title: How to use volcano vGPU
---

## Volcano vgpu device plugin for Kubernetes
## Volcano vGPU device plugin for Kubernetes

**Note**:

You *DON'T* need to install HAMi when using volcano-vgpu, only use
[Volcano vgpu device-plugin](https://github.com/Project-HAMi/volcano-vgpu-device-plugin) is good enough. It can provide device-sharing mechanism for NVIDIA devices managed by volcano.
[Volcano vGPU device-plugin](https://github.com/Project-HAMi/volcano-vgpu-device-plugin) is good enough. It can provide device-sharing mechanism for NVIDIA devices managed by volcano.

This is based on [NVIDIA Device Plugin](https://github.com/NVIDIA/k8s-device-plugin), it uses [HAMi-core](https://github.com/Project-HAMi/HAMi-core) to support hard isolation of GPU card.

Volcano vgpu is only available in volcano > 1.9
Volcano vGPU is only available in volcano > 1.9

## Quick Start

Expand Down Expand Up @@ -89,9 +89,9 @@ status:
volcano.sh/gpu-number: "10" # vGPU resource
```

### Running VGPU Jobs
### Running vGPU Jobs

VGPU can be requested by both set "volcano.sh/vgpu-number" , "volcano.sh/vgpu-cores" and "volcano.sh/vgpu-memory" in resource.limit
vGPU can be requested by both set "volcano.sh/vgpu-number" , "volcano.sh/vgpu-cores" and "volcano.sh/vgpu-memory" in resource.limit

```shell script
$ cat <<EOF | kubectl apply -f -
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: Uninstall
---

The step to uninstall hami is simple:
The step to uninstall HAMi is simple:

```bash
helm uninstall hami -n kube-system
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -51,11 +51,11 @@ kubectl describe cm hami-device-plugin -n kube-system
- hami-scheduler
- hami-device-plugin on 'MIG-NODE-A'

## Custom mig configuration (Optional)
## Custom MIG configuration (Optional)

HAMi currently has a [built-in mig configuration](https://github.com/Project-HAMi/HAMi/blob/master/charts/hami/templates/scheduler/device-configmap.yaml) for MIG.
HAMi currently has a [built-in MIG configuration](https://github.com/Project-HAMi/HAMi/blob/master/charts/hami/templates/scheduler/device-configmap.yaml) for MIG.

You can customize the mig configuration by following the steps below:
You can customize the MIG configuration by following the steps below:

### Change the content of 'device-configmap.yaml' in charts/hami/templates/scheduler, the as follows

Expand Down Expand Up @@ -167,7 +167,7 @@ spec:
nvidia.com/gpumem: 8000
```

In this example above, the task allocates two mig instances, each with at least 8G device memory.
In this example above, the task allocates two MIG instances, each with at least 8G device memory.

## Monitor MIG Instance

Expand All @@ -186,6 +186,6 @@ nodeGPUMigInstance{deviceidx="1",deviceuuid="GPU-30f90f49-43ab-0a78-bf5c-93ed41e

1. You don't need to do anything on MIG node, all are managed by mig-parted in hami-device-plugin.

2. Nvidia devices before Ampere architect can't use 'mig' mode
2. NVIDIA devices before Ampere architect can't use 'mig' mode

3. You won't see any mig resources(ie, `nvidia.com/mig-1g.10gb`) on node, hami uses a unified resource name for both 'mig' and 'hami-core' node
3. You won't see any MIG resources(ie, `nvidia.com/mig-1g.10gb`) on node, HAMi uses a unified resource name for both 'mig' and 'hami-core' node
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: Assign task to mig instance
title: Assign task to MIG instance
---

## This example will allocate 2g.10gb *2 for A100-40GB-PCIE device or 1g.10gb* 2 for A100-80GB-SXM device
Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
title: Default vgpu job
title: Default vGPU job
---

## Job description

VGPU can be requested by both set "volcano.sh/vgpu-number" , "volcano.sh/vgpu-cores" and "volcano.sh/vgpu-memory" in resource.limit
vGPU can be requested by both set "volcano.sh/vgpu-number" , "volcano.sh/vgpu-cores" and "volcano.sh/vgpu-memory" in resource.limit

```yaml
apiVersion: v1
Expand Down
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
---
title: How to use volcano vgpu
title: How to use volcano vGPU
---

## Volcano vgpu device plugin for Kubernetes
## Volcano vGPU device plugin for Kubernetes

**Note**:

You *DON'T* need to install HAMi when using volcano-vgpu, only use
[Volcano vgpu device-plugin](https://github.com/Project-HAMi/volcano-vgpu-device-plugin) is good enough. It can provide device-sharing mechanism for NVIDIA devices managed by volcano.
[Volcano vGPU device-plugin](https://github.com/Project-HAMi/volcano-vgpu-device-plugin) is good enough. It can provide device-sharing mechanism for NVIDIA devices managed by volcano.

This is based on [NVIDIA Device Plugin](https://github.com/NVIDIA/k8s-device-plugin), it uses [HAMi-core](https://github.com/Project-HAMi/HAMi-core) to support hard isolation of GPU card.

Volcano vgpu is only available in volcano > 1.9
Volcano vGPU is only available in volcano > 1.9

## Quick Start

Expand Down Expand Up @@ -89,9 +89,9 @@ status:
volcano.sh/gpu-number: "10" # vGPU resource
```

### Running VGPU Jobs
### Running vGPU Jobs

VGPU can be requested by both set "volcano.sh/vgpu-number" , "volcano.sh/vgpu-cores" and "volcano.sh/vgpu-memory" in resource.limit
vGPU can be requested by both set "volcano.sh/vgpu-number" , "volcano.sh/vgpu-cores" and "volcano.sh/vgpu-memory" in resource.limit

```shell script
$ cat <<EOF | kubectl apply -f -
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ It contains the following metrics:
| Metrics | Description | Example |
|----------|-------------|---------|
| volcano_vgpu_device_allocated_cores | The percentage of gpu compute cores allocated in this card | `{NodeName="aio-node67",devID="GPU-00552014-5c87-89ac-b1a6-7b53aa24b0ec"}` 0 |
| volcano_vgpu_device_allocated_memory | Vgpu memory allocated in this card | `{NodeName="aio-node67",devID="GPU-00552014-5c87-89ac-b1a6-7b53aa24b0ec"}` 32768|
| volcano_vgpu_device_allocated_memory | vGPU memory allocated in this card | `{NodeName="aio-node67",devID="GPU-00552014-5c87-89ac-b1a6-7b53aa24b0ec"}` 32768|
| volcano_vgpu_device_core_allocation_for_a_vertain_pod| The vgpu device core allocated for a certain pod | `{NodeName="aio-node67",devID="GPU-00552014-5c87-89ac-b1a6-7b53aa24b0ec",podName="resnet101-deployment-7b487d974d-jjc8p"}` 0|
| volcano_vgpu_device_memory_allocation_for_a_certain_pod | The vgpu device memory allocated for a certain pod | `{NodeName="aio-node67",devID="GPU-00552014-5c87-89ac-b1a6-7b53aa24b0ec",podName="resnet101-deployment-7b487d974d-jjc8p"}` 16384 |
| volcano_vgpu_device_memory_limit | The number of total device memory in this card | `{NodeName="m5-cloudinfra-online01",devID="GPU-a88b5d0e-eb85-924b-b3cd-c6cad732f745"}` 32768 |
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ title: Deploy HAMi using Helm

This guide will cover:

- Configure nvidia container runtime in each GPU nodes
- Configure NVIDIA container runtime in each GPU nodes
- Install HAMi using helm
- Launch a vGPU task
- Check if the corresponding device resources are limited inside container
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: Uninstall
---

The step to uninstall hami is simple:
The step to uninstall HAMi is simple:

```bash
helm uninstall hami -n kube-system
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,12 @@ translated: true

NVIDIA GPU 内置的共享方法包括:时间片、MPS 和 MIG。时间片共享的上下文切换会浪费一些时间,所以我们选择了 MPS 和 MIG。GPU MIG 配置是可变的,用户可以在配置定义中获取 MIG 设备,但当前实现仅在用户需求之前定义了专用配置。这限制了 MIG 的使用。我们希望开发一个自动切片插件,并在用户需要时创建切片。
对于调度方法,将支持节点级别的 binpack 和 spread。参考 binpack 插件,我们考虑了 CPU、内存、GPU 显存和其他用户定义的资源。
HAMi 是通过使用 [hami-core](https://github.com/Project-HAMi/HAMi-core) 完成的,这是一个 cuda-hacking 库。但 mig 在全球范围内也被广泛使用。需要一个用于动态-mig 和 hami-core 的统一 API。
HAMi 是通过使用 [hami-core](https://github.com/Project-HAMi/HAMi-core) 完成的,这是一个 cuda-hacking 库。但 MIG 在全球范围内也被广泛使用。需要一个用于动态-mig 和 hami-core 的统一 API。

## 目标

- CPU、内存和 GPU 组合调度
- GPU 动态切片:Hami-core 和 MIG
- GPU 动态切片:HAMi-core 和 MIG
- 支持通过 GPU 显存、CPU 和显存的节点级别 binpack 和 spread
- 不同虚拟化技术的统一 vGPU 池
- 任务可以选择使用 MIG、使用 HAMi-core 或同时使用两者。
Expand Down Expand Up @@ -104,7 +104,7 @@ data:

## 示例

动态 mighami 任务兼容,如下例所示:
动态 MIGHAMi 任务兼容,如下例所示:
只需设置 `nvidia.com/gpu` 和 `nvidia.com/gpumem`。

```yaml
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ translated: true

本指南将涵盖:

- 在每个 GPU 节点中配置 nvidia 容器运行时
- 在每个 GPU 节点中配置 NVIDIA 容器运行时
- 使用 helm 安装 HAMi
- 启动 vGPU 任务
- 检查容器内相应的设备资源是否受限
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ translated: true
使用 volcano-vgpu 时,你*不需要*安装 HAMi,只需使用
[Volcano vGPU 设备插件](https://github.com/Project-HAMi/volcano-vgpu-device-plugin) 就足够了。它可以为由 Volcano 管理的 NVIDIA 设备提供设备共享机制。

这基于 [Nvidia 设备插件](https://github.com/NVIDIA/k8s-device-plugin),使用 [HAMi-core](https://github.com/Project-HAMi/HAMi-core) 支持 GPU 卡的硬隔离。
这基于 [NVIDIA 设备插件](https://github.com/NVIDIA/k8s-device-plugin),使用 [HAMi-core](https://github.com/Project-HAMi/HAMi-core) 支持 GPU 卡的硬隔离。

Volcano vGPU 仅在 Volcano > v1.9 中可用。

Expand Down Expand Up @@ -92,7 +92,7 @@ status:
volcano.sh/gpu-number: "10" # vGPU 资源
```

### 运行 VGPU 作业
### 运行 vGPU 作业

可以通过在 resources.limits 中设置 "volcano.sh/vgpu-number"、"volcano.sh/vgpu-cores" 和 "volcano.sh/vgpu-memory" 来请求 vGPU。

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ nodeGPUMigInstance{deviceidx="1",deviceuuid="GPU-30f90f49-43ab-0a78-bf5c-93ed41e

1. 你无需在 MIG 节点上执行任何操作,所有操作均由 hami-device-plugin 中的 mig-parted 管理。

2. Ampere 架构之前的 Nvidia 设备无法使用 MIG 模式
2. Ampere 架构之前的 NVIDIA 设备无法使用 MIG 模式

3. 你不会在节点上看到任何 MIG 资源(即 `nvidia.com/mig-1g.10gb`),HAMi 对 MIG 和 hami-core 节点使用统一的资源名称。

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: 将任务分配给 mig 实例
title: 将任务分配给 MIG 实例
translated: true
---

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ translated: true
使用 volcano-vgpu 时,你*不需要*安装 HAMi,只需使用
[Volcano vGPU 设备插件](https://github.com/Project-HAMi/volcano-vgpu-device-plugin)就足够了。它可以为由 Volcano 管理的 NVIDIA 设备提供设备共享机制。

这基于 [Nvidia 设备插件](https://github.com/NVIDIA/k8s-device-plugin),使用 [HAMi-core](https://github.com/Project-HAMi/HAMi-core) 支持 GPU 卡的硬隔离。
这基于 [NVIDIA 设备插件](https://github.com/NVIDIA/k8s-device-plugin),使用 [HAMi-core](https://github.com/Project-HAMi/HAMi-core) 支持 GPU 卡的硬隔离。

Volcano vGPU 仅在 Volcano > v1.9 中可用

Expand Down
Loading