Skip to content

Commit 8923b73

Browse files
committed
[Docs] Clarify how K8s resources and offers work
1 parent 3c43b25 commit 8923b73

1 file changed

Lines changed: 58 additions & 1 deletion

File tree

docs/docs/concepts/backends.md

Lines changed: 58 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1060,7 +1060,64 @@ projects:
10601060

10611061
Ensure you've created a ClusterRoleBinding to grant the role to the user or the service account you're using.
10621062

1063-
> To learn more, see the [Lambda](../../examples/clusters/lambda/#kubernetes) and [Lambda](../../examples/clusters/crusoe/#kubernetes) examples.
1063+
??? info "Resources and offers"
1064+
[Resources](../concepts/tasks.md#resources) specified in the run configuration are translated to Kubernetes
1065+
[requests and limits](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#requests-and-limits) as follows:
1066+
1067+
- As with other backends, an exact value is translated to a range with the same lower and upper limits,
1068+
e.g., `cpu: 4` is the same as `cpu: 4..4`.
1069+
- The lower limit, if set, is used as a resource request, meaning that it is guaranteed that the container has at least the specified
1070+
amount of the resource.
1071+
- For resources other than `gpu`, the upper limit, if set, is used as a resource limit, meaning that the container is not allowed
1072+
to consume more resources than specified. If the upper limit is not set, the resource limit is also not set.
1073+
- For `gpu` resources, the upper limit is always ignored, and the resource limit is always set to the same value as the resource request,
1074+
that is, to the lower limit of the range.
1075+
1076+
For example, the following resources specification:
1077+
1078+
<div editor-title=".dstack.yml">
1079+
1080+
```yaml
1081+
resources:
1082+
cpu: 32..64
1083+
memory: 1024GB
1084+
disk: 100GB..
1085+
gpu: nvidia:4..8
1086+
```
1087+
1088+
</div>
1089+
1090+
is translated to:
1091+
1092+
| resource | request | limit |
1093+
|---------------------|----------|-----------|
1094+
| `cpu` | `32` | `64` |
1095+
| `memory` | `1024Gi` | `1024Gi` |
1096+
| `ephemeral-storage` | `100Gi` | _not set_ |
1097+
| `nvidia.com/gpu` | `4` | `4` |
1098+
1099+
In offers, `dstack` uses resource requests as offer's resources.
1100+
With the resources spec as in the example above, offers would look like the following,
1101+
even if nodes have more available resources:
1102+
1103+
```
1104+
# BACKEND RESOURCES INSTANCE TYPE PRICE
1105+
1 kubernetes (-) cpu=32 mem=1024GB disk=100GB H100:80GB:4 h100x8 $0
1106+
```
1107+
1108+
As a consequence, if you specify `gpu: 0` or don't specify `gpu` at all (the default value is `0`),
1109+
you won't see GPU resources in offers, even with GPU nodes. The same is true for the `dstack offer` command,
1110+
to see available GPU models, you should specify a miminum amount of GPUs > 0:
1111+
1112+
<div class="termy">
1113+
1114+
```shell
1115+
$ dstack offer --gpu 1
1116+
```
1117+
1118+
</div>
1119+
1120+
> To learn more, see the [Lambda](../../examples/clusters/lambda/#kubernetes) and [Crusoe](../../examples/clusters/crusoe/#kubernetes) examples.
10641121

10651122
### RunPod
10661123

0 commit comments

Comments
 (0)