Skip to content

Commit 6d61113

Browse files
authored
1 parent 165de64 commit 6d61113

File tree

15 files changed

+37
-37
lines changed

15 files changed

+37
-37
lines changed

docs/blog/posts/amd-mi300x-inference-benchmark.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -217,8 +217,8 @@ is the primary sponsor of this benchmark, and we are sincerely grateful for thei
217217
If you'd like to use top-tier bare metal compute with AMD GPUs, we recommend going
218218
with Hot Aisle. Once you gain access to a cluster, it can be easily accessed via `dstack`'s [SSH fleet](../../docs/concepts/fleets.md#ssh-fleets) easily.
219219

220-
### RunPod
220+
### Runpod
221221
If you’d like to use on-demand compute with AMD GPUs at affordable prices, you can configure `dstack` to
222-
use [RunPod](https://runpod.io/). In
222+
use [Runpod](https://runpod.io/). In
223223
this case, `dstack` will be able to provision fleets automatically when you run dev environments, tasks, and
224224
services.

docs/blog/posts/amd-on-runpod.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,25 @@
11
---
2-
title: Supporting AMD accelerators on RunPod
2+
title: Supporting AMD accelerators on Runpod
33
date: 2024-08-21
4-
description: "dstack, the open-source AI container orchestration platform, adds support for AMD accelerators, with RunPod as the first supported cloud provider."
4+
description: "dstack, the open-source AI container orchestration platform, adds support for AMD accelerators, with Runpod as the first supported cloud provider."
55
slug: amd-on-runpod
66
categories:
77
- Changelog
88
---
99

10-
# Supporting AMD accelerators on RunPod
10+
# Supporting AMD accelerators on Runpod
1111

1212
While `dstack` helps streamline the orchestration of containers for AI, its primary goal is to offer vendor independence
1313
and portability, ensuring compatibility across different hardware and cloud providers.
1414

15-
Inspired by the recent `MI300X` benchmarks, we are pleased to announce that RunPod is the first cloud provider to offer
15+
Inspired by the recent `MI300X` benchmarks, we are pleased to announce that Runpod is the first cloud provider to offer
1616
AMD GPUs through `dstack`, with support for other cloud providers and on-prem servers to follow.
1717

1818
<!-- more -->
1919

2020
## Specification
2121

22-
For the reference, below is a comparison of the `MI300X` and `H100 SXM` specs, incl. the prices offered by RunPod.
22+
For the reference, below is a comparison of the `MI300X` and `H100 SXM` specs, incl. the prices offered by Runpod.
2323

2424
| | MI300X | H100X SXM |
2525
|---------------------------------|-------------------------------------------|--------------|
@@ -113,8 +113,8 @@ cloud resources and run the configuration.
113113
1. The examples above demonstrate the use of
114114
[TGI](https://huggingface.co/docs/text-generation-inference/en/installation_amd).
115115
AMD accelerators can also be used with other frameworks like vLLM, Ollama, etc., and we'll be adding more examples soon.
116-
2. RunPod is the first cloud provider where dstack supports AMD. More cloud providers will be supported soon as well.
117-
3. Want to give RunPod and `dstack` a try? Make sure you've signed up for [RunPod](https://www.runpod.io/),
116+
2. Runpod is the first cloud provider where dstack supports AMD. More cloud providers will be supported soon as well.
117+
3. Want to give Runpod and `dstack` a try? Make sure you've signed up for [Runpod](https://www.runpod.io/),
118118
then [set up](../../docs/reference/server/config.yml.md#runpod) the `dstack server`.
119119

120120
> Have questioned or feedback? Join our [Discord](https://discord.gg/u8SmfwPpMd)

docs/blog/posts/beyond-kubernetes-2024-recap-and-whats-ahead.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,7 @@ While `dstack` integrates with leading cloud GPU providers, we aim to expand par
2222
sharing our vision of simplifying AI infrastructure orchestration with a lightweight, efficient alternative to Kubernetes.
2323

2424
This year, we’re excited to welcome our first partners: [Lambda](https://lambdalabs.com/),
25-
[RunPod](https://www.runpod.io/),
25+
[Runpod](https://www.runpod.io/),
2626
[CUDO Compute](https://www.cudocompute.com/),
2727
and [Hot Aisle](https://hotaisle.xyz/).
2828

@@ -114,7 +114,7 @@ This year, we’re particularly proud of our newly added integration with AMD.
114114
`dstack` works seamlessly with any on-prem AMD clusters. For example, you can rent such servers through our partner
115115
[Hot Aisle](https://hotaisle.xyz/).
116116

117-
> Among cloud providers, [AMD](https://www.amd.com/en/products/accelerators/instinct.html) is supported only through RunPod. In Q1 2025, we plan to extend it to
117+
> Among cloud providers, [AMD](https://www.amd.com/en/products/accelerators/instinct.html) is supported only through Runpod. In Q1 2025, we plan to extend it to
118118
[Nscale](https://www.nscale.com/),
119119
> [Hot Aisle](https://hotaisle.xyz/), and potentially other providers open to collaboration.
120120

docs/blog/posts/dstack-sky-own-cloud-accounts.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ To use your own cloud account, open the project settings and edit the correspond
2525
![dstack-sky-banner.png](https://raw.githubusercontent.com/dstackai/static-assets/main/static-assets/images/dstack-sky-edit-backend-config.png){ width=650 }
2626

2727
You can configure your cloud accounts for any of the supported providers, including AWS, GCP, Azure, TensorDock, Lambda,
28-
CUDO, RunPod, and Vast.ai.
28+
CUDO, Runpod, and Vast.ai.
2929

3030
Additionally, you can disable certain backends if you do not plan to use them.
3131

docs/blog/posts/state-of-cloud-gpu-2025.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ These axes split providers into distinct archetypes—each with different econom
2828
| :---- | :---- | :---- |
2929
| **Classical hyperscalers** | General-purpose clouds with GPU SKUs bolted on | AWS, Google Cloud, Azure, OCI |
3030
| **Massive neoclouds** | GPU-first operators built around dense HGX or MI-series clusters | CoreWeave, Lambda, Nebius, Crusoe |
31-
| **Rapidly-catching neoclouds** | Smaller GPU-first players building out aggressively | RunPod, DataCrunch, Voltage Park, TensorWave, Hot Aisle |
31+
| **Rapidly-catching neoclouds** | Smaller GPU-first players building out aggressively | Runpod, DataCrunch, Voltage Park, TensorWave, Hot Aisle |
3232
| **Cloud marketplaces** | Don’t own capacity; sell orchestration + unified API over multiple backends | NVIDIA DGX Cloud (Lepton), Modal, Lightning AI, dstack Sky |
3333
| **DC aggregators** | Aggregate idle capacity from third-party datacenters, pricing via market dynamics | Vast.ai |
3434

@@ -89,7 +89,7 @@ For comparison, below is the price range for H100×GPU clusters across providers
8989

9090
<img src="https://dstack.ai/static-assets/static-assets/images/cloud-providers-cluster-h100.png" width="750"/>
9191

92-
> Most hyperscalers and neoclouds need short- or long-term contracts, though providers like RunPod, DataCrunch, and Nebius offer on-demand clusters. Larger capacity and longer commitments bring bigger discounts — Nebius offers up to 35% off for longer terms.
92+
> Most hyperscalers and neoclouds need short- or long-term contracts, though providers like Runpod, DataCrunch, and Nebius offer on-demand clusters. Larger capacity and longer commitments bring bigger discounts — Nebius offers up to 35% off for longer terms.
9393
9494
## New GPU generations – why they matter
9595

docs/blog/posts/toffee.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ In a recent engineering [blog post](https://research.toffee.ai/blog/how-we-use-d
2020

2121
[Toffee](https://toffee.ai) builds AI-powered experiences backed by LLMs and image-generation models. To serve these workloads efficiently, they combine:
2222

23-
- **GPU neoclouds** such as [RunPod](https://www.runpod.io/) and [Vast.ai](https://vast.ai/) for flexible, cost-efficient GPU capacity
23+
- **GPU neoclouds** such as [Runpod](https://www.runpod.io/) and [Vast.ai](https://vast.ai/) for flexible, cost-efficient GPU capacity
2424
- **AWS** for core, non-AI services and backend infrastructure
2525
- **dstack** as the orchestration layer that provisions GPU resources and exposes AI models via `dstack` [services](../../docs/concepts/services.md) and [gateways](../../docs/concepts/gateways.md)
2626

@@ -68,7 +68,7 @@ Beyond oechestration, Toffee relies on `dstack`’s UI as a central observabilit
6868

6969
<img src="https://dstack.ai/static-assets/static-assets/images/toffee-metrics-dark.png" width="750" />
7070

71-
> *Thanks to dstack’s seamless integration with GPU neoclouds like RunPod and Vast.ai, we’ve been able to shift most workloads off hyperscalers — reducing our effective GPU spend by roughly 2–3× without changing a single line of model code.*
71+
> *Thanks to dstack’s seamless integration with GPU neoclouds like Runpod and Vast.ai, we’ve been able to shift most workloads off hyperscalers — reducing our effective GPU spend by roughly 2–3× without changing a single line of model code.*
7272
>
7373
> *[Nikita Shupeyko](https://www.linkedin.com/in/nikita-shupeyko/), AI/ML & Cloud Infrastructure Architect at Toffee*
7474

docs/blog/posts/volumes-on-runpod.md

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,24 @@
11
---
2-
title: Using volumes to optimize cold starts on RunPod
2+
title: Using volumes to optimize cold starts on Runpod
33
date: 2024-08-13
4-
description: "Learn how to use volumes with dstack to optimize model inference cold start times on RunPod."
4+
description: "Learn how to use volumes with dstack to optimize model inference cold start times on Runpod."
55
slug: volumes-on-runpod
66
categories:
77
- Changelog
88
---
99

10-
# Using volumes to optimize cold starts on RunPod
10+
# Using volumes to optimize cold starts on Runpod
1111

1212
Deploying custom models in the cloud often faces the challenge of cold start times, including the time to provision a
1313
new instance and download the model. This is especially relevant for services with autoscaling when new model replicas
1414
need to be provisioned quickly.
1515

1616
Let's explore how `dstack` optimizes this process using volumes, with an example of
17-
deploying a model on RunPod.
17+
deploying a model on Runpod.
1818

1919
<!-- more -->
2020

21-
Suppose you want to deploy Llama 3.1 on RunPod as a [service](../../docs/concepts/services.md):
21+
Suppose you want to deploy Llama 3.1 on Runpod as a [service](../../docs/concepts/services.md):
2222

2323
<div editor-title="examples/llms/llama31/tgi/service.dstack.yml">
2424

@@ -59,9 +59,9 @@ When starting each replica, `text-generation-launcher` downloads the model to th
5959
usually takes under a minute, but larger models may take longer. Repeated downloads can significantly affect
6060
auto-scaling efficiency.
6161

62-
Great news: RunPod supports network volumes, which we can use for caching models across multiple replicas.
62+
Great news: Runpod supports network volumes, which we can use for caching models across multiple replicas.
6363

64-
With `dstack`, you can create a RunPod volume using the following configuration:
64+
With `dstack`, you can create a Runpod volume using the following configuration:
6565

6666
<div editor-title="examples/mist/volumes/runpod.dstack.yml">
6767

@@ -130,7 +130,7 @@ resources:
130130
In this case, `dstack` attaches the specified volume to each new replica. This ensures the model is downloaded only
131131
once, reducing cold start time in proportion to the model size.
132132

133-
A notable feature of RunPod is that volumes can be attached to multiple containers simultaneously. This capability is
133+
A notable feature of Runpod is that volumes can be attached to multiple containers simultaneously. This capability is
134134
particularly useful for auto-scalable services or distributed tasks.
135135

136136
Using [volumes](../../docs/concepts/volumes.md) not only optimizes inference cold start times but also enhances the

docs/docs/concepts/backends.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1132,9 +1132,9 @@ projects:
11321132

11331133
> To learn more, see the [Lambda](../../examples/clusters/lambda/#kubernetes) and [Crusoe](../../examples/clusters/crusoe/#kubernetes) examples.
11341134

1135-
### RunPod
1135+
### Runpod
11361136

1137-
Log into your [RunPod](https://www.runpod.io/console/) console, click Settings in the sidebar, expand the `API Keys` section, and click
1137+
Log into your [Runpod](https://www.runpod.io/console/) console, click Settings in the sidebar, expand the `API Keys` section, and click
11381138
the button to create a Read & Write key.
11391139

11401140
Then proceed to configuring the backend.

docs/docs/concepts/snippets/manage-fleets.ext

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,4 +7,4 @@ If the run reuses an existing fleet instance, only the fleet's
77

88
If an instance remains `idle`, it is automatically terminated after `idle_duration`.
99

10-
> Not applied for container-based backends (Kubernetes, Vast.ai, RunPod).
10+
> Not applied for container-based backends (Kubernetes, Vast.ai, Runpod).

docs/docs/guides/migration/slurm.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -908,7 +908,7 @@ resources:
908908
909909
#### Network volumes
910910
911-
Network volumes are persistent cloud storage (AWS EBS, GCP persistent disks, RunPod volumes).
911+
Network volumes are persistent cloud storage (AWS EBS, GCP persistent disks, Runpod volumes).
912912
913913
Single-node task:
914914
@@ -936,7 +936,7 @@ resources:
936936
937937
</div>
938938
939-
Network volumes cannot be used with distributed tasks (no multi-attach support), except where multi-attach is supported (RunPod) or via volume interpolation.
939+
Network volumes cannot be used with distributed tasks (no multi-attach support), except where multi-attach is supported (Runpod) or via volume interpolation.
940940
941941
For distributed tasks, use interpolation to attach different volumes to each node.
942942

0 commit comments

Comments
 (0)