Skip to content

Releases: dstackai/dstack

0.20.0rc1

11 Dec 12:59
@r4victor r4victor
0.20.0rc1
669a245
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
0.20.0rc1 Pre-release
Pre-release

dstack 0.20 is a major upcoming release that brings significant improvements and introduces a number of breaking changes. Read below for the most important ones.

Note

You're encouraged to test 0.20rc1 in the staging environment, share your feedback, and ensure you're ready to upgrade before the release goes GA (planned for December 17).

uv tool install "dstack[all]==0.20rc1"

For migration notes, please refer to the migration guide.

Fleets

dstack previously had two different ways to provision instances for runs: using a fleet configuration or using automatic fleet provisioning on run apply. To unify the UX, dstack no longer creates fleets automatically.

Fleets must now be created explicitly before submitting runs. This gives users full control over the provisioning lifecycle. If you don't need any limits on instance provisioning (as was the case with auto-created fleets), you can create a single elastic fleet for all runs:

type: fleet
name: default-fleet
nodes: 0..

Note that multi-node tasks require fleets with placement: cluster, which provides the best possible connectivity. You will need a separate fleet for each cluster.

Note

To keep the old behavior with auto-created fleets, set the DSTACK_FF_AUTOCREATED_FLEETS_ENABLED environment variable.

Runs

Working directory

Previously, the working_dir property had complicated semantics: it defaulted to /workflow, but for tasks and services without commands, the image's working directory was used instead.

This has now been simplified: working_dir always defaults to the image's working directory. The working directory of the default dstack images is now set to /dstack/run.

Repo directory

Working with repos is now more explicit and intuitive. First, dstack now only sets up repos that are explicitly defined in run configurations via repos; repos initialized with dstack init are not set up unless specified:

type: dev-environment
ide: vscode
repos:
  # Clone the repo in the configuration's dir into `working_dir`
  - .

Second, repos[].path now defaults to working_dir (".") instead of /workflow.

Third, cloning a repo into a non-empty directory now raises an error so that mistakes are not silently ignored. The previous behavior of skipping cloning can be specified explicitly with if_exists: skip:

type: dev-environment
ide: vscode
repos:
  - local_path: .
    path: /my_volume/repo
    if_exists: skip

Events

dstack now stores important events—such as resource CRUD operations, status changes, and other information crucial for auditing and debugging. Users can view events using the dstack event CLI command or in the UI.

$ dstack event
[2025-12-11 15:05:20] [👤admin] [run clever-cheetah-1] Run submitted. Status: SUBMITTED
[2025-12-11 15:05:20] [job clever-cheetah-1-0-0] Job created on run submission. Status: SUBMITTED
[2025-12-11 15:05:26] [job clever-cheetah-1-0-0, instance cloud-fleet-0] Job assigned to instance. Instance status: BUSY (1/1 blocks busy)

CLI

JSON output

The dstack ps and dstack gateway commands now support --format json / --json arguments that print results in JSON instead of plaintext:

$ dstack ps --json
{
  "project": "main",
  "runs": [
    {
      "id": "5f2e08b5-2098-4064-86c7-0efe0eb84970",
      "project_name": "main",
      "user": "admin",
      "fleet": {
        "id": "9598d5db-67d8-4a2e-bdd2-842ab93b2f2e",
        "name": "cloud-fleet"
      },
      ...
    }
  ]
}

Verda (formerly Datacrunch)

The datacrunch backend has been renamed to verda, following the company's rebranding.

projects:
  - name: main
    backends:
      - type: verda
        creds:
          type: api_key
          client_id: xfaHBqYEsArqhKWX-e52x3HH7w8T
          client_secret: B5ZU5Qx9Nt8oGMlmMhNI3iglK8bjMhagTbylZy4WzncZe39995f7Vxh8

All breaking changes

  • Fleets are no longer created automatically on run apply and have to be created explicitly before submitting runs.
  • The run's working_dir now always defaults to the image's working directory instead of /workflow. The working directory of dstack default images is now /dstack/run.
  • repos[].path now defaults to working_dir (".") instead of /workflow.
  • Dropped implicitly loaded repos; repos must be specified via repos configuration property.
  • Cloning a repo into a non-empty directory now raises an error. This can be changed by setting if_exists: skip.
  • Dropped CLI commands dstack config, dstack stats, and dstack gateway create.
  • Dropped Python API RunCollection methods RunCollection.get_plan(), RunCollection.exec_plan(), and RunCollection.submit().
  • Dropped local repos support: dstack init --local and dstack.api.LocalRepo.
  • Dropped Azure deprecated VM series Dsv3 and Esv4.
  • Dropped legacy server environment variables DSTACK_SERVER_METRICS_TTL_SECONDS and DSTACK_FORCE_BRIDGE_NETWORK.

Deprecations

  • Deprecated the API endpoint /api/project/{project_name}/fleets/create in favor of /api/project/{project_name}/fleets/apply.
  • Deprecated repo_dir argument in RunCollection.get_run_plan() in favor of repos[].path.

What's changed

New contributors

Full changelog: 0.19.40...0.20.0rc1

Contributors

un-def, olgenn, and 4 other contributors
Loading

0.19.40

27 Nov 15:01
@r4victor r4victor
0.19.40
757f0b6
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
0.19.40

This release fixes a DB migration deadlock that could happen when upgrading multi-replica server with Postgres to 0.19.39.

What's changed

  • [Bug] Split last user/project migration to fix deadlock by @r4victor in #3330

Full changelog: 0.19.39...0.19.40

Contributors

r4victor
Loading

0.19.39

27 Nov 15:00
@r4victor r4victor
0.19.39
e76696c
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
0.19.39

This release includes several important bug fixes, performance improvements, and updated documentation.

Documentation

Contributing

External contributors can now build the dstack documentation locally!

Previously, this wasn’t possible because dstack relied on the premium MKDocs Material Insider theme. Since the maintainers of MKDocs Material recently made the full edition free, we've updated the repository accordingly.

To help contributors get started, we've added a new guide: contributing/DOCS.md.

We welcome and encourage community contributions - whether that's reporting issues or submitting pull requests.

/llms.txt

As more users rely on LLMs to interact with tools, we've added support for /llms.txt to help guide models using dstack.

You can now use:

These files help LLMs better understand how dstack works and enable them to generate more accurate commands, configurations, and answers.

AGENTS.md

We’ve also added AGENTS.md to help modern AI agents automatically understand how to interact with the repository and tooling.

Warning

Be sure to update to 0.19.40, which includes an additional important fix.

What's changed

Full changelog: 0.19.38...0.19.39

Contributors

un-def, Bihan, and 2 other contributors
Loading

0.19.38

21 Nov 08:46
@r4victor r4victor
0.19.38
3b58cae
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
0.19.38

Gateways

Routers

dstack gateways now integrate with SGLang Model Gateway, enabling inference request routing with policies such as cache_aware, power_of_two, round_robin, and random. You can enable it by setting the router property in your gateway configuration to sglang and select any of the available routing policies.

Example configuration:

type: gateway
name: sglang-gateway

backend: aws
region: eu-west-1

domain: example.com
router:
  type: sglang
  policy: cache_aware

Read how the new router property works in the documentation.

Fleets

Run plan

Since 0.19.26 release, dstack has been provisioning instances according to configured fleets, but run plan offers didn’t reflect that — meaning you might not have seen the actual offers used for provisioning.

This has now been fixed, and the run plan shows offers that respect the configured fleets.

For example, you can create a fleet for provisioning spot GPU instances on AWS:

type: fleet
name: cloud-fleet
nodes: 0..
backends: [aws]
spot_policy: spot
resources: 
  gpu: 1..

The run plan for submitted runs now shows offers that match the fleet configuration:

✗ dstack apply                                                      
...
 #  BACKEND          RESOURCES                            INSTANCE TYPE  PRICE    
 1  aws (us-east-1)  cpu=4 mem=16GB disk=100GB T4:16GB:1  g4dn.xlarge    $0.526   
 2  aws (us-east-2)  cpu=4 mem=16GB disk=100GB T4:16GB:1  g4dn.xlarge    $0.526   
 3  aws (us-west-2)  cpu=4 mem=16GB disk=100GB T4:16GB:1  g4dn.xlarge    $0.526   
    ...                                                                           
 Shown 3 of 309 offers, $71.552max

What's changed

Full changelog: 0.19.37...0.19.38

Contributors

un-def, Bihan, and 2 other contributors
Loading

0.19.37

13 Nov 11:40
@r4victor r4victor
0.19.37
fa6875b

Choose a tag to compare

View all tags
0.19.37

CLI

dstack attach --logs --since

The dstack attach --logs command now supports a --since argument to show only the recent logs before following real-time logs. You can specify either a relative duration or an absolute timestamp:

# Show logs from a specific timestamp and follow real-time logs
> dstack attach my-task --logs --since 2025-11-05T08:54:15Z

# Show logs from the last 5 minutes and follow real-time logs
> dstack attach my-task --logs --since 5m

This is especially helpful for long-running services and trainings when you don't need to load the entire logs history.

Fleets

Placement groups for elastic fleets

Previously dstack set up interconnected clusters with placement groups only for fleets with static number of instances such as nodes: 8. Now instances provisioned in fleets with placement: cluster always use placement groups. So now you can use elastic fleets and get best connectivity:

type: fleet
name: cloud-fleet
placement: cluster
nodes: 0..

Important change: Multi-node tasks can now run only on fleets with placement: cluster.

Backends

Gateways on Kubernetes

Previously gateways support on Kubernetes was limited to managed Kubernetes with DNS-based load balancers such as EKS. Now the support is extended to IP-based load balancers such as GKE and Nebius' mk8s.

What's Changed

New Contributors

Full Changelog: 0.19.36...0.19.37

Contributors

un-def, earandap, and 2 other contributors
Loading

0.19.36

06 Nov 16:40
@jvstme jvstme
0.19.36
17cc2bd
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
0.19.36

CLI

dstack ps

The output of dstack ps has been revamped to include colored statuses and a more compact resource view. Full resource details are still available in dstack ps --verbose.

dstack-ps

dstack logs --since

The dstack logs command now supports a --since argument to show only recent logs. You can specify either a relative duration or an absolute timestamp:

# Show logs from a specific timestamp
> dstack logs logs-task --since 2025-11-05T08:54:15Z

# Show logs from the last 5 minutes
> dstack logs logs-task --since 5m

Kubernetes

Improved GPU allocation

The kubernetes backend now requests all available GPU types when scheduling jobs, instead of limiting to just the first available type. This enables more flexible scheduling in heterogeneous Kubernetes clusters with multiple GPU types.

Offers

Optional GPU requirements

When specifying GPU requirements with a lower bound of 0 (e.g., gpu: 0..8:24GB), dstack now includes non-GPU offers in addition to GPU instances. This allows for more flexible resource selection when GPU access is optional.

What's changed

New Contributors

Full Changelog: 0.19.35...0.19.36

Contributors

un-def, antoniojtorres, and 3 other contributors
Loading

0.19.35

30 Oct 09:40
@r4victor r4victor
0.19.35
c8c14cd
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
0.19.35

Runpod

Instant Clusters

dstack adds support for Runpod Instant Clusters enabling multi-node tasks on Runpod:

✗ dstack apply -f nccl-tests.dstack.yaml -b runpod

 Project          main                                    
 User             admin                                   
 Configuration    .dstack/confs/nccl-tests-simple.yaml    
 Type             task                                    
 Resources        cpu=2.. mem=8GB.. disk=100GB.. gpu:1..8 
 Spot policy      auto                                    
 Max price        -                                       
 Retry policy     -                                       
 Creation policy  reuse-or-create                         
 Idle duration    5m                                      
 Max duration     -                                       
 Reservation      -                                       

 #  BACKEND           RESOURCES                          INSTANCE TYPE     PRICE    
 1  runpod (US-KS-2)  cpu=128 mem=2008GB disk=100GB      NVIDIA A100-SXM…  $16.7…   
                      A100:80GB:8                                                   
 2  runpod (US-MO-1)  cpu=128 mem=2008GB disk=100GB      NVIDIA A100-SXM…  $16.7…   
                      A100:80GB:8                                                   
 3  runpod            cpu=160 mem=1504GB disk=100GB      NVIDIA H100 80G…  $25.8…   
    (CA-MTL-1)        H100:80GB:8                                                   
    ...                                                                             
 Shown 3 of 5 offers, $34.464max

Submit the run nccl-tests? [y/n]:

Runpod offers clusters of 2 to 8 nodes with H200, B200, H100, and A100 GPUs and InfiniBand networking up to 3200 Gbps.

What's Changed

New Contributors

Full Changelog: 0.19.34...0.19.35

Contributors

matiasinsaurralde, un-def, and 2 other contributors
Loading

0.19.34

23 Oct 10:32
@r4victor r4victor
0.19.34
7c2db9d
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
0.19.34

UI

Scheduled runs

The Run details page now shows Schedule and Next time for scheduled runs.

Finished times

The Run and Job list and details pages now display Finished times.

Backends

GCP

GCP G4 instances with NVIDIA RTX PRO 6000 GPUs are now generally available:

> dstack offer -b gcp --gpu RTXPRO6000                                          

 #  BACKEND            RESOURCES                                        INSTANCE TYPE    PRICE      
 1  gcp (us-central1)  cpu=48 mem=180GB disk=100GB RTXPRO6000:96GB:1    g4-standard-48   $4.5001    
 2  gcp (us-central1)  cpu=96 mem=360GB disk=100GB RTXPRO6000:96GB:2    g4-standard-96   $9.0002    
 3  gcp (us-central1)  cpu=192 mem=720GB disk=100GB RTXPRO6000:96GB:4   g4-standard-192  $18.0003   
 4  gcp (us-central1)  cpu=384 mem=1440GB disk=100GB RTXPRO6000:96GB:8  g4-standard-384  $36.0006

Also, GCP A4 instances are supported via reservations.

Runs

SSH keys

dstack now uses server-managed user SSH keys when starting new runs. This allows users to attach to the run from different machines, since the SSH key is automatically replicated to all clients. User-supplied SSH keys are still used if specified explicitly.

Docs

Kubernetes

The Kubernetes backend docs now include a list of required permissions.

What's Changed

Full Changelog: 0.19.33...0.19.34

Contributors

r4victor, jvstme, and peterschmidt85
Loading

0.19.33

16 Oct 17:20
@jvstme jvstme
0.19.33
1c2a7bd
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
0.19.33

Kubernetes

AMD

The kubernetes backend now allows you to run workloads on AMD GPU-enabled Kubernetes clusters.

Screenshot 2025-10-10 at 17 36 50

UI

Dev environments

You can now configure and provision dev environments directly from the user interface.

create-dev-environment-2.mp4

Note

CLI version 0.19.33 or later is required to attach to runs created from the UI.

GCP

Reservations

You can now configure specifically-targeted GCP reservations in fleet configurations to leverage reserved compute capacity:

type: fleet
nodes: 4
placement: cluster
backends: [gcp]
reservation: my-reservation

For reservations shared between projects, use the full syntax:

type: fleet
nodes: 4
placement: cluster
backends: [gcp]
reservation: projects/my-proj/reservations/my-reservation

dstack will automatically locate the specified reservation, match offers to the reservation's properties, and provision instances within the reservation. If there are multiple reservations with the specified name, all of them will be considered for provisioning.

Note

Using reservations requires the compute.reservations.list permission in the project that owns the reservation.

G4 preview

If your GCP project has access to the preview G4 instance type, you can now try it out with dstack.

> dstack offer -b gcp --gpu RTXPRO6000

 #  BACKEND            RESOURCES                                      INSTANCE TYPE   PRICE
 1  gcp (us-central1)  cpu=48 mem=180GB disk=100GB RTXPRO6000:96GB:1  g4-standard-48  $0

To use G4, enable its preview in the backend settings.

projects:
- name: main
  backends:
  - type: gcp
    project_id: my-project
    creds:
      type: default
    preview_features: [g4]

Hot Aisle

The hotaisle backend now supports 8x MI300X instances too.

> dstack offer -b hotaisle --gpu 8:MI300X                         

 #  BACKEND                   RESOURCES                                       INSTANCE TYPE                      PRICE    
 1  hotaisle (us-michigan-1)  cpu=104 mem=1792GB disk=12288GB MI300X:192GB:8  8x MI300X 104x Xeon Platinum 8470  $15.92

Docker

Default image

The default Docker image now uses CUDA 12.8 (updated from 12.1).

What's changed

Full changelog: 0.19.32...0.19.33

Contributors

un-def, olgenn, and 4 other contributors
Loading

0.19.32

09 Oct 11:23
@r4victor r4victor
0.19.32
58a6757
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
0.19.32

Fleets

Nodes

Maximum number of nodes

The fleet nodes.max property is now respected that allows limiting maximum number of instances allowed in a fleet. For example, to allow at most 10 instances in the fleet, you can do:

type: fleet
name: cloud-fleet
nodes: 0..10

A fleet will be considered for a run only if the run can fit into the fleet without violating nodes.max. If you don't need to enforce an upper limit, you can omit it:

type: fleet
name: cloud-fleet
nodes: 0..

Backends

Nebius

Tags

Nebius backend now supports backend and resource-level tags to tag cloud resources provisioned via dstack:

type: nebius
creds:
  type: service_account
  # ...
tags:
  team: my_team
  user: jake

Credentials file

It's also possible to configure the nebius backend using a credentials file generated by the nebius CLI:

nebius iam auth-public-key generate \
    --service-account-id <service account ID> \
    --output ~/.nebius/sa-credentials.json
projects:
- name: main
  backends:
  - type: nebius
    creds:
      type: service_account
      filename: ~/.nebius/sa-credentials.json

Hot Aisle

Hot Aisle backend now supports multi-GPU VMs such as 2xMI300X and 4xMI300X.

dstack apply -f .local/.dstack.yml --gpu amd:2
The working_dir is not set — using legacy default "/workflow". Future versions will default to the
image's working directory.

 #  BACKEND               RESOURCES                                 INSTANCE TYPE        PRICE
 1  hotaisle              cpu=26 mem=448GB disk=12288GB             2x MI300X 26x Xeon…  $3.98
    (us-michigan-1)       MI300X:192GB:2

What's changed

New contributors

Full changelog: 0.19.31...0.19.32

Contributors

svanzoest, un-def, and 5 other contributors
Loading