0.19.4-v1

Rate limits for services

You can now configure rate limits for your services running behind a gateway.

type: service
image: my-app:latest
port: 80

rate_limits:
# For /api/auth/* - 1 request per second, no bursts
- prefix: /api/auth/
  rps: 1
# For other URLs - 4 requests per second + bursts of up to 9 requests
- rps: 4
  burst: 9

Examples: TensorRT-LLM and Llama 4

We added a new example on TensorRT-LLM that shows how to deploy both DeepSeek R1 and its distilled version
using TensorRT-LLM and dstack.

The Llama example was updated to demonstrate the deployment of Llama 4 Scout using dstack.

Improved contributing experience

We continue to make contributing to dstack easier and improve dev experience. Since the last release, we moved from pip to uv in CI and dev pipelines. Dependencies installation times went from ~70 seconds to less than 10 seconds. The Development guide was updated to show how to get the dstack development setup with uv. The CI Build pipeline triggered on pull requests were optimized from 9 minutes to 4 minutes.

We also documented uv as one of the recommended installation options for dstack.

What's Changed

• [Landing] Refactoring (WIP) by @peterschmidt85 in dstackai/dstack#2495
• Fix CloudWatchLogStorage with sparse logs by @un-def in dstackai/dstack#2501
• Migrate to uv by @colinjc in dstackai/dstack#2455
• Fix docs build with uv by @r4victor in dstackai/dstack#2504
• [Example] Update Llama 4 Examples by @Bihan in dstackai/dstack#2508
• Move to uv in dstack-server Docker image by @r4victor in dstackai/dstack#2509
• Fix dstack dependency for gateway by @r4victor in dstackai/dstack#2511
• [Docs] Add uv to Installation; Minor improvements by @peterschmidt85 in dstackai/dstack#2510
• Validate usernames by @r4victor in dstackai/dstack#2514
• Run pytest in parallel with pytest-xdist by @r4victor in dstackai/dstack#2515
• Add Llama4 AMD example by @Bihan in dstackai/dstack#2513
• Use exponentially increasing retry delays for pending runs by @r4victor in dstackai/dstack#2519
• Speed up frontend CI by @r4victor in dstackai/dstack#2520
• Service rate limits by @jvstme in dstackai/dstack#2517
• Set no-guess-dev for dev package versions by @r4victor in dstackai/dstack#2521
• Detect dstack version from file instead of git by @r4victor in dstackai/dstack#2524
• Add TensorrRT-LLM Example by @Bihan in dstackai/dstack#2444
• Fix Nginx upstream name conflicts by @jvstme in dstackai/dstack#2526
• Fix detaching from dstack attach by @jvstme in dstackai/dstack#2528

New Contributors

• @colinjc made their first contribution in dstackai/dstack#2455

Full Changelog: dstackai/dstack@0.19.3...0.19.4