@@ -3763,7 +3763,7 @@ <h1 id="using-ssh-fleets-with-tensorwaves-private-amd-cloud">Using SSH fleets wi
37633763< p > Since last month, when we introduced support for private clouds and data centers, it has become easier to use < code > dstack</ code >
37643764to orchestrate AI containers with any AI cloud vendor, whether they provide on-demand compute or reserved clusters.</ p >
37653765< p > In this tutorial, we’ll walk you through how < code > dstack</ code > can be used with
3766- < a href ="https://tensorwave.com/ " target ="_blank "> TensorWave < span class ="twemoji external "> < svg xmlns ="http://www.w3.org/2000/svg " viewBox ="0 0 24 24 "> < path d ="m11.93 5 2.83 2.83L5 17.59 6.42 19l9.76-9.75L19 12.07V5z "/> </ svg > </ span > </ a > using
3766+ < a href ="https://tensorwave.com/ " target ="_blank "> TensorWave < span class ="twemoji external "> < svg xmlns ="http://www.w3.org/2000/svg " viewBox ="0 0 24 24 "> < path d ="m11.93 5 2.83 2.83L5 17.59 6.42 19l9.76-9.75L19 12.07V5z "/> </ svg > </ span > </ a > using
37673767< a href ="../../docs/concepts/fleets/#ssh "> SSH fleets</ a > .</ p >
37683768< p > < img src ="https://dstack.ai/static-assets/static-assets/images/dstack-tensorwave-v2.png " width ="630 "/> </ p >
37693769<!-- more -->
@@ -3776,10 +3776,9 @@ <h1 id="using-ssh-fleets-with-tensorwaves-private-amd-cloud">Using SSH fleets wi
37763776< h2 id ="creating-a-fleet "> Creating a fleet< a class ="headerlink " href ="#creating-a-fleet " title ="Permanent link "> ¶</ a > </ h2 >
37773777< details class ="info ">
37783778< summary > Prerequisites</ summary >
3779- < p > Once < code > dstack</ code > is < a href ="https://dstack.ai/docs/installation "> installed</ a > , create a project repo folder and run < code > dstack init </ code > .</ p >
3779+ < p > Once < code > dstack</ code > is < a href ="https://dstack.ai/docs/installation "> installed</ a > , create a project folder.</ p >
37803780< div class ="termy ">
37813781< div class ="highlight "> < pre > < span > </ span > < code > $< span class ="w "> </ span > mkdir< span class ="w "> </ span > tensorwave-demo< span class ="w "> </ span > < span class ="o "> &&</ span > < span class ="w "> </ span > < span class ="nb "> cd</ span > < span class ="w "> </ span > tensorwave-demo
3782- $< span class ="w "> </ span > dstack< span class ="w "> </ span > init
37833782</ code > </ pre > </ div >
37843783</ div >
37853784</ details >
@@ -3814,9 +3813,9 @@ <h2 id="creating-a-fleet">Creating a fleet<a class="headerlink" href="#creating-
38143813Provisioning...
38153814--->< span class ="w "> </ span > < span class ="m "> 100</ span > %
38163815
3817- < span class ="w "> </ span > FLEET< span class ="w "> </ span > INSTANCE< span class ="w "> </ span > RESOURCES< span class ="w "> </ span > STATUS< span class ="w "> </ span > CREATED< span class =" w " > </ span >
3818- < span class ="w "> </ span > my-tensorwave-fleet< span class ="w "> </ span > < span class ="m "> 0</ span > < span class ="w "> </ span > 8xMI300X< span class ="w "> </ span > < span class ="o "> (</ span > 192GB< span class ="o "> )</ span > < span class ="w "> </ span > < span class ="m "> 0</ span > /8< span class ="w "> </ span > busy< span class ="w "> </ span > < span class ="m "> 3</ span > < span class ="w "> </ span > mins< span class ="w "> </ span > ago< span class =" w " > </ span >
3819- < span class ="w "> </ span > < span class ="m "> 1</ span > < span class ="w "> </ span > 8xMI300X< span class ="w "> </ span > < span class ="o "> (</ span > 192GB< span class ="o "> )</ span > < span class ="w "> </ span > < span class ="m "> 0</ span > /8< span class ="w "> </ span > busy< span class ="w "> </ span > < span class ="m "> 3</ span > < span class ="w "> </ span > mins< span class ="w "> </ span > ago< span class =" w " > </ span >
3816+ < span class ="w "> </ span > FLEET< span class ="w "> </ span > INSTANCE< span class ="w "> </ span > RESOURCES< span class ="w "> </ span > STATUS< span class ="w "> </ span > CREATED
3817+ < span class ="w "> </ span > my-tensorwave-fleet< span class ="w "> </ span > < span class ="m "> 0</ span > < span class ="w "> </ span > 8xMI300X< span class ="w "> </ span > < span class ="o "> (</ span > 192GB< span class ="o "> )</ span > < span class ="w "> </ span > < span class ="m "> 0</ span > /8< span class ="w "> </ span > busy< span class ="w "> </ span > < span class ="m "> 3</ span > < span class ="w "> </ span > mins< span class ="w "> </ span > ago
3818+ < span class ="w "> </ span > < span class ="m "> 1</ span > < span class ="w "> </ span > 8xMI300X< span class ="w "> </ span > < span class ="o "> (</ span > 192GB< span class ="o "> )</ span > < span class ="w "> </ span > < span class ="m "> 0</ span > /8< span class ="w "> </ span > busy< span class ="w "> </ span > < span class ="m "> 3</ span > < span class ="w "> </ span > mins< span class ="w "> </ span > ago
38203819</ code > </ pre > </ div >
38213820
38223821</ div >
@@ -3827,7 +3826,7 @@ <h2 id="running-workloads">Running workloads<a class="headerlink" href="#running
38273826< p > Once the fleet is created, you can use < code > dstack</ code > to run workloads.</ p >
38283827< h3 id ="dev-environments "> Dev environments< a class ="headerlink " href ="#dev-environments " title ="Permanent link "> ¶</ a > </ h3 >
38293828< p > A dev environment lets you access an instance through your desktop IDE.</ p >
3830- < div editor-title =".dstack.yml ">
3829+ < div editor-title =".dstack.yml ">
38313830
38323831< div class ="highlight "> < pre > < span > </ span > < code > < span class ="nt "> type</ span > < span class ="p "> :</ span > < span class ="w "> </ span > < span class ="l l-Scalar l-Scalar-Plain "> dev-environment</ span >
38333832< span class ="nt "> name</ span > < span class ="p "> :</ span > < span class ="w "> </ span > < span class ="l l-Scalar l-Scalar-Plain "> vscode</ span >
@@ -3860,8 +3859,8 @@ <h3 id="dev-environments">Dev environments<a class="headerlink" href="#dev-envir
38603859< p > Open the link to access the dev environment using your desktop IDE.</ p >
38613860< h3 id ="tasks "> Tasks< a class ="headerlink " href ="#tasks " title ="Permanent link "> ¶</ a > </ h3 >
38623861< p > A task allows you to schedule a job or run a web app. Tasks can be distributed and support port forwarding.</ p >
3863- < p > Below is a distributed training task configuration: </ p >
3864- < div editor-title ="train.dstack.yml ">
3862+ < p > Below is a distributed training task configuration:</ p >
3863+ < div editor-title ="train.dstack.yml ">
38653864
38663865< div class ="highlight "> < pre > < span > </ span > < code > < span class ="nt "> type</ span > < span class ="p "> :</ span > < span class ="w "> </ span > < span class ="l l-Scalar l-Scalar-Plain "> task</ span >
38673866< span class ="nt "> name</ span > < span class ="p "> :</ span > < span class ="w "> </ span > < span class ="l l-Scalar l-Scalar-Plain "> train-distrib</ span >
@@ -3894,13 +3893,13 @@ <h3 id="tasks">Tasks<a class="headerlink" href="#tasks" title="Permanent link">&
38943893
38953894</ div >
38963895
3897- < p > < code > dstack</ code > automatically runs the container on each node while passing
3896+ < p > < code > dstack</ code > automatically runs the container on each node while passing
38983897< a href ="../../docs/concepts/tasks/#system-environment-variables "> system environment variables</ a >
38993898which you can use with < code > torchrun</ code > , < code > accelerate</ code > , or other distributed frameworks.</ p >
39003899< h3 id ="services "> Services< a class ="headerlink " href ="#services " title ="Permanent link "> ¶</ a > </ h3 >
39013900< p > A service allows you to deploy a model or any web app as a scalable and secure endpoint.</ p >
39023901< p > Create the following configuration file inside the repo:</ p >
3903- < div editor-title ="deepseek.dstack.yml ">
3902+ < div editor-title ="deepseek.dstack.yml ">
39043903
39053904< div class ="highlight "> < pre > < span > </ span > < code > < span class ="nt "> type</ span > < span class ="p "> :</ span > < span class ="w "> </ span > < span class ="l l-Scalar l-Scalar-Plain "> service</ span >
39063905< span class ="nt "> name</ span > < span class ="p "> :</ span > < span class ="w "> </ span > < span class ="l l-Scalar l-Scalar-Plain "> deepseek-r1-sglang</ span >
@@ -3910,7 +3909,7 @@ <h3 id="services">Services<a class="headerlink" href="#services" title="Permanen
39103909< span class ="w "> </ span > < span class ="p p-Indicator "> -</ span > < span class ="w "> </ span > < span class ="l l-Scalar l-Scalar-Plain "> MODEL_ID=deepseek-ai/DeepSeek-R1</ span >
39113910< span class ="w "> </ span > < span class ="p p-Indicator "> -</ span > < span class ="w "> </ span > < span class ="l l-Scalar l-Scalar-Plain "> HSA_NO_SCRATCH_RECLAIM=1</ span >
39123911< span class ="nt "> commands</ span > < span class ="p "> :</ span >
3913- < span class ="w "> </ span > < span class ="p p-Indicator "> -</ span > < span class ="w "> </ span > < span class ="l l-Scalar l-Scalar-Plain "> python3 -m sglang.launch_server --model-path $MODEL_ID --port 8000 --tp 8 --trust-remote-code</ span > < span class =" w " > </ span >
3912+ < span class ="w "> </ span > < span class ="p p-Indicator "> -</ span > < span class ="w "> </ span > < span class ="l l-Scalar l-Scalar-Plain "> python3 -m sglang.launch_server --model-path $MODEL_ID --port 8000 --tp 8 --trust-remote-code</ span >
39143913< span class ="nt "> port</ span > < span class ="p "> :</ span > < span class ="w "> </ span > < span class ="l l-Scalar l-Scalar-Plain "> 8000</ span >
39153914< span class ="nt "> model</ span > < span class ="p "> :</ span > < span class ="w "> </ span > < span class ="l l-Scalar l-Scalar-Plain "> deepseek-ai/DeepSeek-R1</ span >
39163915
@@ -3933,7 +3932,7 @@ <h3 id="services">Services<a class="headerlink" href="#services" title="Permanen
39333932Provisioning< span class ="w "> </ span > < span class ="sb "> `</ span > deepseek-r1-sglang< span class ="sb "> `</ span > ...
39343933--->< span class ="w "> </ span > < span class ="m "> 100</ span > %
39353934
3936- Service< span class ="w "> </ span > is< span class ="w "> </ span > published< span class ="w "> </ span > at:< span class =" w " > </ span >
3935+ Service< span class ="w "> </ span > is< span class ="w "> </ span > published< span class ="w "> </ span > at:
39373936< span class ="w "> </ span > http://localhost:3000/proxy/services/main/deepseek-r1-sglang/
39383937Model< span class ="w "> </ span > deepseek-ai/DeepSeek-R1< span class ="w "> </ span > is< span class ="w "> </ span > published< span class ="w "> </ span > at:
39393938< span class ="w "> </ span > http://localhost:3000/proxy/models/main/
@@ -3948,7 +3947,7 @@ <h2 id="see-it-in-action">See it in action<a class="headerlink" href="#see-it-in
39483947< div class ="admonition info ">
39493948< p class ="admonition-title "> What's next?</ p >
39503949< ol >
3951- < li > See < a href ="../../docs/concepts/fleets/#ssh "> SSH fleets</ a > </ li >
3950+ < li > See < a href ="../../docs/concepts/fleets/#ssh "> SSH fleets</ a > </ li >
39523951< li > Read about < a href ="../../docs/concepts/dev-environments/ "> dev environments</ a > , < a href ="../../docs/concepts/tasks/ "> tasks</ a > , and < a href ="../../docs/concepts/services/ "> services</ a > </ li >
39533952< li > Join < a href ="https://discord.gg/u8SmfwPpMd "> Discord < span class ="twemoji external "> < svg xmlns ="http://www.w3.org/2000/svg " viewBox ="0 0 24 24 "> < path d ="m11.93 5 2.83 2.83L5 17.59 6.42 19l9.76-9.75L19 12.07V5z "/> </ svg > </ span > </ a > </ li >
39543953</ ol >
0 commit comments