Skip to content

Commit 4b0ee88

Browse files
committed
calculator: add liability disclaimer + rename Production->Deployment (Alex review)
Per review feedback on PR #34: - Add a free-utility / no-warranty / no-liability / AS-IS disclaimer to the README, both web pages (visible banner), and the calc.py header. - Rename PRODUCTION-GUIDE.md -> DEPLOYMENT-GUIDE.md; remove the word 'Production' from all content (-> deployment / live environment). - Clean dangling driver/sweep.py refs in shipped files -> 'a live GPU sweep' (the sweep harness lives in the companion repo, noted in the scope banner). Author: Anton Alexander
1 parent 47b44aa commit 4b0ee88

5 files changed

Lines changed: 39 additions & 13 deletions

File tree

Container-Root/eks/deployment/inference/agentic-ai/disagg-calculator/README.md

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,14 @@
22

33
Author: Anton Alexander
44

5+
> **Disclaimer.** This calculator is provided **free of charge, as a utility** to help quickly
6+
> **estimate** and plan model-serving topologies. It produces **first-order estimates and guidance only**.
7+
> AWS and the authors make **no representation, warranty, or guarantee** — express or implied — about model
8+
> performance, throughput, latency, cost, or fitness for any purpose, and **accept no liability** for any
9+
> outcome arising from its use. Actual measured results will differ from the calculated numbers. Always
10+
> validate on your own hardware with your own workload before making deployment or purchasing decisions.
11+
> The calculator is provided **"AS IS"** without support obligation.
12+
513
An architecture-first calculator that answers a single question for any model on any GPU:
614
**should I serve it aggregated (prefill + decode on one worker) or disaggregated (prefill and decode
715
split across workers, KV cache moved over the network)?** It derives the answer from the model's real
@@ -28,7 +36,7 @@ disagg-calculator/
2836
index.html # interactive calculator (pick a model + GPU + workload → verdict), runs in-browser
2937
calculator.html # the same engine as a single-purpose page (shareable link)
3038
docs/
31-
PRODUCTION-GUIDE.md # when to disaggregate vs. scale aggregated replicas — rules of thumb + worked cases
39+
DEPLOYMENT-GUIDE.md # when to disaggregate vs. scale aggregated replicas — rules of thumb + worked cases
3240
METHODOLOGY.md # the math, the physics assumptions, and how the calibration loop refines them
3341
```
3442

@@ -77,7 +85,7 @@ accurate **without changing the physics**. See [docs/METHODOLOGY.md](docs/METHOD
7785

7886
## When to use which verdict
7987

80-
See [docs/PRODUCTION-GUIDE.md](docs/PRODUCTION-GUIDE.md) for the rules of thumb. In short:
88+
See [docs/DEPLOYMENT-GUIDE.md](docs/DEPLOYMENT-GUIDE.md) for the rules of thumb. In short:
8189

8290
- **Aggregated replicas** when: short-context chat, prefill cheap relative to decode, cost/throughput-bound,
8391
loose-to-moderate ITL SLO. At the 1P:1D floor, disaggregation buys little but costs ≥2× the GPUs.

Container-Root/eks/deployment/inference/agentic-ai/disagg-calculator/analyze/calc.py

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,10 @@
33
Author: Anton Alexander
44
Source: github.com/dmvevents/dynamo-disagg-optimizer · live: dmvevents.github.io/dynamo-disagg-optimizer/calculator.html
55
6+
DISCLAIMER: Provided free, as a utility, for estimation and planning ONLY. AWS and the authors make no
7+
warranty or guarantee of model performance and accept no liability for results obtained from its use.
8+
Actual measured numbers will differ; validate on your own hardware before deployment decisions. AS IS.
9+
610
calc.py — analytical "should I disaggregate?" model for ANY (instance, model, workload).
711
812
Given an AWS instance type + model characteristics + workload SLO, estimate — WITHOUT running GPUs —
@@ -12,7 +16,7 @@
1216
1317
Grounded in first-order rooflines and CALIBRATED to this org's measured Nemotron-3 Ultra 550B / p5en
1418
numbers (see CAL/calibration below). It is an ESTIMATOR (order-of-magnitude + direction), not a simulator —
15-
it tells you where to spend GPU hours, then the live sweep (driver/sweep.py) confirms.
19+
it tells you where to spend GPU hours, then the live sweep (a live GPU sweep) confirms.
1620
1721
Pure-Python stdlib only, so it runs headless and the HTML pages reimplement the SAME formulas in JS
1822
(kept honest by tests/parity.py, which asserts calc.py == calculator.html == index.html on a grid).
@@ -496,12 +500,12 @@ def _caveats(model: dict, itl_cal: dict | None = None) -> list[str]:
496500
if itl_cal is not None:
497501
if itl_cal.get("calibration_n", 0) >= 1:
498502
out.append("decode ITL is calibrated to a MEASURED anchor (n=%d, %s); raw roofline shown alongside. "
499-
"Single-point calibration — re-confirm off the bf16/H200/TP8 anchor with driver/sweep.py."
503+
"Single-point calibration — re-confirm off the bf16/H200/TP8 anchor with a live GPU sweep."
500504
% (itl_cal["calibration_n"], itl_cal.get("calibration_confidence", "")))
501505
else:
502506
out.append("decode ITL is the RAW HBM roofline (no measured anchor for this family) — likely OPTIMISTIC "
503507
"for eager-mode / low-concurrency; treat as a lower bound on latency.")
504-
out.append("Estimator only: directionally accurate, calibrated to 550B/p5en (n=1). Confirm the verdict with driver/sweep.py on real GPUs.")
508+
out.append("Estimator only: directionally accurate, calibrated to 550B/p5en (n=1). Confirm the verdict with a live GPU sweep on real GPUs.")
505509
return out
506510

507511

Container-Root/eks/deployment/inference/agentic-ai/disagg-calculator/docs/PRODUCTION-GUIDE.md renamed to Container-Root/eks/deployment/inference/agentic-ai/disagg-calculator/docs/DEPLOYMENT-GUIDE.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Production guide: aggregation vs disaggregation, end-to-end
1+
# Deployment guide: aggregation vs disaggregation, end-to-end
22

33
*Author: Anton Alexander*
44
*Source: github.com/dmvevents/dynamo-disagg-optimizer · live: dmvevents.github.io/dynamo-disagg-optimizer/calculator.html*
@@ -7,7 +7,7 @@
77
A decision-then-deploy runbook. You start with a model + an SLA + a budget, and finish with a deployed,
88
benchmarked Dynamo serving config — having spent GPU hours **only where the cheap analytical layers said to**.
99

10-
The question this guide answers in production terms: **for MY model, instance, SLA, and budget — do I
10+
The question this guide answers in deployment terms: **for MY model, instance, SLA, and budget — do I
1111
aggregate or disaggregate prefill/decode, and what does each cost per million tokens?**
1212

1313
> Latency and cost can disagree. Disaggregation often wins on **TTFT / SLA-attainment** while
@@ -119,10 +119,10 @@ python3 analyze/cost.py --instance p5en.48xlarge --config m.json \
119119
--price-per-hr <your_rate_if_capacity_block> # default = live AWS on-demand
120120
```
121121
Output: $/Mtok for agg vs disagg, cheaper-at-SLA, savings %, and the latency-vs-cost note. **This is the
122-
production decision artifact** — pair it with the SLA-attainment from step 6.
122+
deployment decision artifact** — pair it with the SLA-attainment from step 6.
123123

124124
## 8. DEPLOY the winner
125-
`kubectl apply -f <chosen YAML>` (or the aiconfigurator DGD). Re-run step 6 in prod periodically; pod-freshness
125+
`kubectl apply -f <chosen YAML>` (or the aiconfigurator DGD). Re-run step 6 in your live environment periodically; pod-freshness
126126
matters (a stale pod can show 4× worse latency — restart before trusting a baseline).
127127

128128
---

Container-Root/eks/deployment/inference/agentic-ai/disagg-calculator/web/calculator.html

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,13 @@ <h1>Aggregation vs Disaggregation Calculator</h1>
116116
transfer-toll vs interference → <b>verdict</b> → $/Mtok → optimal operating point.
117117
<b class="honest">Disaggregation is not free — it helps for some workloads and regresses for others. This tool tells you which: when it helps, when it&rsquo;s a no-op, and when it costs you (at the 1P:1D floor it&rsquo;s a latency play, not a cost play).</b>
118118
Physics mirrors <code>analyze/calc.py</code> + <code>cost.py</code> + <code>optimizer.py</code>, calibrated to measured 550B/p5en.</div>
119+
<div style="margin:10px 0;padding:8px 12px;border:1px solid #c89b3c;background:#2a2410;color:#e8d9a0;border-radius:6px;font-size:12px;line-height:1.45">
120+
<b>Disclaimer.</b> Provided free of charge, as a utility, for quick <b>estimation and planning only</b>. AWS and the
121+
authors make <b>no representation, warranty, or guarantee</b> of model performance, throughput, latency, or cost, and
122+
<b>accept no liability</b> for any outcome arising from its use. Actual measured results will differ from the
123+
calculated numbers — always validate on your own hardware with your own workload before deployment or purchasing
124+
decisions. Provided <b>&ldquo;AS IS&rdquo;</b>.
125+
</div>
119126
</header>
120127
<div class="wrap">
121128
<!-- ============ INPUTS (the architecture) ============ -->
@@ -205,7 +212,7 @@ <h3>2 · Compute + workload + targets</h3>
205212
<b>How to read this:</b> a <b>directional L0 screen</b>, calibrated to a single measured anchor
206213
(<b>n=1</b>): Nemotron-3 Ultra 550B, <b>bf16</b>, <b>H200 / p5en</b>, <b>TP8</b>. It models per-request
207214
single-stream latency — <b>not</b> a validated multi-tenant simulator. Confirm any verdict with
208-
<code>driver/sweep.py</code> on real GPUs before acting.
215+
<code>a live GPU sweep</code> on real GPUs before acting.
209216
<div class="provlegend">Number provenance:
210217
<span class="badge meas tiny">MEASURED</span> real GPU benchmark
211218
<span class="badge der tiny">DERIVED</span> computed from your inputs

Container-Root/eks/deployment/inference/agentic-ai/disagg-calculator/web/index.html

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -139,7 +139,14 @@
139139
<body>
140140
<header>
141141
<h1>Aggregation vs Disaggregation Optimizer <span class="brand">— should you split prefill/decode?</span></h1>
142-
<div class="sub">Enter an AWS instance + model + workload → get the <b>topology verdict</b>: aggregate, hybrid, or disaggregate. <b class="honest">Disaggregation is not free — it helps for some workloads and regresses for others. This tool tells you which: when it helps, when it&rsquo;s a no-op, and when it costs you (at the 1P:1D floor it&rsquo;s a latency play, not a cost play).</b> Physics mirrors <code>analyze/calc.py</code>; calibrated to measured Nemotron-3 Ultra 550B / p5en. Estimator — confirm with <code>driver/sweep.py</code>.</div>
142+
<div class="sub">Enter an AWS instance + model + workload → get the <b>topology verdict</b>: aggregate, hybrid, or disaggregate. <b class="honest">Disaggregation is not free — it helps for some workloads and regresses for others. This tool tells you which: when it helps, when it&rsquo;s a no-op, and when it costs you (at the 1P:1D floor it&rsquo;s a latency play, not a cost play).</b> Physics mirrors <code>analyze/calc.py</code>; calibrated to measured Nemotron-3 Ultra 550B / p5en. Estimator — confirm with a live sweep.</div>
143+
<div style="margin:10px 0;padding:8px 12px;border:1px solid #c89b3c;background:#2a2410;color:#e8d9a0;border-radius:6px;font-size:12px;line-height:1.45">
144+
<b>Disclaimer.</b> Provided free of charge, as a utility, for quick <b>estimation and planning only</b>. AWS and the
145+
authors make <b>no representation, warranty, or guarantee</b> of model performance, throughput, latency, or cost, and
146+
<b>accept no liability</b> for any outcome arising from its use. Actual measured results will differ from the
147+
calculated numbers — always validate on your own hardware with your own workload before deployment or purchasing
148+
decisions. Provided <b>&ldquo;AS IS&rdquo;</b>.
149+
</div>
143150
</header>
144151

145152
<!-- ============ EXPLAINER ============ -->
@@ -391,7 +398,7 @@ <h4>2. Prefill/Decode pool scaling — the at-scale agg-vs-disagg crossover</h4>
391398
<b>How to read this:</b> a <b>directional L0 screen</b>, calibrated to a single measured anchor
392399
(<b>n=1</b>): Nemotron-3 Ultra 550B, <b>bf16</b>, <b>H200 / p5en</b>, <b>TP8</b>. It models per-request
393400
single-stream latency — <b>not</b> a validated multi-tenant simulator. Confirm any verdict with
394-
<code>driver/sweep.py</code> on real GPUs before acting.
401+
<code>a live GPU sweep</code> on real GPUs before acting.
395402
</div>
396403
<div class="trustchip"><span class="dot" aria-hidden="true"></span><span>Honest by design — across our 240-cell catalog the verdict splits roughly evenly: <b>~52% disaggregate, ~48% aggregate-or-hybrid</b>. And on the cost axis at the 1P:1D floor, disagg almost always regresses (it&rsquo;s a latency play). A no-op or a regression is a real result, not a failure.</span></div>
397404
<details class="whatis">
@@ -710,7 +717,7 @@ <h4>✗ It is NOT</h4>
710717
document.getElementById('why').textContent=r.why;
711718
document.getElementById('vmodel').textContent=(preset.value||"custom")+" ("+m.family+")";
712719
document.getElementById('vinst').textContent=document.getElementById('inst').value+" ("+g.gpu+")";
713-
document.getElementById('cav').textContent = m.family==="hybrid-mamba"?"⚠ hybrid-Mamba: PP1 only on EFA; KV pull descriptor-bound. Confirm with driver/sweep.py.":"Estimator — confirm verdict with driver/sweep.py on real GPUs.";
720+
document.getElementById('cav').textContent = m.family==="hybrid-mamba"?"⚠ hybrid-Mamba: PP1 only on EFA; KV pull descriptor-bound. Confirm with a live GPU sweep.":"Estimator — confirm verdict with a live GPU sweep on real GPUs.";
714721
kKV.textContent=r.kv_MB.toFixed(1)+" MB"; kXfer.textContent=r.xfer.toFixed(1)+" ms"; kPre.textContent=r.pre.toFixed(0)+" ms";
715722
kITL.textContent=r.itl.toFixed(1)+" ms"; kLine.textContent=r.line.toFixed(0)+" GB/s"; kImb.textContent=r.imb.toFixed(1)+"×";
716723
bar('cTime',['KV xfer','Prefill','Decode/tok','Decode phase'],[r.xfer,r.pre,r.itl,r.dphase],['#58a6ff','#d29922','#3fb950','#8957e5'],'Latency components (ms)');

0 commit comments

Comments
 (0)