Skip to content

Commit 40fe5fc

Browse files
committed
[NV] llm-d: project recipe yaml down to EPP-only keys before passing to epp
Signed-off-by: Ezra Silvera <ezra@il.ibm.com>
1 parent 00f38e5 commit 40fe5fc

1 file changed

Lines changed: 17 additions & 1 deletion

File tree

benchmarks/multi_node/llm-d/server.sh

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -235,8 +235,24 @@ PY
235235

236236
# EPP config: recipe override, else the default mounted by job.slurm
237237
# at /etc/epp/config.yaml (sourced from benchmarks/llm-d/epp-config.yaml).
238+
#
239+
# The recipe yaml is a single-file mix: EPP scheduler keys
240+
# (apiVersion, kind, plugins, schedulingProfiles, dataLayer) plus
241+
# per-role vLLM extra-args (prefill, decode) plus slurm hints. EPP's
242+
# strict YAML decoder rejects the latter ("unknown field \"prefill\""),
243+
# so when a recipe is in play we project it down to just the EPP keys
244+
# and hand EPP that.
238245
if [[ -n "$CONFIG_FILE" && -f "/etc/llmd-recipes/$CONFIG_FILE" ]]; then
239-
EPP_CONFIG="/etc/llmd-recipes/$CONFIG_FILE"
246+
EPP_CONFIG="/tmp/epp-config-from-recipe.yaml"
247+
python3 - <<PY
248+
import yaml
249+
recipe = yaml.safe_load(open('/etc/llmd-recipes/${CONFIG_FILE}'))
250+
# Keys EPP's strict decoder accepts. Anything else (prefill, decode,
251+
# slurm, ...) is dropped before passing to --config-file.
252+
keep = {'apiVersion', 'kind', 'plugins', 'schedulingProfiles', 'dataLayer'}
253+
yaml.safe_dump({k: v for k, v in recipe.items() if k in keep},
254+
open('${EPP_CONFIG}', 'w'))
255+
PY
240256
else
241257
EPP_CONFIG="/etc/epp/config.yaml"
242258
fi

0 commit comments

Comments
 (0)