Skip to content

Commit 613f3d1

Browse files
committed
new conflict semantic
Signed-off-by: Shengliang Xu <shengliangx@nvidia.com>
1 parent 8135649 commit 613f3d1

4 files changed

Lines changed: 46 additions & 47 deletions

File tree

CHANGELOG.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ Changelog
1414
- Enable PTQ workflow for the Step3.5-Flash MoE model with NVFP4 W4A4 + FP8 KV cache quantization. See `modelopt_recipes/models/Step3.5-Flash/nvfp4-mlp-only.yaml <https://github.com/NVIDIA/Model-Optimizer/blob/main/modelopt_recipes/models/Step3.5-Flash/nvfp4-mlp-only.yaml>`_ for more details.
1515
- Add support for vLLM fakequant reload using ModelOpt state for HF models. See `examples/vllm_serve/README.md <https://github.com/NVIDIA/Model-Optimizer/tree/main/examples/vllm_serve#load-qatptq-model-and-serve-in-vllm-wip>`_ for more details.
1616
- [Early Testing] Add Claude Code PTQ skill (``.claude/skills/ptq/``) for agent-assisted post-training quantization. The skill guides the agent through environment detection, model support checking, format selection, and execution via the launcher or manual SLURM/Docker/bare GPU paths. Includes handling for unlisted models with custom module patching. This feature is in early testing — use with caution.
17-
- Add composable ``$import`` system for recipe YAML configs. Recipes can now declare an ``imports`` section mapping names to reusable config snippet files. The ``{$import: name}`` marker resolves at load time — as a dict value it replaces the content (with optional extend and multi-import via ``$import: [a, b]``), as a list element it splices the snippet entries. Key conflicts between imports or inline keys raise errors. Resolution is recursive with circular import detection. All built-in PTQ recipes converted to use imports with shared snippets under ``modelopt_recipes/configs/``. See :ref:`composable-imports` for the full specification.
17+
- Add composable ``$import`` system for recipe YAML configs. Recipes can now declare an ``imports`` section mapping names to reusable config snippet files. The ``{$import: name}`` marker resolves at load time — as a dict value it replaces the content with ordered override precedence (later imports override earlier, inline keys override all), as a list element it splices the snippet entries. Supports multi-import (``$import: [a, b]``) and inline extension/override. Resolution is recursive with circular import detection. All built-in PTQ recipes converted to use imports with shared snippets under ``modelopt_recipes/configs/``. See :ref:`composable-imports` for the full specification.
1818

1919
**Backward Breaking Changes**
2020

docs/source/guides/10_recipes.rst

Lines changed: 21 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -179,33 +179,38 @@ confused with literal values. The marker can appear anywhere in the recipe:
179179
- As a **list element** — the snippet (which must itself be a list) is spliced
180180
into the surrounding list.
181181

182-
As a **dict value**, ``$import`` supports three composition modes:
182+
As a **dict value**, ``$import`` supports composition with clear override
183+
precedence (lowest to highest):
183184

184-
- **Single import:** ``$import: name`` — replaced with the snippet content.
185-
- **Multiple imports:** ``$import: [name1, name2]`` — snippets are merged into
186-
one dict. The snippets must not have overlapping keys.
187-
- **Import + extend:** extra keys alongside ``$import`` are merged in after the
188-
import(s). Extra keys must not conflict with any imported key.
185+
1. **Imports in list order** — ``$import: [base, override]``: later snippets
186+
override earlier ones on key conflicts.
187+
2. **Inline keys** — extra keys alongside ``$import`` override all imported
188+
values.
189+
190+
This is equivalent to calling ``dict.update()`` in order: imports first (in
191+
list order), then inline keys last.
189192

190193
.. code-block:: yaml
191194
192195
# Single import
193196
cfg:
194-
$import: fp8
197+
$import: nvfp4
195198
196-
# Multiple imports — merge two non-overlapping snippets
199+
# Import + override — import nvfp4_dynamic, then override type inline
197200
cfg:
198-
$import: [bits, scale]
201+
$import: nvfp4 # imports {num_bits: e2m1, block_sizes: {-1: 16, type: dynamic, ...}}
202+
block_sizes:
203+
-1: 16
204+
type: static # overrides type: dynamic → static calibration
199205
200-
# Import + extend — add axis on top of imported fp8
206+
# Multiple imports — later snippet overrides earlier on conflict
201207
cfg:
202-
$import: fp8
203-
axis: 0 # result: {num_bits: e4m3, axis: 0}
208+
$import: [base_format, kv_tweaks] # kv_tweaks wins on shared keys
204209
205-
Key conflicts are never allowed — whether between imported snippets or between
206-
imports and inline keys. If a key appears in more than one source, the loader
207-
raises an error. This avoids ambiguous merge semantics. If you need different
208-
values for an existing key, create a new snippet instead.
210+
# All three: multi-import + inline override
211+
cfg:
212+
$import: [bits, scale]
213+
axis: 0 # highest precedence
209214
210215
As a **list element**, ``$import`` must be the only key — extra keys alongside
211216
a list splice are not supported.

modelopt/recipe/loader.py

Lines changed: 7 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -116,12 +116,15 @@ def _lookup(ref_name: str, context: str) -> Any:
116116
and isinstance(entry.get("cfg"), dict)
117117
and _IMPORT_KEY in entry["cfg"]
118118
):
119-
# cfg: {$import: name_or_list, ...extra} → import, merge, extend
119+
# cfg: {$import: name_or_list, ...inline} → import then override
120+
#
121+
# Precedence (lowest → highest):
122+
# 1. Imports in list order (later imports override earlier)
123+
# 2. Inline keys (override all imports)
120124
ref = entry["cfg"].pop(_IMPORT_KEY)
121-
extra_keys = dict(entry["cfg"]) # remaining inline keys
125+
inline_keys = dict(entry["cfg"]) # remaining inline keys
122126
ref_names = ref if isinstance(ref, list) else [ref]
123127

124-
# Merge all imported snippets, detecting conflicts between them
125128
merged: dict[str, Any] = {}
126129
for name in ref_names:
127130
snippet = _lookup(name, f"cfg of {entry}")
@@ -130,25 +133,9 @@ def _lookup(ref_name: str, context: str) -> Any:
130133
f"$import {name!r} in cfg must resolve to a dict, "
131134
f"got {type(snippet).__name__}."
132135
)
133-
conflicts = set(snippet) & set(merged)
134-
if conflicts:
135-
raise ValueError(
136-
f"$import {name!r} conflicts with keys from prior imports: "
137-
f"{sorted(conflicts)}. Imported snippets must not overlap."
138-
)
139136
merged.update(snippet)
140137

141-
# Extend with inline keys, detecting conflicts with imports
142-
if extra_keys:
143-
conflicts = set(extra_keys) & set(merged)
144-
if conflicts:
145-
raise ValueError(
146-
f"Inline keys {sorted(conflicts)} conflict with imported "
147-
f"values. Cannot override imported values — create a new "
148-
f"snippet instead."
149-
)
150-
merged.update(extra_keys)
151-
138+
merged.update(inline_keys)
152139
entry["cfg"] = merged
153140
resolved_cfg.append(entry)
154141
else:

tests/unit/recipe/test_loader.py

Lines changed: 17 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -532,9 +532,9 @@ def test_import_cfg_extend(tmp_path):
532532
assert cfg == {"num_bits": (4, 3), "axis": 0}
533533

534534

535-
def test_import_cfg_conflict_raises(tmp_path):
536-
"""$import in cfg with conflicting keys raises ValueError."""
537-
(tmp_path / "fp8.yml").write_text("num_bits: e4m3\n")
535+
def test_import_cfg_inline_overrides_import(tmp_path):
536+
"""Inline keys override imported values (highest precedence)."""
537+
(tmp_path / "fp8.yml").write_text("num_bits: e4m3\naxis:\n")
538538
recipe_file = tmp_path / "recipe.yml"
539539
recipe_file.write_text(
540540
f"imports:\n"
@@ -549,8 +549,12 @@ def test_import_cfg_conflict_raises(tmp_path):
549549
f" $import: fp8\n"
550550
f" num_bits: 8\n"
551551
)
552-
with pytest.raises(ValueError, match="conflict with imported"):
553-
load_recipe(recipe_file)
552+
recipe = load_recipe(recipe_file)
553+
cfg = recipe.quantize["quant_cfg"][0]["cfg"]
554+
# inline num_bits: 8 overrides imported num_bits: e4m3 → (4,3)
555+
assert cfg["num_bits"] == 8
556+
# imported axis: None is preserved (no inline override)
557+
assert cfg["axis"] is None
554558

555559

556560
def test_import_cfg_multi_import(tmp_path):
@@ -576,9 +580,9 @@ def test_import_cfg_multi_import(tmp_path):
576580
assert cfg == {"num_bits": (4, 3), "axis": 0}
577581

578582

579-
def test_import_cfg_multi_import_conflict_raises(tmp_path):
580-
"""$import with a list of names raises when snippets have overlapping keys."""
581-
(tmp_path / "a.yml").write_text("num_bits: e4m3\n")
583+
def test_import_cfg_multi_import_later_overrides_earlier(tmp_path):
584+
"""In $import list, later snippets override earlier ones on key conflicts."""
585+
(tmp_path / "a.yml").write_text("num_bits: e4m3\naxis: 0\n")
582586
(tmp_path / "b.yml").write_text("num_bits: 8\n")
583587
recipe_file = tmp_path / "recipe.yml"
584588
recipe_file.write_text(
@@ -594,8 +598,11 @@ def test_import_cfg_multi_import_conflict_raises(tmp_path):
594598
f" cfg:\n"
595599
f" $import: [a, b]\n"
596600
)
597-
with pytest.raises(ValueError, match="conflicts with keys from prior imports"):
598-
load_recipe(recipe_file)
601+
recipe = load_recipe(recipe_file)
602+
cfg = recipe.quantize["quant_cfg"][0]["cfg"]
603+
# b overrides a's num_bits; a's axis is preserved
604+
assert cfg["num_bits"] == 8
605+
assert cfg["axis"] == 0
599606

600607

601608
def test_import_cfg_multi_import_with_extend(tmp_path):

0 commit comments

Comments
 (0)