Skip to content

Commit a477918

Browse files
authored
Migrate OnnxDAG usage to onnx_ir + onnxscript rewriter and remove onnx_dag.py (microsoft#2558)
## Describe your changes Replaces all `OnnxDAG` usage with `onnx_ir` (onnx-ir) and the `onnxscript` rewriter, following the pattern from microsoft#2550, then deletes the `onnx_dag.py` implementation. **Production passes migrated to `onnx_ir`:** - `vitis_ai/preprocess.py`, `static_llm.py`, `float16_conversion.py`, `quantization.py`, `common.py` - `mnb_to_qdq.py` - `split.py` — `_run_for_config` rebuilt on ir; split graphs constructed via per-split value maps, missing value-info resolved through `SymbolicShapeInference` - `compose.py` — `_get_composed_model` rebuilt on ir; merge-by-name with `serde`-based node dedup and numpy shared-initializer equality - `graph_surgeries.py` — `TieWordEmbeddings` moved from `ProtoSurgeon` (proto/DAG) to `Surgeon` (`call_ir` on `ir.Model`); removed dead `find_node` and the DAG-based base `add_reshape_node` - `peephole_optimizer.py` — `fuse_reshape_operations` reimplemented on `onnx_ir` (removing the `onnxruntime.transformers.onnx_model.OnnxModel` dependency), replacing the previous stale OnnxDAG TODO **onnx_ir idiom cleanups (from review feedback):** - Use `ir_model.graphs()` (`common.py`), `ir_model.graph.all_nodes()` (`mnb_to_qdq.py`), and `node.predecessors()` / `node.successors()` (`split.py`) instead of hand-rolled helpers - Use `ir.node` for node/attribute construction (`graph_surgeries.py`, `mnb_to_qdq.py`, `vitis_ai/preprocess.py`) - Use `ir.convenience.create_value_mapping` to build the value-info map in `split.py` - Use `value.dtype` directly in `compose.py` - Moved the `onnx_ir` import to module top in `quantization.py` **Correctness fixes for optional node names / opsets (from review feedback):** - `compose.py` — only dedup nodes by name when the name is non-empty, so different unnamed nodes (`name=""`/`None`) no longer collide on the same key and trigger false "Node mismatch" assertions - `compose.py` — merge `opset_imports` across all component models (max version per domain) instead of copying only from the first model, so composing models with different domains/opset versions produces a valid model - `float16_conversion.py` — when `node_include_list` is set, raise a clear error if the model contains unnamed nodes (which cannot be included by name), preserving include-only semantics **Tests:** - `test_split_model.py`, `test_mnb_to_qdq.py` — migrated to ir idioms - `test_graph_surgeries.py` — new read-only `GraphInspector` (ir-backed) replaces `OnnxDAG` inspection sites; it reproduces OnnxDAG's unique-naming of unnamed/duplicate nodes so name-based assertions stay stable **Cleanup:** - Deleted `olive/passes/onnx/onnx_dag.py` - No remaining `OnnxDAG` / `onnx_dag` references Note: unlike OnnxDAG, ir does not auto-assign node names, so call sites that keyed on names now guard against empty/`None` names (e.g. `split.py`, `mnb_to_qdq.py`, `compose.py`, `float16_conversion.py`). The quantized `TieWordEmbeddings` path is validated structurally only (no ORT inference available offline). ## Checklist before requesting a review - [x] Add unit tests for this change. - [x] Make sure all tests can pass. - [ ] Update documents if necessary. - [x] Lint and apply fixes to your code by running `lintrunner -a` - [ ] Is this a user-facing change? If yes, give a description of this change to be included in the release notes. ## (Optional) Issue link --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
1 parent d9cc5e6 commit a477918

14 files changed

Lines changed: 782 additions & 1610 deletions

olive/passes/onnx/common.py

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,6 @@
2222

2323
from olive.common.utils import StrEnumBase, hardlink_copy_file
2424
from olive.model import CompositeModelHandler, ONNXModelHandler
25-
from olive.passes.onnx.onnx_dag import OnnxDAG
2625
from olive.passes.pass_config import BasePassConfig, PassConfigParam
2726
from olive.resource_path import LocalFile, LocalFolder
2827

@@ -539,12 +538,13 @@ def model_has_adapters_from_torchscript(
539538
:param model_path: The path to the model.
540539
:return: True if the model has adapters, False otherwise.
541540
"""
542-
dag = OnnxDAG(onnx.load(model_path, load_external_data=False))
543-
if adapter_type == AdapterType.LOHA and is_loha_model(dag):
541+
ir_model = ir.from_proto(onnx.load(model_path, load_external_data=False))
542+
if adapter_type == AdapterType.LOHA and is_loha_model(ir_model):
544543
return True
545544
else:
546-
for node_name in dag.get_node_names():
547-
op_type = dag.get_node_op_type(node_name)
545+
for node in ir_model.graph.all_nodes():
546+
op_type = node.op_type
547+
node_name = node.name or ""
548548
if (adapter_type == AdapterType.LORA and is_lora_node(op_type, node_name)) or (
549549
adapter_type == AdapterType.DORA and is_dora_node(op_type, node_name)
550550
):
@@ -564,10 +564,10 @@ def is_lora_node(op_type: str, node_name: str) -> bool:
564564
)
565565

566566

567-
def is_loha_model(dag: OnnxDAG) -> bool:
568-
for graph in dag.graphs:
569-
for initializer in graph.initializer:
570-
if any(re.match(pattern, initializer.name) for pattern in LOHA_NAME_PATTERNS_TORCHSCRIPT):
567+
def is_loha_model(ir_model: ir.Model) -> bool:
568+
for graph in ir_model.graphs():
569+
for initializer_name in graph.initializers:
570+
if any(re.match(pattern, initializer_name) for pattern in LOHA_NAME_PATTERNS_TORCHSCRIPT):
571571
return True
572572
return False
573573

olive/passes/onnx/compose.py

Lines changed: 136 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,9 @@
77
from typing import Optional, Union
88

99
import numpy as np
10+
import onnx_ir as ir
11+
from onnx_ir import serde
12+
from onnx_ir.passes.common import TopologicalSortPass
1013

1114
from olive.hardware.accelerator import AcceleratorSpec
1215
from olive.model import CompositeModelHandler, ONNXModelHandler
@@ -19,7 +22,6 @@
1922
model_proto_to_file,
2023
process_llm_pipeline,
2124
)
22-
from olive.passes.onnx.onnx_dag import OnnxDAG
2325
from olive.passes.pass_config import BasePassConfig, PassConfigParam
2426

2527
logger = logging.getLogger(__name__)
@@ -109,91 +111,165 @@ def _get_composed_model(
109111
:param as_model_dir: Use model parent directory as output model_path.
110112
:return: Composed ONNX model.
111113
"""
112-
dags = [OnnxDAG.from_model_path(path) for path in onnx_model_paths]
114+
115+
def shape_list(value: ir.Value):
116+
if value.shape is None:
117+
return None
118+
return [dim.value if isinstance(dim, ir.SymbolicDim) else dim for dim in value.shape]
119+
120+
ir_models = []
121+
for path in onnx_model_paths:
122+
ir_model = ir.load(path)
123+
TopologicalSortPass()(ir_model)
124+
ir_models.append(ir_model)
113125

114126
seen_inputs = set()
115127
seen_outputs = set()
116-
for dag in dags:
117-
dag_inputs = set(dag.get_input_names())
118-
dag_outputs = set(dag.get_output_names())
128+
for ir_model in ir_models:
129+
graph_inputs = {value.name for value in ir_model.graph.inputs}
130+
graph_outputs = {value.name for value in ir_model.graph.outputs}
119131
# avoid circular connection, model_2 output cannot be model_1 input
120-
assert dag_outputs.isdisjoint(seen_inputs), (
121-
f"Output names {dag_outputs.intersection(seen_inputs)} are already used as input names."
132+
assert graph_outputs.isdisjoint(seen_inputs), (
133+
f"Output names {graph_outputs.intersection(seen_inputs)} are already used as input names."
122134
)
123135
# avoid reused output name
124-
assert dag_outputs.isdisjoint(seen_outputs), (
125-
f"Output names {dag_outputs.intersection(seen_outputs)} are already used as output names."
136+
assert graph_outputs.isdisjoint(seen_outputs), (
137+
f"Output names {graph_outputs.intersection(seen_outputs)} are already used as output names."
126138
)
127139

128140
# update seen inputs and outputs
129-
seen_inputs.update(dag_inputs)
130-
seen_outputs.update(dag_outputs)
141+
seen_inputs.update(graph_inputs)
142+
seen_outputs.update(graph_outputs)
131143

132144
# will only keep the unused outputs
133145
# inputs will be automatically taken care of during compose
134146
final_outputs = seen_outputs - seen_inputs
135147

136-
# compose
137-
composed_dag = dags.pop(0)
138-
while dags:
139-
dag = dags.pop(0)
140-
141-
cd_input_names = set(composed_dag.get_input_names())
142-
cd_output_names = set(composed_dag.get_output_names())
143-
cd_initializer_names = set(composed_dag.get_initializer_names())
144-
for input_name in dag.get_input_names():
145-
if input_name in cd_input_names | cd_output_names:
146-
assert dag.get_io_shape(input_name) == composed_dag.get_io_shape(input_name), (
147-
f"Input shape mismatch: {input_name}"
148-
)
149-
assert dag.get_io_dtype(input_name) == composed_dag.get_io_dtype(input_name), (
150-
f"Input dtype mismatch: {input_name}"
151-
)
148+
# compose by relinking values across models by name
149+
composed_values: dict[str, ir.Value] = {}
150+
composed_inputs: list[ir.Value] = []
151+
composed_input_names: set[str] = set()
152+
composed_initializers: dict[str, ir.Value] = {}
153+
composed_nodes: list[ir.Node] = []
154+
composed_node_names: dict[str, ir.Node] = {}
155+
composed_output_names: list[str] = []
156+
produced_names: set[str] = set()
157+
158+
def get_value(name: str) -> ir.Value:
159+
value = composed_values.get(name)
160+
if value is None:
161+
value = ir.Value(name=name)
162+
composed_values[name] = value
163+
return value
164+
165+
for ir_model in ir_models:
166+
graph = ir_model.graph
167+
168+
for inp in graph.inputs:
169+
name = inp.name
170+
if name in composed_input_names or name in produced_names:
171+
# already a graph input or an internal connection from a previous model
172+
existing = composed_values[name]
173+
assert shape_list(inp) == shape_list(existing), f"Input shape mismatch: {name}"
174+
assert inp.dtype == existing.dtype, f"Input dtype mismatch: {name}"
152175
continue
153176

154-
# will add to graph 0 for now
155-
# this will fail if the connection already exists in the graph = expected behavior.
156-
composed_dag.add_input(dag.get_input_proto(input_name), 0)
157-
158-
for output_name in dag.get_output_names():
159-
composed_dag.add_output(dag.get_output_proto(output_name), 0)
160-
161-
for init_name in dag.get_initializer_names():
162-
if init_name in cd_initializer_names:
163-
(
164-
np.testing.assert_array_equal(
165-
dag.get_initializer_np_array(init_name), composed_dag.get_initializer_np_array(init_name)
166-
),
167-
f"Initializer mismatch: {init_name}",
177+
# will add to the composed graph inputs
178+
value = get_value(name)
179+
value.type = inp.type
180+
value.shape = inp.shape
181+
composed_inputs.append(value)
182+
composed_input_names.add(name)
183+
184+
for init in graph.initializers.values():
185+
name = init.name
186+
if name in composed_initializers:
187+
np.testing.assert_array_equal(
188+
init.const_value.numpy(),
189+
composed_initializers[name].const_value.numpy(),
190+
err_msg=f"Initializer mismatch: {name}",
168191
)
169192
continue
170193

171-
composed_dag.add_initializer(dag.get_initializer_proto(init_name), 0)
172-
173-
for intermediate_name in dag.get_intermediate_names():
174-
if value_info := dag.get_value_info_proto(intermediate_name):
175-
composed_dag.add_value_info(value_info, 0)
176-
177-
for node_name in dag.topological_sort():
178-
node = dag.get_node_proto(node_name)
179-
if composed_dag.has_node(node_name):
194+
value = get_value(name)
195+
value.const_value = init.const_value
196+
value.type = init.type if init.type is not None else ir.TensorType(init.const_value.dtype)
197+
value.shape = init.shape if init.shape is not None else ir.Shape(init.const_value.shape)
198+
composed_initializers[name] = value
199+
200+
for node in graph:
201+
name = node.name
202+
# ONNX node names are optional, so only dedup nodes that carry a non-empty name.
203+
# Unnamed nodes always get added to avoid different unnamed nodes colliding on the
204+
# same "" / None key (which could trigger false "Node mismatch" assertions).
205+
if name and name in composed_node_names:
180206
# there might be some dq nodes for initializers that are common between models
181207
# since split model keeps dq with the consumer op
182-
assert composed_dag.get_node_proto(node_name) == node, f"Node mismatch: {node_name}"
208+
assert (
209+
serde.serialize_node(composed_node_names[name]).SerializeToString()
210+
== serde.serialize_node(node).SerializeToString()
211+
), f"Node mismatch: {name}"
183212
continue
184213

185-
composed_dag.add_node(node, 0)
186-
187-
for output_name in composed_dag.get_output_names():
188-
if output_name not in final_outputs:
189-
composed_dag.remove_output(output_name)
190-
191-
# update the model graph
192-
composed_dag.update()
214+
new_inputs = [get_value(inp.name) if inp is not None else None for inp in node.inputs]
215+
new_node = ir.Node(
216+
node.domain,
217+
node.op_type,
218+
inputs=new_inputs,
219+
attributes=list(node.attributes.values()),
220+
overload=node.overload,
221+
num_outputs=len(node.outputs),
222+
version=node.version,
223+
name=name,
224+
)
225+
for old_output, new_output in zip(node.outputs, new_node.outputs):
226+
out_name = old_output.name
227+
if not out_name:
228+
continue
229+
new_output.name = out_name
230+
new_output.type = old_output.type
231+
new_output.shape = old_output.shape
232+
composed_values[out_name] = new_output
233+
produced_names.add(out_name)
234+
composed_nodes.append(new_node)
235+
if name:
236+
composed_node_names[name] = new_node
237+
238+
for out in graph.outputs:
239+
name = out.name
240+
value = composed_values.get(name)
241+
if value is None:
242+
value = get_value(name)
243+
value.type = out.type
244+
value.shape = out.shape
245+
if name not in composed_output_names:
246+
composed_output_names.append(name)
247+
248+
# merge opset imports across all component models, taking the max version per domain
249+
composed_opset_imports: dict[str, int] = {}
250+
for ir_model in ir_models:
251+
for domain, version in ir_model.graph.opset_imports.items():
252+
if domain not in composed_opset_imports or version > composed_opset_imports[domain]:
253+
composed_opset_imports[domain] = version
254+
255+
composed_graph = ir.Graph(
256+
inputs=composed_inputs,
257+
outputs=[composed_values[name] for name in composed_output_names if name in final_outputs],
258+
nodes=composed_nodes,
259+
initializers=list(composed_initializers.values()),
260+
name=ir_models[0].graph.name,
261+
opset_imports=composed_opset_imports,
262+
)
263+
composed_model = ir.Model(
264+
composed_graph,
265+
ir_version=ir_models[0].ir_version,
266+
producer_name="olive",
267+
)
268+
TopologicalSortPass()(composed_model)
193269

194270
# save the composed model
195271
output_model_path = Path(output_model_path)
196-
has_external_data = model_proto_to_file(composed_dag.model, output_model_path, **external_config)
272+
has_external_data = model_proto_to_file(ir.to_proto(composed_model), output_model_path, **external_config)
197273

198274
# copy over context binary files if any
199275
saved_cb_files = saved_cb_files if saved_cb_files is not None else {}

olive/passes/onnx/float16_conversion.py

Lines changed: 20 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,13 @@
55
from pathlib import Path
66
from typing import Union
77

8+
import onnx_ir as ir
9+
810
from olive.hardware.accelerator import AcceleratorSpec
911
from olive.model import ONNXModelHandler
1012
from olive.model.utils import resolve_onnx_path
1113
from olive.passes import Pass
1214
from olive.passes.onnx.common import get_external_data_config, model_proto_to_olive_model
13-
from olive.passes.onnx.onnx_dag import OnnxDAG
1415
from olive.passes.pass_config import BasePassConfig, PassConfigParam
1516

1617

@@ -67,20 +68,27 @@ def _run_for_config(
6768
output_model_path = resolve_onnx_path(output_model_path, Path(model.model_path).name)
6869

6970
loaded_model = model.load_model()
70-
dag = OnnxDAG(loaded_model)
7171

7272
op_block_list = config.op_block_list
73-
if config.op_include_list:
74-
if op_block_list is not None:
75-
raise ValueError("op_include_list and op_block_list are mutually exclusive.")
76-
op_block_list = [op_type for op_type in dag.get_node_op_types() if op_type not in config.op_include_list]
7773
node_block_list = config.node_block_list
78-
if config.node_include_list:
79-
if node_block_list is not None:
80-
raise ValueError("node_include_list and node_block_list are mutually exclusive.")
81-
node_block_list = [
82-
node_name for node_name in dag.get_node_names() if node_name not in config.node_include_list
83-
]
74+
if config.op_include_list or config.node_include_list:
75+
ir_model = ir.from_proto(loaded_model)
76+
all_nodes = list(ir_model.graph.all_nodes())
77+
if config.op_include_list:
78+
if op_block_list is not None:
79+
raise ValueError("op_include_list and op_block_list are mutually exclusive.")
80+
op_block_list = list({node.op_type for node in all_nodes if node.op_type not in config.op_include_list})
81+
if config.node_include_list:
82+
if node_block_list is not None:
83+
raise ValueError("node_include_list and node_block_list are mutually exclusive.")
84+
# node_include_list works by name, so every node must be named; otherwise unnamed
85+
# nodes would silently be converted to float16 even though they can't be included.
86+
if any(not node.name for node in all_nodes):
87+
raise ValueError(
88+
"node_include_list requires all nodes to be named, but the model contains unnamed nodes. "
89+
"Please ensure all nodes in the model have names or use node_block_list instead."
90+
)
91+
node_block_list = [node.name for node in all_nodes if node.name not in config.node_include_list]
8492

8593
# using the float16 converter from onnxruntime since it is regularly updated
8694
# and can handle large models (>2GB) as well as ort contrib ops

0 commit comments

Comments
 (0)