Skip to content

Commit efead9d

Browse files
committed
Cortex-M backend: address #19470 review feedback
Aligns CortexMTargetConfig with the design Erik proposes in #19505 while keeping the wider plumbing in place. The earlier CortexMCompileConfig is renamed to CortexMTargetConfig (and its file moved to target_config.py) to disambiguate from EdgeCompileConfig — this dataclass models a compilation *target*, not a step in the compile pipeline. Adopted from Erik's feedback: * CortexM enum replaces the Cpu/Isa Literals — typo-safe and IDE-friendly. * `.backend` property returns `cmsis_nn.Backend` directly, resolved via `cmsis_nn.resolve_backend(cmsis_nn.CortexM.<X>)`. The hand-rolled `_CPU_DEFAULT_ISA` dict is gone — cmsis_nn is the single source of truth for the CPU → backend mapping. * CortexMPass abstract base class added; CortexMPassManager.transform() uses signature inspection to inject both `exported_program` and `target_config` into passes that declare them (mirroring Erik's proposal). The pass manager also gains stricter validation — the exported_program must be a real ExportedProgram and the pass list must contain classes, not instances — failing fast instead of producing opaque errors deep in _transform. * cmsis_nn is now a hard dependency for the cortex_m tests: the top-level `import cmsis_nn` in test_target_config.py replaces the previous skipif-on-find_spec dance, addressing Erik's concern that skipping tests on missing deps can mask regressions. * `+int8` dropped from cortex-m target strings — quantization is a result of the export flow, not a CPU attribute. TARGETS, help text, from_target_string, CI script and README aligned. * Logging in `_to_edge_cortex_m` and the --delegate-ignored warning switched to f-strings. * `__init__` docstring on CortexMPassManager documents the exported_program / passes / target_config defaults (including the M55+MVE fallback that matches pre-config behaviour). * `import-not-found` removed from the cmsis_nn type-ignore — only `import-untyped` actually fires, and if cmsis_nn ever ships stubs the unused ignore will become a tripwire. Kept the optional `isa` override field for the optional-extension cases (M55 without MVE, M33 without DSP, etc.) — different from Erik's enum-only design, but the override remains useful for cores where ISA extensions are optional. A `_SUPPORTED_BACKENDS` table encodes the per-CPU architectural capability set so overrides validate at construction; forcing MVE on an M0 raises ValueError with the actual supported list. The SCALAR ⊂ DSP ⊂ MVE supersession reflects that an MVE-capable core also runs DSP and scalar code. Defers Erik's `ANY` proposal. In #19505 ANY falls back to MVE, but an honest "any cortex-m" choice would have to do worst-case scratch buffer planning across the ISA classes (which may not be MVE). Deferring until the scratch-buffer side lands and we can implement the worst-case analysis properly. Authored with Claude.
1 parent 3000f9c commit efead9d

11 files changed

Lines changed: 342 additions & 253 deletions

File tree

.ci/scripts/test_cortex_m_e2e.sh

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,9 +17,9 @@ MODEL=$1
1717
script_dir=$(realpath "$(dirname "${BASH_SOURCE[0]}")")
1818
et_root_dir=$(realpath "${script_dir}/../..")
1919

20-
# Quantization is the default for the cortex-m55+int8 target; run.sh's
20+
# Quantization is the default for the cortex-m55 target; run.sh's
2121
# arg parser only recognizes --no_quantize, so we omit any explicit flag.
2222
bash "${et_root_dir}/examples/arm/run.sh" \
2323
--model_name="${MODEL}" \
24-
--target=cortex-m55+int8 \
24+
--target=cortex-m55 \
2525
--bundleio

backends/arm/scripts/aot_arm_compiler.py

Lines changed: 22 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -33,13 +33,13 @@
3333
from executorch.backends.arm.util._factory import create_partitioner, create_quantizer
3434

3535
from executorch.backends.arm.vgf import VgfCompileSpec
36-
from executorch.backends.cortex_m.compile_config import CortexMCompileConfig
3736
from executorch.backends.cortex_m.passes.cortex_m_pass_manager import CortexMPassManager
3837

3938
from executorch.backends.cortex_m.passes.replace_quant_nodes_pass import (
4039
ReplaceQuantNodesPass,
4140
)
4241
from executorch.backends.cortex_m.quantizer.quantizer import CortexMQuantizer
42+
from executorch.backends.cortex_m.target_config import CortexMTargetConfig
4343
from executorch.devtools import BundledProgram, generate_etrecord
4444
from executorch.devtools.backend_debug import get_delegation_info
4545
from executorch.devtools.bundled_program.config import MethodTestCase, MethodTestSuite
@@ -466,17 +466,16 @@ def forward(self, x):
466466
"TOSA-1.0+INT",
467467
"TOSA-1.0+FP",
468468
"TOSA-1.0+INT+int16",
469-
"cortex-m0+int8",
470-
"cortex-m0plus+int8",
471-
"cortex-m3+int8",
472-
"cortex-m4+int8",
473-
"cortex-m7+int8",
474-
"cortex-m23+int8",
475-
"cortex-m33+int8",
476-
"cortex-m35p+int8",
477-
"cortex-m52+int8",
478-
"cortex-m55+int8",
479-
"cortex-m85+int8",
469+
"cortex-m0",
470+
"cortex-m0plus",
471+
"cortex-m3",
472+
"cortex-m4",
473+
"cortex-m7",
474+
"cortex-m23",
475+
"cortex-m33",
476+
"cortex-m35p",
477+
"cortex-m55",
478+
"cortex-m85",
480479
]
481480

482481

@@ -577,7 +576,7 @@ def _get_args():
577576
required=False,
578577
default="ethos-u55-128",
579578
choices=TARGETS,
580-
help=f"Target backend. For delegated models: Ethos-U/VGF/TOSA variants. For non-delegated: cortex-m<variant>+int8 (CMSIS-NN portable kernels). Valid targets: {TARGETS}",
579+
help=f"Target backend. For delegated models: Ethos-U/VGF/TOSA variants. For non-delegated: cortex-m<variant> (CMSIS-NN portable kernels). Valid targets: {TARGETS}",
581580
)
582581
# TODO: Remove --evaluate and --evaluate_config completely after a suitable time.
583582
# They are deprecated and no longer functional in this script.
@@ -871,13 +870,12 @@ def _to_edge_cortex_m(
871870
model: GraphModule,
872871
example_inputs: Tuple[torch.Tensor],
873872
calibration_samples: Optional[List[Tuple[torch.Tensor, ...]]],
874-
config: CortexMCompileConfig,
873+
target_config: CortexMTargetConfig,
875874
):
876875
"""Cortex-M/CMSIS-NN compilation path with no delegation."""
877876
logging.info(
878-
"Using Cortex-M/CMSIS-NN compilation path for cpu=%s isa=%s",
879-
config.cpu,
880-
config.isa,
877+
f"Using Cortex-M/CMSIS-NN compilation path for cpu={target_config.cpu.name} "
878+
f"backend={target_config.backend.name}"
881879
)
882880

883881
def _to_channels_last(x):
@@ -931,7 +929,9 @@ def _to_channels_last(x):
931929
),
932930
)
933931

934-
pass_manager = CortexMPassManager(edge.exported_program(), config=config)
932+
pass_manager = CortexMPassManager(
933+
edge.exported_program(), target_config=target_config
934+
)
935935
edge._edge_programs["forward"] = pass_manager.transform()
936936

937937
return model_quant, edge
@@ -1025,12 +1025,11 @@ def main() -> None: # noqa: C901
10251025

10261026
if args.target.startswith("cortex-m"):
10271027
# Cortex-M path: CMSIS-NN portable kernels, no delegation
1028-
cortex_m_config = CortexMCompileConfig.from_target_string(args.target)
1028+
target_config = CortexMTargetConfig.from_target_string(args.target)
10291029
if args.delegate:
10301030
logging.warning(
1031-
"--delegate is ignored for target %r "
1032-
"(this target does not use delegated ops).",
1033-
args.target,
1031+
f"--delegate is ignored for target {args.target!r} "
1032+
"(this target does not use delegated ops)."
10341033
)
10351034
args.delegate = False
10361035
model_quant, edge = _to_edge_cortex_m(
@@ -1039,7 +1038,7 @@ def main() -> None: # noqa: C901
10391038
model,
10401039
example_inputs,
10411040
calibration_samples,
1042-
cortex_m_config,
1041+
target_config,
10431042
)
10441043
elif args.delegate:
10451044
# As we can target multiple output encodings, one must

backends/cortex_m/compile_config.py

Lines changed: 0 additions & 98 deletions
This file was deleted.

backends/cortex_m/passes/__init__.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ def _ensure_cortex_m_dependencies() -> None:
3636
from .activation_fusion_pass import ActivationFusionPass # noqa
3737
from .clamp_hardswish_pass import ClampHardswishPass # noqa
3838
from .convert_to_cortex_m_pass import ConvertToCortexMPass # noqa
39+
from .cortex_m_pass import CortexMPass # noqa
3940
from .decompose_hardswish_pass import DecomposeHardswishPass # noqa
4041
from .decompose_mean_pass import DecomposeMeanPass # noqa
4142
from .quantized_clamp_activation_pass import QuantizedClampActivationPass # noqa
Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# Copyright (c) Meta Platforms, Inc. and affiliates.
2+
# All rights reserved.
3+
#
4+
# This source code is licensed under the BSD-style license found in the
5+
# LICENSE file in the root directory of this source tree.
6+
7+
from executorch.backends.cortex_m.target_config import CortexMTargetConfig
8+
from executorch.exir.pass_base import ExportPass
9+
from torch.export import ExportedProgram
10+
11+
12+
class CortexMPass(ExportPass):
13+
"""Base class for passes that need the Cortex-M target config.
14+
15+
Passes that subclass this declare `exported_program` and `target_config`
16+
in their `__init__`; `CortexMPassManager.transform()` injects both
17+
automatically when running the pass list.
18+
"""
19+
20+
def __init__(
21+
self,
22+
exported_program: ExportedProgram,
23+
target_config: CortexMTargetConfig,
24+
) -> None:
25+
super().__init__()
26+
self._exported_program = exported_program
27+
self._target_config = target_config
28+
29+
@property
30+
def exported_program(self) -> ExportedProgram:
31+
return self._exported_program
32+
33+
@property
34+
def target_config(self) -> CortexMTargetConfig:
35+
return self._target_config

backends/cortex_m/passes/cortex_m_pass_manager.py

Lines changed: 43 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -5,13 +5,13 @@
55

66

77
import inspect
8-
from typing import Callable, cast, Optional, Type
8+
from typing import Any, Optional, Type
99

1010
from executorch.backends.arm._passes import (
1111
FoldAndAnnotateQParamsPass,
1212
ScalarsToAttributePass,
1313
)
14-
from executorch.backends.cortex_m.compile_config import CortexMCompileConfig
14+
from executorch.backends.cortex_m.target_config import CortexM, CortexMTargetConfig
1515
from executorch.backends.transforms.remove_getitem_op import RemoveGetItemPass
1616
from executorch.backends.transforms.replace_scalar_with_tensor import (
1717
ReplaceScalarWithTensorArgPass,
@@ -20,9 +20,6 @@
2020
from executorch.exir.pass_manager import PassManager
2121
from executorch.exir.program._program import _transform, lift_constant_tensor_pass
2222
from torch.export import ExportedProgram
23-
from torch.fx.passes.infra.pass_base import PassResult
24-
25-
from torch.nn import Module
2623

2724
from .activation_fusion_pass import ActivationFusionPass
2825
from .clamp_hardswish_pass import ClampHardswishPass
@@ -59,17 +56,32 @@ class CortexMPassManager(PassManager):
5956

6057
def __init__(
6158
self,
62-
exported_program,
59+
exported_program: ExportedProgram | None,
6360
passes: Optional[list[PassClass]] = None,
64-
config: Optional[CortexMCompileConfig] = None,
61+
target_config: Optional[CortexMTargetConfig] = None,
6562
) -> None:
63+
"""Initialize the Cortex-M pass manager.
64+
65+
Args:
66+
exported_program: The exported program to transform. Required
67+
before calling ``transform()``; may be ``None`` for callers
68+
that only use ``transform_for_annotation()``.
69+
passes: Optional override of the pass list. Defaults to
70+
``CortexMPassManager.pass_list``.
71+
target_config: Compilation target for passes that need it.
72+
Defaults to ``CortexMTargetConfig(cpu=CortexM.M55)``, which
73+
resolves through cmsis_nn to the MVE backend — matching the
74+
pre-config historical behaviour.
75+
"""
6676
super().__init__(passes=[])
6777
self.exported_program = exported_program
6878
# PassManager.passes is typed as callables; this manager stores pass classes which are initialized at transform time with the exported_program.
6979
self.passes: list[PassClass] = ( # type: ignore[assignment]
7080
passes if passes is not None else self.pass_list # type: ignore[assignment]
7181
)
72-
self.config: CortexMCompileConfig = config or CortexMCompileConfig()
82+
self.target_config: CortexMTargetConfig = target_config or CortexMTargetConfig(
83+
cpu=CortexM.M55
84+
)
7385

7486
def transform_for_annotation(self, model):
7587
passes = self.pass_list_transform_for_annotation
@@ -78,18 +90,31 @@ def transform_for_annotation(self, model):
7890
return model
7991

8092
def transform(self) -> ExportedProgram:
81-
ep = self.exported_program
93+
exported_program = self.exported_program
94+
if not isinstance(exported_program, ExportedProgram):
95+
raise ValueError(
96+
f"{type(self).__name__}.transform() needs a real ExportedProgram, "
97+
f"got {exported_program!r}"
98+
)
99+
82100
for pass_cls in self.passes:
101+
if not isinstance(pass_cls, type):
102+
raise ValueError(
103+
f"{type(self).__name__} expects pass classes, not instances; "
104+
f"got {pass_cls!r}"
105+
)
106+
83107
signature = inspect.signature(pass_cls)
108+
kwargs: dict[str, Any] = {}
84109
if "exported_program" in signature.parameters:
85-
ep_pass_ctor = cast(Callable[[ExportedProgram], ExportPass], pass_cls)
86-
transform_pass = ep_pass_ctor(ep)
87-
else:
88-
transform_pass = pass_cls()
89-
pass_callable = cast(Callable[[Module], PassResult], transform_pass)
90-
ep = _transform(ep, pass_callable)
110+
kwargs["exported_program"] = exported_program
111+
if "target_config" in signature.parameters:
112+
kwargs["target_config"] = self.target_config
113+
114+
transform_pass = pass_cls(**kwargs)
115+
exported_program = _transform(exported_program, transform_pass)
91116

92117
# All constant tensors should be lifted to buffers at this point, re-run
93-
# lift_constant_tensor_pass in case new ones have been introduced by the passes above.
94-
ep = lift_constant_tensor_pass(ep)
95-
return ep
118+
# lift_constant_tensor_pass in case new ones have been introduced.
119+
exported_program = lift_constant_tensor_pass(exported_program)
120+
return exported_program

0 commit comments

Comments
 (0)