NXP backend: Unify testing infrastructure. by MartinPavella · Pull Request #19112 · pytorch/executorch

MartinPavella · 2026-04-24T16:29:58Z

Summary

Unify infrastructure for tests comparing edge dialect <-> Neutron IR with tests comparing lowered edge programs <-> lowered Neutron delegated programs.

Test plan

Tested by all existing tests.

cc @robert-kalmar @JakeStevens @digantdesai

pytorch-bot · 2026-04-24T16:30:04Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19112

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

Rolling out OSDC (ARC) runners on pull & trunk workflows in PyTorch main

❌ 15 New Failures

As of commit 52cac91 with merge base 32a6cec ():

NEW FAILURES - The following jobs have failed:

pull / test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job (gh)
RuntimeError: Command docker exec -t 29a926e0399a37a602697a3b1995ea841abb072d9904908157900b86090ffb5c /exec failed with exit code 1
pull / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh)
RuntimeError: Command docker exec -t 19ef4b38abb84f93f0a268b8a1537c8a612b506576f8ec00070ccedb4811b611 /exec failed with exit code 1
pull / test-qnn-models-linux (dl3) / linux-job (gh)
RuntimeError: Command docker exec -t 53c48d0888376e9d2125d443c019e982c4b72c058940cde0a991b384c3c7b38a /exec failed with exit code 1
pull / test-qnn-models-linux (mv2) / linux-job (gh)
RuntimeError: Command docker exec -t e581294c2d2e375dbf139153a108354d2c96190688f25d043e626bc04749d355 /exec failed with exit code 1
pull / test-qnn-models-linux (mv3) / linux-job (gh)
RuntimeError: Command docker exec -t aaa9106bb5d42e011a5cfbce1b700f6d34273ea08b6a71719ac3ca005319280f /exec failed with exit code 1
pull / test-sqnr-static-llm-qnn-linux (smollm2_135m) / linux-job (gh)
test_static_llm_model
pull / test-static-llama-qnn-linux (stories_110m) / linux-job (gh)
test_llama_stories_110m
pull / test-static-llama-qnn-linux (stories_260k_bc) / linux-job (gh)
test_llama_stories_260k
pull / unittest / linux / linux-job (gh)
backends/xnnpack/test/passes/test_propagate_custom_meta_pass.py::TestPropagateCustomMetaPass::test_quantize_
pull / unittest / macos / macos-job (gh)
exir/_serialize/test/test_flatbuffer_program.py::TestFlatbufferProgram::test_bad_alignment_fails
pull / unittest / windows / windows-job (gh)
Process completed with exit code 2.
pull / unittest-editable / linux / linux-job (gh)
export/tests/test_target_recipes.py::TestTargetRecipes::test_w2l_model
pull / unittest-editable / macos / macos-job (gh)
backends/xnnpack/test/recipes/test_xnnpack_recipes.py::TestXnnpackRecipes::test_all_models_with_recipes
pull / unittest-editable / windows / windows-job (gh)
Process completed with exit code 2.
pull / unittest-nxp-neutron / linux-job (gh)
RuntimeError: Command docker exec -t 20fa528f232eb8fd17500b6c7b2fabd7e536493d5e33762fa4584614d2cd796d /exec failed with exit code 3

This comment was automatically generated by Dr. CI and updates every 15 minutes.

github-actions · 2026-04-24T16:30:44Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

MartinPavella · 2026-04-24T16:31:21Z

Corresponding internal PR is here.
The internal CI build is here.

…m` duplicates.

roman-janik-nxp · 2026-04-27T12:35:00Z

 ) -> EdgeProgramManager:
    calibration_inputs = get_random_calibration_inputs(to_model_input_spec(input_spec))

    example_input = calibration_inputs[0]


Probably should also use
example_input = _get_example_input(input_spec)

roman-janik-nxp · 2026-04-27T12:41:29Z

                    case _:
                        raise ValueError(f"Unsupported dim_order: {spec.dim_order}")

                sample_vector = rng.random(np.prod(shape), spec.type).reshape(shape)


Use

from executorch.backends.nxp.backend.ir.converter.conversion.translator import torch_type_to_numpy_type` sample_vector = rng.random(np.prod(shape), torch_type_to_numpy_type(spec.dtype)).reshape(shape)

roman-janik-nxp · 2026-04-27T12:41:40Z

 @dataclass
 class ModelInputSpec:
    shape: tuple[int, ...]
+    type: np.dtype = np.float32


Please don't add separate numpy type, keep only torch type. Adding numpy type means duplicity and space for unequal numpy and torch types. It was pointed to me by some external dev I don't remember when I added it several months ago, so I removed it. I use conversion function
from executorch.backends.nxp.backend.ir.converter.conversion.translator import torch_type_to_numpy_type
in dataset_creator.py.
https://github.com/pytorch/executorch/pull/19112/changes#r3147341415

roman-janik-nxp · 2026-04-27T12:44:20Z

👍 for moving tests to generic dir.

roman-janik-nxp · 2026-04-27T12:49:51Z

                        raise ValueError(f"Unsupported dim_order: {spec.dim_order}")

                sample_vector = rng.random(np.prod(shape), spec.type).reshape(shape)
                sample_vector.tofile(


Suggested change

file_name = f"{str(spec_idx).zfill(2)}.bin" if len(input_spec) > 1 else f"{str(idx).zfill(4)}.bin"

sample_vector.tofile(os.path.join(sample_dir, file_name))

This is the fix data creation for single model input.

roman-janik-nxp · 2026-04-27T12:53:41Z

Are those inits needed or were they added by IDE?

Change type: ignore[import-not-found] to type: ignore[import] to match MYPY's expected error code for these dynamic module imports.

MartinPavella requested review from StrycekSimon and roman-janik-nxp April 24, 2026 16:29

MartinPavella self-assigned this Apr 24, 2026

MartinPavella requested a review from robert-kalmar as a code owner April 24, 2026 16:29

MartinPavella added the module: nxp Issues related to NXP Neutron NPU delegation and code under backends/nxp/ label Apr 24, 2026

meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 24, 2026

JakeStevens reviewed Apr 25, 2026

View reviewed changes

Comment thread backends/nxp/tests/generic_tests/test_aot_example.py Outdated

MartinPavella force-pushed the nxg01483/EIEX-896-unify-lowering-and-testing-functions branch 2 times, most recently from b1d018f to c213c9f Compare April 27, 2026 06:02

MartinPavella mentioned this pull request Apr 27, 2026

NXP backend: Test avg_pool2d with new Neutron flow. #19147

Draft

MartinPavella added 2 commits April 27, 2026 10:12

Unify to_quantized_edge_program and `to_quantized_executorch_progra…

532036f

…m` duplicates.

Move generic tests into a separate subdirectory.

3d483d8

MartinPavella force-pushed the nxg01483/EIEX-896-unify-lowering-and-testing-functions branch 2 times, most recently from 624eae4 to 61606df Compare April 27, 2026 08:27

Move files from /tests_models to /tests.

9d66af7

MartinPavella force-pushed the nxg01483/EIEX-896-unify-lowering-and-testing-functions branch 2 times, most recently from 7171bff to 8f5cc1d Compare April 27, 2026 11:44

MartinPavella requested review from Gasoonjia, kirklandsign, larryliu0820 and manuelcandales as code owners April 27, 2026 12:15

roman-janik-nxp reviewed Apr 27, 2026

View reviewed changes

MartinPavella force-pushed the nxg01483/EIEX-896-unify-lowering-and-testing-functions branch from ea50847 to 352195d Compare April 27, 2026 13:16

MartinPavella added 3 commits April 27, 2026 15:40

Update GitHub CI nxp-build-test job.

6dda0bf

Set PROJECT_DIR variable for unittest tests.

03fd99c

Fix MYPY type ignore comments for dynamic imports.

52cac91

Change type: ignore[import-not-found] to type: ignore[import] to match MYPY's expected error code for these dynamic module imports.

MartinPavella force-pushed the nxg01483/EIEX-896-unify-lowering-and-testing-functions branch from 352195d to 52cac91 Compare April 27, 2026 13:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NXP backend: Unify testing infrastructure.#19112

NXP backend: Unify testing infrastructure.#19112
MartinPavella wants to merge 6 commits intopytorch:mainfrom
nxp-upstream:nxg01483/EIEX-896-unify-lowering-and-testing-functions

MartinPavella commented Apr 24, 2026 •

edited

Loading

Uh oh!

pytorch-bot Bot commented Apr 24, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Apr 24, 2026

Uh oh!

MartinPavella commented Apr 24, 2026 •

edited

Loading

Uh oh!

Uh oh!

roman-janik-nxp Apr 27, 2026

Uh oh!

roman-janik-nxp Apr 27, 2026

Uh oh!

roman-janik-nxp Apr 27, 2026

Uh oh!

roman-janik-nxp Apr 27, 2026

Uh oh!

roman-janik-nxp Apr 27, 2026

Uh oh!

roman-janik-nxp Apr 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants


	file_name = f"{str(spec_idx).zfill(2)}.bin" if len(input_spec) > 1 else f"{str(idx).zfill(4)}.bin"
	sample_vector.tofile(os.path.join(sample_dir, file_name))

Conversation

MartinPavella commented Apr 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

pytorch-bot Bot commented Apr 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19112

❗ 1 Active SEVs

❌ 15 New Failures

Uh oh!

github-actions Bot commented Apr 24, 2026

This PR needs a release notes: label

Uh oh!

MartinPavella commented Apr 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

roman-janik-nxp Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

roman-janik-nxp Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

roman-janik-nxp Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

roman-janik-nxp Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

roman-janik-nxp Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

roman-janik-nxp Apr 27, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

MartinPavella commented Apr 24, 2026 •

edited

Loading

pytorch-bot Bot commented Apr 24, 2026 •

edited

Loading

This PR needs a `release notes:` label

MartinPavella commented Apr 24, 2026 •

edited

Loading