Add dynamic shape tests for xnnpack model tests#18701
Add dynamic shape tests for xnnpack model tests#18701GregoryComer merged 2 commits intopytorch:mainfrom
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18701
Note: Links to docs will display an error until the docs builds have been completed. ❗ 1 Active SEVsThere are 1 currently active SEVs. If your PR is affected, please view them below: ❌ 1 New Failure, 2 Unrelated FailuresAs of commit 3e2193a with merge base ac68932 ( NEW FAILURE - The following job has failed:
BROKEN TRUNK - The following jobs failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
@pytorchbot label "release notes: xnnpack" |
|
@Lidang-Jiang Thanks for the contribution. Looks good. Can you resolve the lint failure (run link)? I can go ahead and merge once the lint job is green. |
Add dynamic shape support to xnnpack model tests that previously only tested with static inputs. This covers DeepLab V3, EDSR, Inception V3/V4, Emformer RNNT, and MobileBERT using the same DynamicWrapper + interpolate pattern established by ResNet and ViT. Addresses pytorch#11585 Signed-off-by: Lidang-Jiang <lidangjiang@gmail.com>
Signed-off-by: Lidang-Jiang <lidangjiang@gmail.com>
d3be8be to
3e2193a
Compare
|
Good catch, fixed in 3e2193a. I rebased onto latest |
|
I'll go ahead and merge. Thanks for the contribution! |
## Summary Add dynamic shape tests for xnnpack model tests that previously only tested with static inputs, as requested in pytorch#11585. **Models covered:** - **DeepLab V3** — dynamic height/width with DynamicWrapper + interpolate - **EDSR** — dynamic height/width with DynamicWrapper + interpolate - **Inception V3/V4** — dynamic height/width with DynamicWrapper + interpolate - **Emformer RNNT** — dynamic batch dimension for joiner and transcriber - **MobileBERT** — dynamic sequence length All dynamic tests follow the established pattern from `resnet.py` (`DynamicResNet`) and `torchvision_vit.py` (`DynamicViT`): a wrapper module that resizes dynamic spatial inputs to fixed dimensions via `F.interpolate` before feeding into the model. **Models already covered by existing dynamic tests (no changes):** - MobileNet V2, MobileNet V3, ResNet, ViT **Models skipped:** - `llama2_et_example.py` — LLM, requires separate dynamic shape strategy - `very_big_model.py` — synthetic test model <details> <summary>Before (only static tests)</summary> ``` $ python -m pytest backends/xnnpack/test/models/deeplab_v3.py -v backends/xnnpack/test/models/deeplab_v3.py::TestDeepLabV3::test_fp32_dl3 PASSED (no dynamic shape test exists) ``` </details> <details> <summary>After (static + dynamic tests pass)</summary> ``` $ python -m pytest backends/xnnpack/test/models/deeplab_v3.py::TestDeepLabV3::test_fp32_dl3_dynamic -v backends/xnnpack/test/models/deeplab_v3.py::TestDeepLabV3::test_fp32_dl3_dynamic PASSED [100%] ======================== 1 passed in 28.84s ============================== $ python -m pytest backends/xnnpack/test/models/mobilebert.py::TestMobilebert::test_fp32_mobilebert_dynamic -v backends/xnnpack/test/models/mobilebert.py::TestMobilebert::test_fp32_mobilebert_dynamic PASSED [100%] ======================== 1 passed in 160.79s (0:02:40) =================== ``` </details> ## Test plan - [x] `test_fp32_dl3_dynamic` passes locally - [x] `test_fp32_mobilebert_dynamic` passes locally - [ ] CI xnnpack test suite passes cc @GregoryComer @digantdesai @cbilgin --------- Signed-off-by: Lidang-Jiang <lidangjiang@gmail.com>
Summary
Add dynamic shape tests for xnnpack model tests that previously only tested with static inputs, as requested in #11585.
Models covered:
All dynamic tests follow the established pattern from
resnet.py(DynamicResNet) andtorchvision_vit.py(DynamicViT): a wrapper module that resizes dynamic spatial inputs to fixed dimensions viaF.interpolatebefore feeding into the model.Models already covered by existing dynamic tests (no changes):
Models skipped:
llama2_et_example.py— LLM, requires separate dynamic shape strategyvery_big_model.py— synthetic test modelBefore (only static tests)
After (static + dynamic tests pass)
Test plan
test_fp32_dl3_dynamicpasses locallytest_fp32_mobilebert_dynamicpasses locallycc @GregoryComer @digantdesai @cbilgin