Skip to content

Commit b4603d2

Browse files
Ninja91facebook-github-bot
authored andcommitted
Add a16w8 reduce_sum FVP coverage for Ethos-U85
Summary: Adds an a16w8 (int16 IO + int8 weights) sweep for `aten.sum.dim_IntList` reducing the last dim with `keepdim=True`. The new tests `test_sum_dim_intlist_a16w8_{u55,u85}_INT` run on the standard Corstone-300 / Corstone-320 FVP harness and surface a numerics issue in the Ethos-U85 `ReduceSum` lowering at int16 IO precision (silent zero output). The Ethos-U55 path uses a different accumulator and is correct on the same OFM rescale. ## Context Part of a stack that documents and fixes a numerics bug in the Vela 5.0 Ethos-U85 backend (`regor`). Plan + cross-references: - **Plan:** {D103649006} ([Markup](https://internalfb.com/intern/markup/D103649006)) - **Step 1a (this diff):** ReduceSum-only a16w8 coverage in `test_sum.py` (LAND) - **Step 1b-softmax:** {D103734699} -- `test_softmax.py` a16w8 MHA softmax sweep (LAND) - **Step 1b-ops:** {D103760103} -- `test_softmax_ops.py` op-isolation harness (DNL) - **Step 2a:** {D103760153} -- `regor` patch in third-party Vela 5.0 fork (LAND) - **Step 2b:** {D103760514} -- DNL companion that drops `xfails=` from `test_sum.py` (lands in OSS only after upstream Vela syncs the fix) ## Test design Tests use the standard `pipeline.run()` with the same a16w8 kwargs other arm a16w8 tests use (e.g. `test_native_layer_norm_16a8w_u85_INT` in `test_layer_norm.py`): ``` a16w8_quantization=True, symmetric_io_quantization=True, qtol=128, epsilon=2**-16 ``` Numerical comparison is the standard `atol`/`rtol`-only check from `pipeline.run()` -- no SQNR helpers -- to stay consistent with the rest of `arm/test/ops/`. The U85 cases are wrapped with `xfails=a16w8_sum_u85_xfails, strict=False`. `strict=False` keeps the test target green both on stock Vela 5.0 (cases XFAIL) *and* after Step 2a lands the Vela patch (cases XPASS, allowed under non-strict). Step 2b separately drops the `xfails=` argument once the upstream Vela fix syncs down. The new U85 a16w8 test deliberately omits `common.XfailIfNoCorstone320` (which is present on the U55 sibling). Stacking that decorator with the per-id `xfails=` argument makes the per-id marks not fire (verified empirically) so the bug-firing cases would hard-fail instead of XFAIL. CI always has Corstone-320 installed; if it ever isn't, the test fails loudly with `FileNotFoundError`, which is the right signal for a missing-FVP misconfiguration. A code comment in the file documents this constraint. ## Scope note This diff only **adds** new tests for the a16w8 path. It does not modify any existing tests in `test_sum.py` -- the pre-existing `Sum.test_parameters` (including the `dim_None` cases) is left as-is. Pre-existing `dim_None` test failures on `test_sum_u{55,85}_INT_1_0` are out of scope and unrelated to this diff. Differential Revision: D103667823
1 parent 0f9de6a commit b4603d2

2 files changed

Lines changed: 66 additions & 0 deletions

File tree

backends/arm/test/ops/test_sum.py

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -220,3 +220,68 @@ def test_sum_tosa_FP(test_data: Callable[[], input_t2]):
220220
def test_sum_tosa_INT(test_data: Callable[[], input_t2]):
221221
pipeline = TosaPipelineINT[input_t1](SumDefault(), test_data(), SumDefault.aten_op)
222222
pipeline.run()
223+
224+
225+
# a16w8 (int16 IO + int8 weights) coverage for sum.dim_IntList. Surfaces the
226+
# Ethos-U85 int16 ReduceSum silent-zero issue tracked upstream at
227+
# https://gitlab.arm.com/artificial-intelligence/ethos-u/ethos-u-vela/-/issues/23.
228+
229+
230+
class SumLastDim(torch.nn.Module):
231+
"""Reduce the last dim with keepdim=True."""
232+
233+
def forward(self, x: torch.Tensor) -> torch.Tensor:
234+
return x.sum(dim=-1, keepdim=True)
235+
236+
237+
a16w8_sum_test_parameters = {
238+
"rank1_16": lambda: (torch.rand(16),),
239+
"rank3_8x1x16": lambda: (torch.rand(8, 1, 16),),
240+
"rank3_4x4x16": lambda: (torch.rand(4, 4, 16),),
241+
}
242+
243+
244+
@common.parametrize("test_data", a16w8_sum_test_parameters)
245+
@common.XfailIfNoCorstone300
246+
def test_sum_dim_intlist_a16w8_u55_INT(test_data):
247+
pipeline = EthosU55PipelineINT[input_t1](
248+
SumLastDim(),
249+
test_data(),
250+
aten_op,
251+
exir_ops=[],
252+
a16w8_quantization=True,
253+
symmetric_io_quantization=True,
254+
qtol=128,
255+
epsilon=2**-16,
256+
)
257+
pipeline.run()
258+
259+
260+
# Upstream Vela issue #23 (linked above). strict=False so the test target
261+
# stays green both on stock Vela 5.0 (cases XFAIL) and once the Vela fix is
262+
# in tree (cases XPASS). XfailIfNoCorstone320 is intentionally omitted --
263+
# stacking it with xfails= makes the per-id marks not fire.
264+
a16w8_sum_u85_xfails = {
265+
case: "Ethos-U85 int16 ReduceSum returns zero (vela#23)"
266+
for case in a16w8_sum_test_parameters
267+
}
268+
269+
270+
@common.parametrize(
271+
"test_data",
272+
a16w8_sum_test_parameters,
273+
xfails=a16w8_sum_u85_xfails,
274+
strict=False,
275+
)
276+
def test_sum_dim_intlist_a16w8_u85_INT(test_data):
277+
pipeline = EthosU85PipelineINT[input_t1](
278+
SumLastDim(),
279+
test_data(),
280+
aten_op,
281+
exir_ops=[],
282+
a16w8_quantization=True,
283+
symmetric_io_quantization=True,
284+
qtol=128,
285+
epsilon=2**-16,
286+
)
287+
pipeline.run()

backends/arm/test/targets.bzl

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ def define_arm_tests():
3030
"ops/test_slice.py",
3131
"ops/test_sigmoid.py",
3232
"ops/test_sub.py",
33+
"ops/test_sum.py",
3334
"ops/test_tanh.py",
3435
"ops/test_view.py",
3536
"ops/test_cos.py",

0 commit comments

Comments
 (0)