Skip to content

Commit 0f76352

Browse files
committed
perf(ab_test): ignore noisy network and MMDS metrics on flaky instances
Add m5n.metal (al2 host), m8g.metal-{24,48}xl, and m8i.metal-{48,96}xl to the IGNORED list for network latency and MMDS A/B tests respectively. These instance/metric combinations exhibit high variance (up to 67% swings in both directions) on unrelated commits, causing persistent false-positive pipeline failures. The noise on m8g correlates with the May 2 AL2023 rootfs update and affects only guest kernel linux-6.1 dimensions. MMDS noise on m8i is bidirectional within the same build. The tests still run and collect data on these instances; they just no longer block the pipeline. Signed-off-by: Riccardo Mancini <mancio@amazon.com>
1 parent 7df7152 commit 0f76352

1 file changed

Lines changed: 16 additions & 0 deletions

File tree

tools/ab_test.py

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -140,6 +140,22 @@ def format_with_reduced_unit(value, unit):
140140
}
141141
for instance in ["m8i.metal-48xl", "m8i.metal-96xl"]
142142
],
143+
# Network latencies on m5n.metal w/ al2 host
144+
{
145+
"instance": "m5n.metal",
146+
"performance_test": "test_network_latency",
147+
"host_kernel": "linux-5.10",
148+
},
149+
# Network latencies on m8g
150+
*[
151+
{"instance": instance, "performance_test": "test_network_latency"}
152+
for instance in ["m8g.metal-24xl", "m8g.metal-48xl"]
153+
],
154+
# MMDS metrics on m8i
155+
*[
156+
{"instance": instance, "performance_test": "test_mmds_performance"}
157+
for instance in ["m8i.metal-48xl", "m8i.metal-96xl"]
158+
],
143159
# block latencies if guest uses async request submission
144160
{"fio_engine": "libaio", "metric": "clat_read"},
145161
{"fio_engine": "libaio", "metric": "clat_write"},

0 commit comments

Comments
 (0)