Skip to content

Commit ae4fdb5

Browse files
authored
Set test seed per-test (#19744)
### Summary In #19651, I added a global seed for pytest runs. This was intended to reduce random tolerance flakes, but didn't actually do so in practice. This is because the parallel test runners don't guarantee any ordering, so random state is unstable between runs. I've updated it to set the seed per-test. This should hopefully make the random state invariant of test execution order.
1 parent 29c3a23 commit ae4fdb5

2 files changed

Lines changed: 8 additions & 2 deletions

File tree

backends/cadence/aot/tests/test_replace_ops_passes.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1250,6 +1250,7 @@ def test_replace_conv1d_with_linear(self) -> None:
12501250
inputs,
12511251
"ReplaceTrivialConvWithLinear",
12521252
rtol=2e-5,
1253+
atol=5e-6,
12531254
)
12541255

12551256
# Assert that conv1d is trivially converted to linear
@@ -1294,6 +1295,7 @@ def test_replace_conv2d_with_linear(self) -> None:
12941295
inputs,
12951296
"ReplaceTrivialConvWithLinear",
12961297
rtol=2e-5,
1298+
atol=5e-6,
12971299
)
12981300

12991301
# Assert that conv2d is trivially converted to linear

conftest.py

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
import hashlib
12
import sys
23

34
import torch
@@ -13,5 +14,8 @@
1314
"backends/apple/**",
1415
]
1516

16-
# Seed the run
17-
torch.manual_seed(42)
17+
18+
def pytest_runtest_setup(item):
19+
# Set a stable seed for each test based on a hash of the test name.
20+
seed = int(hashlib.sha256(item.nodeid.encode()).hexdigest(), 16) % (2**32)
21+
torch.manual_seed(seed)

0 commit comments

Comments
 (0)