Commit 0ee31fc
authored
Bump iOS XCTest timeout for ExecuTorchLLMTests (#19354)
Summary:
The 13 XCTestCase methods in
`xplat/executorch/extension/llm/apple:ExecuTorchLLMTests`
(testLLaMA, testPhi4, testGemma, testLLaVA, testVoxtral and their
reset variants) regularly hit the 1800-second per-test ceiling
enforced by `fbobjc/Tools/xctest_runner` for the `long_running`
label. LLM inference on iOS-sim CPU (1B-class models,
128-768 token sequences, each test calls `generate()` twice)
routinely exceeds 30 minutes per test method, producing spurious
"Test timed out after 1800 seconds" flakes on the test-issues
dashboard for owner `ai_infra_mobile_platform`.
Per the runner formula
`TEST_CASE_TIMEOUT(60s) * label_multiplier * 3`:
| label | multiplier | per-XCTestCase budget |
|----------------|-----------:|----------------------:|
| long_running | x10 | 1800s |
| glacial (here) | x30 | 5400s |
Switching to `glacial` (the highest tier supported by the runner)
gives each test 90 minutes. Adding
`test_test_rule_timeout_ms = 14400000` sets the bundle-level
wall-clock budget to 4h, which is comfortable headroom for ~5
testcases at 90 min each plus xctest setup/teardown.
Note: this diff is unrelated to T269848646. T269848646 tracks a
separate cluster of 446 iOS-sim test-run *cancellations*
(`duration: 0.00`, "test execution was cancelled because the test
run was cancelled") that is owned by testinfra and is not
addressed here.
Reviewed By: shoumikhin
Differential Revision: D1041473131 parent 1414bc1 commit 0ee31fc
1 file changed
Lines changed: 11 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
19 | | - | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
20 | 30 | | |
21 | 31 | | |
22 | 32 | | |
| |||
0 commit comments