Skip to content

fix: CI regression — Run CI doubled from ~17 min to ~35 min after PR #742 #903

@planetf1

Description

@planetf1

Observed behaviour

The Run CI workflow duration jumped from ~17–20 minutes to ~35–40 minutes, first seen on 2026-03-31. The increase is consistent across all three Python matrix jobs.

Root cause

PR #742 (#742) accidentally removed @pytest.mark.block_network from test_openai_compat in test/formatters/granite/test_intrinsics_formatters.py during the marker taxonomy audit.

That test deliberately connects to an unreachable URL (http://example.com:98765/...) and expects an openai.APIConnectionError. The comment in the test still reads "Note that network access is blocked for this test case."

With the marker in place, pytest-recording intercepts the socket call immediately. Without it, the OpenAI client makes a real TCP connection and waits for the OS-level connect timeout — approximately 31 seconds per call. The test makes 2 calls per parameterised case across 17 cases:

17 cases × 2 calls × ~31 s = ~18 minutes extra per matrix job

Evidence

Run Tests Passed Skipped
2026-03-30 (pre-regression) 711 s 721 151
2026-04-07 (post-regression) 1666 s 873 125

Pre-regression each test_openai_compat case completed in ~2.8 s; post-regression each takes ~63 s.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No fields configured for Bug.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions