Skip to content

Commit f9ade32

Browse files
authored
test: fix flaky Windows CI failures in statistics and browser-pool tests (#1947)
Two unit tests flaked on a master push build ([run](https://github.com/apify/crawlee-python/actions/runs/26953000755/job/79522749183), `windows-latest`, Python 3.10). - **`test_crawler_intermediate_statistics`** — `poll_until_condition(lambda: crawler.statistics.active)` used the helper's default 5s ceiling. Under xdist load on a slow Windows runner, crawler startup didn't reach the active state in time, so the poll returned `False`. Pass a generous `timeout=30` (the happy path still returns in <1s, so there's no cost on green runs). - **`test_with_default_plugin_constructor`** — launching a real Firefox browser timed out even with `operation_timeout=60s` while competing with other xdist workers on the constrained Windows runner. The existing `@run_alone_on_mac` only isolated macOS, so Windows still ran it in the parallel pool. Upgraded to `@pytest.mark.run_alone` so the heavy browser launch gets the full machine on every platform; the 60s timeout is kept. Both flakes are Windows-resource-specific, so local Linux re-runs (10/10 and 5/5 green) confirm the fixes are valid but can't reproduce the original failure mode — CI re-verifies the Windows path.
1 parent f0c4550 commit f9ade32

2 files changed

Lines changed: 7 additions & 5 deletions

File tree

tests/unit/browsers/test_browser_pool.py

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,6 @@
99
from crawlee.browsers import BrowserPool, PlaywrightBrowserPlugin
1010
from crawlee.browsers._browser_controller import BrowserController
1111
from crawlee.browsers._types import CrawleePage
12-
from tests.unit.utils import run_alone_on_mac
1312

1413
if TYPE_CHECKING:
1514
from collections.abc import Mapping
@@ -103,9 +102,11 @@ async def test_new_page_with_each_plugin(server_url: URL) -> None:
103102
assert browser_pool.total_pages_count == 2
104103

105104

106-
@run_alone_on_mac
105+
@pytest.mark.run_alone
107106
async def test_with_default_plugin_constructor(server_url: URL) -> None:
108-
# Use a generous operation timeout so that Firefox has enough time to launch on slow Windows CI.
107+
# Launching a real Firefox browser is resource-heavy and flakes under xdist parallelism (it can time out
108+
# even with a generous operation timeout when several workers compete for the runner). Run it alone and
109+
# keep a generous operation timeout so that Firefox has enough time to launch on slow CI.
109110
async with BrowserPool.with_default_plugin(
110111
headless=True, browser_type='firefox', operation_timeout=timedelta(seconds=60)
111112
) as browser_pool:

tests/unit/crawlers/_basic/test_basic_crawler.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1993,9 +1993,10 @@ async def test_crawler_intermediate_statistics() -> None:
19931993
async def handler(_: BasicCrawlingContext) -> None:
19941994
await asyncio.sleep(check_time.total_seconds() * 5)
19951995

1996-
# Start crawler and wait until statistics are initialized.
1996+
# Start crawler and wait until statistics are initialized. Use a generous timeout so that crawler startup
1997+
# has enough time to reach the active state on slow CI runners under xdist load.
19971998
crawler_task = asyncio.create_task(crawler.run(['https://a.placeholder.com']))
1998-
assert await poll_until_condition(lambda: crawler.statistics.active)
1999+
assert await poll_until_condition(lambda: crawler.statistics.active, timeout=30)
19992000

20002001
# Wait some time and check that runtime is updated.
20012002
await asyncio.sleep(check_time.total_seconds())

0 commit comments

Comments
 (0)