Commit 6b0d686
authored
chore(bigquery): fix timeout flakiness in job tests (#17826)
## The Problem
Two unit tests in `google-cloud-bigquery`
(`test_result_w_retry_wo_state` and `test_result_w_custom_retry`) were
failing intermittently with a `TimeoutError` or `RetryError`. This
flakiness occurs because the tests use a very short inner retry deadline
of 0.1 seconds. Under heavy load (such as during parallel testing or on
constrained continuous integration runners), this deadline can be
exceeded, causing the test to fail confusingly.
## The Solution
Increased the `deadline` parameter for the custom retry objects in both
tests from 0.1 seconds to 1.0 seconds. This provides enough buffer for
the test assertions to complete without timing out, aligning with other
similar tests in the suite.
## Notes to Reviewers
- This change only affects unit tests and does not alter production
code.
- This issue is blocking:
- #17642
- #17608
Here is a snippet of the failure:
```python
def test_result_w_retry_wo_state(global_time_lock):
from google.cloud.bigquery.retry import DEFAULT_GET_JOB_TIMEOUT
begun_job_resource = helpers._make_job_resource(
job_id=JOB_ID, project_id=PROJECT, location="EU", started=True
)
done_job_resource = helpers._make_job_resource(
job_id=JOB_ID,
project_id=PROJECT,
location="EU",
started=True,
ended=True,
)
conn = helpers.make_connection(
exceptions.NotFound("not normally retriable"),
begun_job_resource,
exceptions.NotFound("not normally retriable"),
done_job_resource,
)
client = helpers._make_client(project=PROJECT, connection=conn)
job = google.cloud.bigquery.job._AsyncJob(
google.cloud.bigquery.job._JobReference(JOB_ID, PROJECT, "EU"), client
)
custom_predicate = mock.Mock()
custom_predicate.return_value = True
custom_retry = google.api_core.retry.Retry(
predicate=custom_predicate,
initial=0.001,
maximum=0.001,
deadline=0.1,
)
> assert job.result(retry=custom_retry) is job
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
tests/unit/job/test_async_job_retry.py:115:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
google/cloud/bigquery/job/base.py:1047: in result
return super(_AsyncJob, self).result(timeout=timeout, retry=retry)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.nox/unit-3-12-test_type-unit/lib/python3.12/site-packages/google/api_core/future/polling.py:256: in result
self._blocking_poll(timeout=timeout, retry=retry, polling=polling)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = _AsyncJob<project=test-project, location=EU, id=test-job-id>
timeout = None
retry = <google.api_core.retry.retry_unary.Retry object at 0x7ff2beda32c0>
polling = <google.api_core.retry.retry_unary.Retry object at 0x7ff2beda01d0>
def _blocking_poll(self, timeout=_DEFAULT_VALUE, retry=None, polling=None):
"""Poll and wait for the Future to be resolved."""
if self._result_set:
return
polling = polling or self._polling
if timeout is not PollingFuture._DEFAULT_VALUE:
polling = polling.with_timeout(timeout)
try:
polling(self._done_or_raise)(retry=retry)
except exceptions.RetryError:
> raise concurrent.futures.TimeoutError(
f"Operation did not complete within the designated timeout of "
f"{polling.timeout} seconds."
)
E TimeoutError: Operation did not complete within the designated timeout of None seconds.
```1 parent ef89c2c commit 6b0d686
2 files changed
Lines changed: 5 additions & 8 deletions
File tree
- packages/google-cloud-bigquery/tests/unit/job
Lines changed: 2 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
15 | 15 | | |
16 | 16 | | |
17 | 17 | | |
| 18 | + | |
18 | 19 | | |
19 | 20 | | |
20 | 21 | | |
21 | | - | |
22 | | - | |
23 | 22 | | |
24 | 23 | | |
25 | 24 | | |
| |||
110 | 109 | | |
111 | 110 | | |
112 | 111 | | |
113 | | - | |
| 112 | + | |
114 | 113 | | |
115 | 114 | | |
116 | 115 | | |
| |||
Lines changed: 3 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| 14 | + | |
14 | 15 | | |
15 | 16 | | |
16 | | - | |
17 | 17 | | |
18 | | - | |
19 | 18 | | |
20 | 19 | | |
21 | | - | |
| 20 | + | |
22 | 21 | | |
23 | 22 | | |
24 | 23 | | |
| |||
27 | 26 | | |
28 | 27 | | |
29 | 28 | | |
30 | | - | |
31 | 29 | | |
32 | 30 | | |
33 | 31 | | |
| |||
105 | 103 | | |
106 | 104 | | |
107 | 105 | | |
108 | | - | |
| 106 | + | |
109 | 107 | | |
110 | 108 | | |
111 | 109 | | |
| |||
0 commit comments