Skip to content

fix: improve HTTP error messages with source attribution and stream context #918

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Patrick Nilan (pnilan) wants to merge 1 commit into from

fix: improve user-facing HTTP error messages with source attribution …

cd9aa33
Select commit
Loading
Failed to load commit list.
Sign in for the full log view
Closed

fix: improve HTTP error messages with source attribution and stream context #918

fix: improve user-facing HTTP error messages with source attribution …
cd9aa33
Select commit
Loading
Failed to load commit list.
GitHub Actions / PyTest Results (Fast) failed Feb 24, 2026 in 0s

1 fail, 12 skipped, 3 101 pass in 6m 27s

3 114 tests   3 101 ✅  6m 27s ⏱️
    1 suites     12 💤
    1 files        1 ❌

Results for commit cd9aa33.

Annotations

Check warning on line 0 in unit_tests.sources.streams.http.test_availability_strategy

See this annotation in the file changed.

@github-actions github-actions / PyTest Results (Fast)

test_send_handles_retries_when_checking_availability (unit_tests.sources.streams.http.test_availability_strategy) failed 

build/test-results/pytest-results.xml [took 0s]
Raw output 
assert 'Service unavailable' in '{"type":"LOG","log":{"level":"INFO","message":"Backing off _send(...) for 1.0s (airbyte_cdk.sources.streams.http.exceptions.DefaultBackoffException: Stream \'mock_http_stream\': HTTP 429. Rate limit exceeded on source\'s API.)"}}\n{"type":"LOG","log":{"level":"INFO","message":"Caught retryable error \'Stream \'mock_http_stream\': HTTP 429. Rate limit exceeded on source\'s API.\' after 1 tries. Waiting 1 seconds then retrying..."}}\n{"type":"LOG","log":{"level":"INFO","message":"Backing off _send(...) for 2.0s (airbyte_cdk.sources.streams.http.exceptions.DefaultBackoffException: Stream \'mock_http_stream\': HTTP 503. Source\'s API is temporarily unavailable.)"}}\n{"type":"LOG","log":{"level":"INFO","message":"Caught retryable error \'Stream \'mock_http_stream\': HTTP 503. Source\'s API is temporarily unavailable.\' after 2 tries. Waiting 2 seconds then retrying..."}}\n'
 +  where '{"type":"LOG","log":{"level":"INFO","message":"Backing off _send(...) for 1.0s (airbyte_cdk.sources.streams.http.exceptions.DefaultBackoffException: Stream \'mock_http_stream\': HTTP 429. Rate limit exceeded on source\'s API.)"}}\n{"type":"LOG","log":{"level":"INFO","message":"Caught retryable error \'Stream \'mock_http_stream\': HTTP 429. Rate limit exceeded on source\'s API.\' after 1 tries. Waiting 1 seconds then retrying..."}}\n{"type":"LOG","log":{"level":"INFO","message":"Backing off _send(...) for 2.0s (airbyte_cdk.sources.streams.http.exceptions.DefaultBackoffException: Stream \'mock_http_stream\': HTTP 503. Source\'s API is temporarily unavailable.)"}}\n{"type":"LOG","log":{"level":"INFO","message":"Caught retryable error \'Stream \'mock_http_stream\': HTTP 503. Source\'s API is temporarily unavailable.\' after 2 tries. Waiting 2 seconds then retrying..."}}\n' = <_pytest.logging.LogCaptureFixture object at 0x7fa3c4f13290>.text
mocker = <pytest_mock.plugin.MockerFixture object at 0x7fa3c4f133e0>
caplog = <_pytest.logging.LogCaptureFixture object at 0x7fa3c4f13290>

    def test_send_handles_retries_when_checking_availability(mocker, caplog):
        mocker.patch("time.sleep", lambda x: None)
        http_stream = MockHttpStream()
    
        req_1 = requests.Response()
        req_1.status_code = 429
        req_2 = requests.Response()
        req_2.status_code = 503
        req_3 = requests.Response()
        req_3.status_code = 200
        mock_send = mocker.patch.object(requests.Session, "send", side_effect=[req_1, req_2, req_3])
    
        with caplog.at_level(logging.INFO):
            stream_is_available, _ = HttpAvailabilityStrategy().check_availability(
                stream=http_stream, logger=logger
            )
    
        assert stream_is_available
        assert mock_send.call_count == 3
        for message in ["Caught retryable error", "Service unavailable", "Service unavailable"]:
>           assert message in caplog.text
E           assert 'Service unavailable' in '{"type":"LOG","log":{"level":"INFO","message":"Backing off _send(...) for 1.0s (airbyte_cdk.sources.streams.http.exceptions.DefaultBackoffException: Stream \'mock_http_stream\': HTTP 429. Rate limit exceeded on source\'s API.)"}}\n{"type":"LOG","log":{"level":"INFO","message":"Caught retryable error \'Stream \'mock_http_stream\': HTTP 429. Rate limit exceeded on source\'s API.\' after 1 tries. Waiting 1 seconds then retrying..."}}\n{"type":"LOG","log":{"level":"INFO","message":"Backing off _send(...) for 2.0s (airbyte_cdk.sources.streams.http.exceptions.DefaultBackoffException: Stream \'mock_http_stream\': HTTP 503. Source\'s API is temporarily unavailable.)"}}\n{"type":"LOG","log":{"level":"INFO","message":"Caught retryable error \'Stream \'mock_http_stream\': HTTP 503. Source\'s API is temporarily unavailable.\' after 2 tries. Waiting 2 seconds then retrying..."}}\n'
E            +  where '{"type":"LOG","log":{"level":"INFO","message":"Backing off _send(...) for 1.0s (airbyte_cdk.sources.streams.http.exceptions.DefaultBackoffException: Stream \'mock_http_stream\': HTTP 429. Rate limit exceeded on source\'s API.)"}}\n{"type":"LOG","log":{"level":"INFO","message":"Caught retryable error \'Stream \'mock_http_stream\': HTTP 429. Rate limit exceeded on source\'s API.\' after 1 tries. Waiting 1 seconds then retrying..."}}\n{"type":"LOG","log":{"level":"INFO","message":"Backing off _send(...) for 2.0s (airbyte_cdk.sources.streams.http.exceptions.DefaultBackoffException: Stream \'mock_http_stream\': HTTP 503. Source\'s API is temporarily unavailable.)"}}\n{"type":"LOG","log":{"level":"INFO","message":"Caught retryable error \'Stream \'mock_http_stream\': HTTP 503. Source\'s API is temporarily unavailable.\' after 2 tries. Waiting 2 seconds then retrying..."}}\n' = <_pytest.logging.LogCaptureFixture object at 0x7fa3c4f13290>.text

unit_tests/sources/streams/http/test_availability_strategy.py:133: AssertionError
View more details on GitHub Actions