fix: re-raise DNS failures as ConnectionError in SSRF filter for universal retryability#1059
Conversation
…iversal retryability The SSRF protection in filtered_send caught socket.gaierror from _is_private_url and re-raised it as requests.exceptions.InvalidURL. While CDK streams handle InvalidURL as retryable (via PR #384), external SDKs (e.g. facebook-business) do not, causing transient DNS blips to produce unrecoverable sync failures. Change the exception type to requests.ConnectionError, which is universally retried by both CDK streams and external SDK error handlers. Co-Authored-By: bot_apk <apk@cognition.ai>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. 💡 Show Tips and TricksTesting This CDK VersionYou can test this version of the CDK using the following: # Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1782293727-fix-dns-gaierror-filtered-send#egg=airbyte-python-cdk[dev]' --help
# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1782293727-fix-dns-gaierror-filtered-sendPR Slash CommandsAirbyte Maintainers can execute the following slash commands on your PR:
|
PyTest Results (Fast)916 tests - 3 203 905 ✅ - 3 203 10m 57s ⏱️ + 2m 41s For more details on these failures, see this check. Results for commit 701f38b. ± Comparison against base commit 977f051. This pull request removes 3207 and adds 4 tests. Note that renamed tests count towards both.♻️ This comment has been updated with latest results. |
Update connector_builder tests that checked for 'Invalid URL' in stacktraces to expect 'DNS resolution failed for domainwithoutextension' instead. Also clean up empty except clause in test per bot feedback. Co-Authored-By: bot_apk <apk@cognition.ai>
PyTest Results (Full)4 126 tests +4 4 113 ✅ +3 17m 0s ⏱️ + 5m 24s For more details on these failures, see this check. Results for commit 701f38b. ± Comparison against base commit 977f051. |
Summary
The SSRF protection in
filtered_send(entrypoint.py) catchessocket.gaierrorfrom_is_private_url()and re-raises it asrequests.exceptions.InvalidURL. While PR #384 mappedInvalidURL→ RETRY in the CDK'sDEFAULT_ERROR_MAPPING, this only helps CDK HTTP streams that use the error handler. External SDKs (e.g.facebook-businessin source-facebook-marketing) bypass that mapping entirely, so a transient DNS blip ([Errno -3] Temporary failure in name resolution) causes an unrecoverable sync failure instead of a retry.Fix: Re-raise
socket.gaierrorasrequests.ConnectionErrorinstead ofInvalidURL.ConnectionErroris universally retried by both CDK streams and external SDK error handlers.The error message is also improved:
"DNS resolution failed for graph.facebook.com: ..."instead of the misleading"Invalid URL ParseResult(...): ...".Resolves https://github.com/airbytehq/oncall/issues/12976:
Prior art:
InvalidURL→ RETRY in error mapping, but only covers CDK HTTP streamsDNSResolutionErrorclass approachBreaking Change Evaluation
Not breaking. This change makes error handling more lenient (retryable instead of fatal) without changing any connector spec, schema, state format, or stream definition. No version bump artifacts needed — CDK uses dynamic versioning.
Test Coverage
Added parametrized tests in
unit_tests/test_entrypoint.pycovering thefiltered_sendcode path directly:test_filtered_send_raises_connection_error_on_dns_failure— parametrized with EAI_AGAIN, EAI_NONAME, EAI_FAIL errno valuestest_filtered_send_does_not_raise_invalid_url_on_dns_failure— verifies old behavior (InvalidURL) no longer occurstest_cloud_incorrect_ip_format_is_rejectedto expectConnectionErrorLink to Devin session: https://app.devin.ai/sessions/ad69979101834fc8a9d3b334b1d131f9