fix(cdk): Handle TypeError in _literal_eval for nested set literals (AI-Triage PR)#936
Conversation
When ast.literal_eval() encounters a string like "{{'\''web'\''}, {'\''discover'\''}}"
that resembles nested Python set literals, it raises TypeError: unhashable
type: '\''set'\'' because sets cannot contain other sets. The existing except
clause only caught (ValueError, SyntaxError) but not TypeError.
This fix adds TypeError to the except clause so the string is returned
as-is, matching the existing behavior for other unparseable literals.
Resolves airbytehq/oncall#11543
Co-Authored-By: bot_apk <apk@cognition.ai>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. 💡 Show Tips and TricksTesting This CDK VersionYou can test this version of the CDK using the following: # Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1772779194-fix-literal-eval-typeerror#egg=airbyte-python-cdk[dev]' --help
# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1772779194-fix-literal-eval-typeerrorPR Slash CommandsAirbyte Maintainers can execute the following slash commands on your PR:
|
There was a problem hiding this comment.
Pull request overview
This PR prevents unhandled exceptions during Jinja interpolation by treating TypeError from ast.literal_eval() as a non-fatal parse failure and falling back to returning the raw string, matching existing behavior for other unparseable literals. This addresses crashes triggered by rendered strings that resemble nested Python set literals.
Changes:
- Broadened
_literal_eval()exception handling to also catchTypeErrorfromast.literal_eval(). - Added a unit test reproducing the nested-set-literal
TypeErrorscenario through the publicinterpolation.eval()flow.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
airbyte_cdk/sources/declarative/interpolation/jinja.py |
Catch TypeError in _literal_eval() to prevent sync crashes and preserve fallback-to-string behavior. |
unit_tests/sources/declarative/interpolation/test_jinja.py |
Add regression test ensuring nested set-literal-like strings don’t crash interpolation and are returned as-is. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-Authored-By: bot_apk <apk@cognition.ai>
Summary
Adds
TypeErrorto the exception handler inJinjaInterpolation._literal_eval(). Whenast.literal_eval()encounters a string resembling nested Python set literals (e.g.{{'web'}, {'discover'}}), it raisesTypeError: unhashable type: 'set'because sets cannot contain other sets. The existing handler only caughtValueErrorandSyntaxError, so thisTypeErrorpropagated unhandled and crashed syncs.This was reported for the
source-google-search-consoleconnector whereAddFieldstransformations render record values via Jinja templates. When a record field value happens to look like a nested set literal,_literal_evalfails on the post-render parse step.The fix returns the string as-is (matching the existing fallback behavior for other unparseable literals).
Resolves https://github.com/airbytehq/oncall/issues/11543:
Updates since last revision
# literal_eval is able to handle None→# result may be None; on error we return it unchanged) per Copilot review. The old comment impliedliteral_evalhandlesNonegracefully, but it actually raisesTypeErroronNoneinput — which is now correctly caught.Review & Testing Checklist for Human
TypeErrorbreadth is acceptable: This catches allTypeErrors fromast.literal_eval, not just the unhashable-set case. Confirm there's no scenario where aTypeErrorhere should propagate rather than fall back to returning the raw string. (Note:literal_eval(None)also raisesTypeError, meaning the old code had a latent crash path forNoneinputs too — catching it is the correct behavior.){{'web'}, {'discover'}}through the fullinterpolation.eval()flow, but verify this matches howAddFieldsinvokes it at runtime (viaInterpolatedString.eval→JinjaInterpolation.eval→_literal_eval).source-google-search-consolewith a site that returnssearchTypevalues like{'web'}, {'discover'}in thesearch_analyticsstreams, or manually invokeJinjaInterpolation.eval()with a config containing nested-set-like string values and confirm no crash.Notes
airbyte_cdk(core CDK library), but is trivially safe — it only broadens a catch clause that already catches two other exception types in the same fallback pattern.