fix: treat empty string cursor field as no cursor field#875
Conversation
When cursor_field is [''] (list with empty string) instead of [] (empty list), treat it as no cursor field. This can occur when the platform serializes 'no cursor configured' streams incorrectly. Fixes airbytehq/oncall#10700 Co-Authored-By: unknown <>
Original prompt from API User |
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. Testing This CDK VersionYou can test this version of the CDK using the following: # Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1767815288-fix-empty-cursor-field#egg=airbyte-python-cdk[dev]' --help
# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1767815288-fix-empty-cursor-fieldHelpful ResourcesPR Slash CommandsAirbyte Maintainers can execute the following slash commands on your PR:
|
PyTest Results (Full)3 824 tests 3 812 ✅ 10m 56s ⏱️ Results for commit a4a0bac. |
|
/prerelease
UPDATE from Aaron ("AJ") Steers (@aaronsteers): PyPI had an outage at the time. Resolved now and CDK is now published: |
a67a6cf
into
main
This update is part of the effort to ensure connectors include the fix for empty cursor field handling (airbytehq/airbyte-python-cdk#875). Related: airbytehq/oncall#11005 Parent: airbytehq/oncall#10700 Co-Authored-By: unknown <>
Summary
Fixes a bug where
cursor_field: [""](list with empty string) was incorrectly treated as a valid cursor field, causing invalid SQL generation in destinations like Redshift. The error manifests aszero-length delimited identifier at or near """"when the platform serializes "no cursor configured" streams as[""]instead of[].This PR adds defensive checks in two locations to treat empty string cursor fields as "no cursor field":
_get_catalog_defined_cursor_field()inmodel_to_component_factory.pyget_cursor_field_from_stream()inhelpers.pyWhen an empty string cursor is detected, the code now falls back to the default cursor field defined in the manifest.
Fixes airbytehq/oncall#10700
Review & Testing Checklist for Human
not configured_stream.cursor_fieldmust evaluateTruefor empty lists[]to prevent IndexError oncursor_field[0][""]as "no cursor" is the correct behavior (vs. raising an error)helpers.pychange - consider if this is acceptableSuggested test plan:
pytest unit_tests/sources/declarative/parsers/test_model_to_component_factory.py -k "cursor" -vNotes
_get_catalog_defined_cursor_fieldmethod