fix(source-pylon): switch issues stream to POST /issues/search with updated_at cursor#76085
Conversation
…pdated_at cursor Switch the issues stream from GET /issues (filtered by created_at) to POST /issues/search (filtered by updated_at) so that incremental syncs capture both newly created and recently modified issues. Changes: - Change HTTP method from GET to POST with path issues/search - Replace query parameter injection with request_body_json containing nested Pylon filter structure (time_is_after / time_is_before on updated_at field) - Add dedicated issues_paginator using body_json injection for cursor and limit (POST endpoint requires body params, not query params) - Update cursor_field from created_at to updated_at - Add updated_at field to issues schema - Remove start_time_option/end_time_option (replaced by request_body_json) - Add unit tests validating manifest structure - Bump version 0.0.5 -> 0.0.6 - Update docs and changelog Co-Authored-By: bot_apk <apk@cognition.ai>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Co-Authored-By: bot_apk <apk@cognition.ai>
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. 💡 Show Tips and TricksPR Slash CommandsAirbyte Maintainers (that's you!) can execute the following slash commands on your PR:
📚 Show Repo GuidanceHelpful Resources
|
|
Co-Authored-By: bot_apk <apk@cognition.ai>
|
Deploy preview for airbyte-docs ready! ✅ Preview Built with commit 4c0e8c2. |
|
A human-authored PR targeting the same fix exists: airbytehq/airbyte#76083 by Henry Bogardus (@usbogie) — Both PRs switch the Linked issue: https://github.com/airbytehq/oncall/issues/11880 |
1 similar comment
|
A human-authored PR targeting the same fix exists: airbytehq/airbyte#76083 by Henry Bogardus (@usbogie) — Both PRs switch the Linked issue: https://github.com/airbytehq/oncall/issues/11880 |
1 participant
What
Fixes the
source-pylonissuesstream so incremental syncs capture modified issues, not just newly created ones.Resolves: https://github.com/airbytehq/oncall/issues/11880
Related: #76084
The previous implementation used
GET /issueswithstart_time/end_timequery parameters that filter bycreated_at. This meant incremental syncs missed all modifications to existing issues (status changes, reassignments, new messages, tag updates, CSAT responses, etc.).How
All changes are YAML-only (manifest-only connector, no custom Python).
GET /issues→POST /issues/searchwhich supports filtering onupdated_atcreated_at→updated_atstart_time_option/end_time_option(query param injection) and addedrequest_body_jsonwith a Jinja-templated nested Pylon filter structure usingtime_is_after/time_is_beforeoperators onupdated_atissues_paginator: POST endpoint requirescursor/limitin the request body (inject_into: body_json) rather than query parametersupdated_atto issues schemaReview guide
manifest.yaml— Core fix. Focus on therequest_body_jsonfilter template (lines 80-97) and the newissues_paginatordefinition (lines 510-527).unit_tests/test_manifest.py— 19 new manifest structure tests.metadata.yaml,pylon.md,sample_state.json,configured_catalog.json— Version bump, docs, and test fixture updates.Items for reviewer attention:
request_body_jsonfilter template: The filter value is a YAML block scalar (|) containing Jinja-templated JSON. Verify the CDK correctly evaluates Jinja and parses this as a JSON object in the request body (not a raw string). The"field": "updated_at"at the top level of the compound"and"filter may be extraneous — the Pylon Filter schema usesfieldon leaf filters, not compound ones. Confirm the API accepts or ignores this.stream_intervalavailability: Thestart_time_option/end_time_optionwere removed since the filter now references{{ stream_interval.start_time }}and{{ stream_interval.end_time }}directly. Verify the CDK still populatesstream_intervalfor theDatetimeBasedCursorwhen these options are absent.User Impact
issuesstream will now detect and sync modified issues, not just newly created ones.created_atcursor will be invalidated — users will need to reset theissuesstream. This is acceptable for an alpha (0.0.x) connector with a patch bump.issue_messages,issue_threads) are unaffected since thePOST /issues/searchresponse format matchesGET /issues.Can this PR be safely reverted and rolled back?
Link to Devin session: https://app.devin.ai/sessions/514d69b5ee1840da8803308ae8afe885