feat(source-zendesk-support): Add user-configurable page_size option for ticket_comments stream#71752
Merged
Alfredo Garcia (agarctfi) merged 28 commits intoFeb 2, 2026
Conversation
Add a page_size configuration option to the connector spec that allows
users to control the number of records per page for API requests.
Lower values can help prevent timeouts on large datasets.
Changes:
- Add page_size to spec with default 100, max 1000
- Update all paginators to use config.get('page_size', 100)
- Add page_size_option to end_of_stream_paginator and after_url_paginator
Resolves: airbytehq/oncall#10879
Co-Authored-By: unknown <>
Contributor
Author
Original prompt from API User |
Contributor
Author
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Contributor
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. Helpful Resources
PR Slash CommandsAirbyte Maintainers (that's you!) can execute the following slash commands on your PR:
|
Contributor
|
…ze_option to paginators - Update airbyte-cdk from ^6 to ^7 in unit_tests/pyproject.toml to match metadata.yaml - Add page_size_option to after_url_paginator and end_of_stream_paginator - Add page_size: 100 to end_of_stream_paginator (was missing) - Revert user-configurable page_size interpolation (CursorPagination doesn't support it) Related to: airbytehq/oncall#10879 Co-Authored-By: unknown <>
devin-ai-integration
Bot
changed the title
Co-Authored-By: unknown <>
- Add page_size config option to connector spec (default: 100, max: 1000) - Update all paginators to use interpolated page_size from config - Add page_size_option to after_url_paginator and end_of_stream_paginator This allows users to reduce page_size to help prevent timeouts on large datasets. Related to: airbytehq/oncall#10879 Co-Authored-By: unknown <>
devin-ai-integration
Bot
changed the title
…to prevent parameter propagation
CDK 7.x has stricter schema validation that catches $parameters being
propagated to CursorPagination components. This fix adds $parameters: {}
to all CursorPagination definitions to prevent stream-level parameters
(name, path, data_path, primary_key) from being inherited by the
pagination_strategy, which would cause validation errors.
Co-Authored-By: Alfredo Garcia <alfredo.garcia@airbyte.io>
Co-Authored-By: alfredo.garcia@airbyte.io <freddy.garcia7.fg@gmail.com>
The CDK ^7 version bump caused manifest validation failures due to
stricter schema validation. Reverting to ^6 to fix the 173 test failures.
Also reverts the : {} additions which didn't fix the issue.
Co-Authored-By: alfredo.garcia@airbyte.io <freddy.garcia7.fg@gmail.com>
Co-Authored-By: alfredo.garcia@airbyte.io <freddy.garcia7.fg@gmail.com>
Contributor
|
/poe build
❌ Poe command |
Contributor
Author
|
The Root cause: The CDK's Approaches tried
Options to move forward:
Alfredo Garcia (@agarctfi) - Which approach would you like me to pursue? |
…ev21075186567 Co-Authored-By: alfredo.garcia@airbyte.io <freddy.garcia7.fg@gmail.com>
- Fix check_connection API change: use check() method instead of check_connection() - Add with_any_query_params() to mock server tests to handle per_page parameter - Update helpers.py, test_organizations.py, test_custom_roles.py, test_schedules.py, test_tickets.py, test_topics.py, test_user_identities.py, test_users.py, unit_test.py Co-Authored-By: alfredo.garcia@airbyte.io <freddy.garcia7.fg@gmail.com>
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
| HttpResponse(body=json.dumps(response), status_code=status_code), | ||
| ) | ||
| ok, _ = get_source(config=config, state=None).check_connection(logger=logging.Logger(name="airbyte"), config=config) | ||
| result = get_source(config=config, state=None).check(logger=logging.Logger(name="airbyte"), config=config) |
- Remove duplicate with_any_query_params() calls in pagination tests - For pagination tests, only use with_any_query_params() on first page request - Second page requests (with cursor/page param) don't need with_any_query_params() as the cursor/page param already makes them unique Co-Authored-By: alfredo.garcia@airbyte.io <freddy.garcia7.fg@gmail.com>
Co-Authored-By: alfredo.garcia@airbyte.io <freddy.garcia7.fg@gmail.com>
Co-Authored-By: alfredo.garcia@airbyte.io <freddy.garcia7.fg@gmail.com>
Contributor
|
/format-fix
|
| import pytest | ||
|
|
||
| # Patch HttpClient to never use caching | ||
| from unittest.mock import patch |
- Remove page_size_option from after_url_paginator (wasn't there before) - Remove page_size and page_size_option from end_of_stream_paginator (weren't there before) - Keep page_size interpolation only for paginators that already had page_size defined - This preserves the original API request behavior for streams using these paginators Co-Authored-By: alfredo.garcia@airbyte.io <freddy.garcia7.fg@gmail.com>
Contributor
|
/format-fix
|
| from airbyte_cdk.sources.streams.http.http_client import HttpClient, LimiterSession | ||
|
|
||
|
|
||
| _original_request_session = HttpClient._request_session |
Contributor
|
Deploy preview for airbyte-docs ready! ✅ Preview Built with commit e762a27. |
1 task
…s stream only Remove page_size interpolation from retriever, links_next_paginator, and after_url_paginator paginators, keeping it only in the ticket_comments stream's inline paginator. This ensures the page_size config option only affects the ticket_comments stream as intended. Co-Authored-By: alfredo.garcia@airbyte.io <freddy.garcia7.fg@gmail.com>
Contributor
|
/format-fix
|
- Update metadata.yaml base image from dev version to official 7.7.0 - Update unit_tests pyproject.toml to use airbyte-cdk 7.7.0 - Regenerate poetry.lock file Co-Authored-By: alfredo.garcia@airbyte.io <freddy.garcia7.fg@gmail.com>
devin-ai-integration
Bot
changed the title
Alfredo Garcia (agarctfi)
approved these changes
Feb 2, 2026
Alfredo Garcia (agarctfi)
deleted the
devin/1768415988-zendesk-configurable-page-size
branch
1 task
Kevin Gavino (kpgavino)
pushed a commit
to kpgavino/airbyte
that referenced
this pull request
Feb 3, 2026
…for ticket_comments stream (airbytehq#71752) Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Alfredo Garcia <alfredo.garcia@airbyte.io> Co-authored-by: Alfredo Garcia <alfredo.garcia@hallmark.edu> Co-authored-by: Octavia Squidington III <octavia-squidington-iii@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a user-configurable
page_sizeoption to the Zendesk Support connector for theticket_commentsstream, allowing users to reduce the number of records per API request. This helps prevent timeouts on large datasets.Related to: https://github.com/airbytehq/oncall/issues/10879
How
page_sizeconfig option to connector spec (default: 100, max: 1000) specifically for theticket_commentsstreamticket_commentsstream with interpolated page_size:{{ config.get('page_size', 100) }}7.7.0which includes interpolation support forCursorPagination.page_size(from CDK PR feat(declarative): Add interpolation support for CursorPagination page_size airbyte-python-cdk#878)Note: Only the
ticket_commentsstream is affected by this config option. Other streams retain their original pagination behavior.Updates since last revision
Scoped page_size to ticket_comments only:
retriever,links_next_paginator, andafter_url_paginatorticket_commentsstream with page_size and page_size_optionUpdated to official CDK release:
7.6.3.post8.dev21075186567to official7.7.0CDK v7 compatibility fixes:
check_connectionAPI change: CDK v7 usescheck()method returningAirbyteConnectionStatuswith_any_query_params()to handle query parameters flexiblyReview guide
ticket_commentsstream with page_size interpolationcheck()API migrationwith_any_query_params()for flexible query param matchingReviewer checklist
ticket_commentsstream is correctly configuredpage_sizeconfig option only affectsticket_commentsstream as intendedUser Impact
Users can configure
page_size(1-1000, default 100) to control records per page for theticket_commentsstream. Lower values may help prevent timeouts on large datasets.Can this PR be safely reverted and rolled back?
Requested by: Alfie Richards (@AlfGalf)
Link to Devin run: https://app.devin.ai/sessions/60834cce8286425da190005c9a48ba2b