Skip to content

feat: add send_refresh_request_as_query_params toggle to OAuth authenticators#1041

Merged
sophiecuiy merged 6 commits into
mainfrom
devin/1780092927-oauth-refresh-request-query-params
Jun 10, 2026
Merged

feat: add send_refresh_request_as_query_params toggle to OAuth authenticators#1041
sophiecuiy merged 6 commits into
mainfrom
devin/1780092927-oauth-refresh-request-query-params

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented May 29, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds an opt-in boolean — send_refresh_request_as_query_params — that routes the standard OAuth refresh args (grant_type, refresh_token, client credentials when not in an Authorization header, scopes, plus any refresh_request_body extras) onto the URL query string of the refresh request instead of the form-encoded POST body.

This unblocks providers like Gong that document their refresh endpoint as POST with refresh args on the URL query string and an empty body. Today the CDK always sends those args in the body, which Gong's refresh endpoint rejects.

Resolves airbytehq/airbyte-internal-issues#16467.

What the toggle does

type: OAuthAuthenticator
token_refresh_endpoint: https://app.gong.io/oauth2/generate-customer-token
client_id:     "{{ config['credentials']['client_id'] }}"
client_secret: "{{ config['credentials']['client_secret'] }}"
refresh_token: "{{ config['credentials']['refresh_token'] }}"
grant_type:    refresh_token
refresh_request_headers:
  Authorization: "Basic {{ ... | base64encode }}"
send_refresh_request_as_query_params: true   # <-- new
send_refresh_request_as_query_params request body URL query string
false (default) standard refresh args (none)
true (empty) standard refresh args

When true, client_id / client_secret are still omitted from the query string if a refresh Authorization header is set, mirroring the pre-existing body-shaping logic.

Implementation

A shared helper _build_standard_refresh_args() (in AbstractOauth2Authenticator) is the single source of truth for "what goes into a refresh request" — it's used by both build_refresh_request_body() and build_refresh_request_query_params(), gated by a new abstract getter should_send_refresh_request_as_query_params() -> bool (default False):

def _build_standard_refresh_args(self) -> MutableMapping[str, Any]:
    args = {self.get_grant_type_name(): self.get_grant_type()}
    if not credentials_in_header(self):
        args[self.get_client_id_name()]     = self.get_client_id()
        args[self.get_client_secret_name()] = self.get_client_secret()
    args[self.get_refresh_token_name()] = self.get_refresh_token()
    if self.get_scopes(): args["scopes"] = self.get_scopes()
    for k, v in (self.get_refresh_request_body() or {}).items():
        if k not in args: args[k] = v
    return args

def build_refresh_request_body(self) -> Mapping[str, Any]:
    return {} if self.should_send_refresh_request_as_query_params() \
              else self._build_standard_refresh_args()

def build_refresh_request_query_params(self) -> Mapping[str, Any] | None:
    return self._build_standard_refresh_args() \
           if self.should_send_refresh_request_as_query_params() else None

Because the helper pulls live values from get_refresh_token() etc., rotation persistence is automatic — rotated refresh tokens are picked up on every refresh, no separate fix needed (the live test below exercises this end-to-end against Gong).

Files touched

  • airbyte_cdk/sources/streams/http/requests_native_auth/abstract_oauth.py — new _build_standard_refresh_args() helper, new abstract getter should_send_refresh_request_as_query_params(), body / query-param builders rewritten in terms of the helper.
  • airbyte_cdk/sources/streams/http/requests_native_auth/oauth.pyOauth2Authenticator and SingleUseRefreshTokenOauth2Authenticator accept send_refresh_request_as_query_params: bool = False and override the getter.
  • airbyte_cdk/sources/declarative/auth/oauth.pyDeclarativeOauth2Authenticator exposes send_refresh_request_as_query_params: bool = False (no InterpolatedMapping needed) and threads it into the concrete authenticator.
  • airbyte_cdk/sources/declarative/declarative_component_schema.yaml + regenerated Pydantic model — adds the boolean field on OAuthAuthenticator.
  • airbyte_cdk/sources/declarative/parsers/model_to_component_factory.py — passes the boolean through for both declarative authenticator code paths.
  • Unit tests rewritten to exercise the new API: behavior is asserted by inspecting the actual params= and data= kwargs on the underlying requests.request(...) call (not just the helper return values), including a rotation regression test on SingleUseRefreshTokenOauth2Authenticator.

Live-tenant verification (production Gong, this PR's CDK + the matching source-gong manifest in airbytehq/airbyte#78521)

# op input refresh_token (jti) rotated to (jti) result
1 check sFIkaCoMTE/4 (bootstrap) 5m9iFtGDqndS CONNECTION_STATUS: SUCCEEDED
2 check 5m9iFtGDqndS (rotated) f0rMIv0V0eRJ CONNECTION_STATUS: SUCCEEDED
3 discover f0rMIv0V0eRJ (no refresh — static catalog) CATALOG with 6 streams
4 read users (backdated expiry) f0rMIv0V0eRJ → forced refresh aHePkxlMUgq1 10 RECORD, 1 STATE, 0 errors

check #2 is the smoking gun for rotation persistence: Gong invalidates the previous refresh token on rotation, so without the live-getter behavior described above it would have 401'd.

Backward compatibility

  • Default is False, so all existing authenticators retain their previous behavior (params in body, no query string on the refresh URL).
  • The refresh_request_body field is unchanged and still flows into the body when the toggle is False. When the toggle is True it flows through _build_standard_refresh_args() into the query string.

Link to Devin session: https://app.devin.ai/sessions/f35f567405264ba4b512e57abf9be8cd

Add support for sending OAuth refresh token request parameters as URL
query string parameters instead of in the POST body. This is required
by OAuth providers like Gong that document their refresh endpoint with
grant_type and refresh_token on the URL query string.

Changes:
- Add get_refresh_request_params() to AbstractOauth2Authenticator
- Add build_refresh_request_query_params() that returns params or None
- Pass params= to requests.request() in _make_handled_request()
- Exclude query param keys from request body to prevent duplication
- Add refresh_request_params to Oauth2Authenticator and
  SingleUseRefreshTokenOauth2Authenticator
- Add refresh_request_query_params to declarative schema, Pydantic
  model, DeclarativeOauth2Authenticator, and model_to_component_factory
- Add unit tests covering query params behavior for both native and
  declarative authenticators

Co-Authored-By: bot_apk <apk@cognition.ai>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@github-actions

Copy link
Copy Markdown

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1780092927-oauth-refresh-request-query-params#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1780092927-oauth-refresh-request-query-params

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

  • /autofix - Fixes most formatting and linting issues
  • /poetry-lock - Updates poetry.lock file
  • /test - Runs connector tests with the updated CDK
  • /prerelease - Triggers a prerelease publish with default arguments
  • /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
  • /poe <command> - Runs any poe command in the CDK environment
📚 Show Repo Guidance

Helpful Resources

📝 Edit this welcome message.

@github-actions

github-actions Bot commented May 29, 2026

Copy link
Copy Markdown

PyTest Results (Fast)

4 100 tests  +6   4 088 ✅ +5   7m 16s ⏱️ - 1m 10s
    1 suites ±0      12 💤 +1 
    1 files   ±0       0 ❌ ±0 

Results for commit 947e145. ± Comparison against base commit a7b0929.

This pull request skips 1 test.
unit_tests.sources.declarative.test_concurrent_declarative_source ‑ test_read_with_concurrent_and_synchronous_streams

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented May 29, 2026

Copy link
Copy Markdown

PyTest Results (Full)

4 103 tests  +6   4 091 ✅ +6   11m 19s ⏱️ -18s
    1 suites ±0      12 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit 947e145. ± Comparison against base commit a7b0929.

♻️ This comment has been updated with latest results.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds support for sending OAuth refresh-token parameters on the URL query string (in addition to the form body), required by providers like Gong whose refresh endpoint rejects body-only parameters. The change is additive — when the new field is unset, behavior is unchanged.

Changes:

  • Adds get_refresh_request_params() / build_refresh_request_query_params() on AbstractOauth2Authenticator, wires params= into the refresh HTTP call, and removes any duplicated keys from the body.
  • Threads a new refresh_request_params kwarg through Oauth2Authenticator and SingleUseRefreshTokenOauth2Authenticator, and a new refresh_request_query_params field through the declarative authenticator, schema, generated model, and factory.
  • Adds unit tests for both stacks and updates mock_request helpers to accept the new params= kwarg.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.

Show a summary per file
File Description
airbyte_cdk/sources/streams/http/requests_native_auth/abstract_oauth.py Adds query-param plumbing in refresh request build/send path and dedupes body keys.
airbyte_cdk/sources/streams/http/requests_native_auth/oauth.py Adds refresh_request_params kwarg + getter on Oauth2Authenticator; forwarded by SingleUseRefreshTokenOauth2Authenticator.
airbyte_cdk/sources/declarative/auth/oauth.py Adds refresh_request_query_params dataclass field interpolated via InterpolatedMapping.
airbyte_cdk/sources/declarative/declarative_component_schema.yaml Exposes new optional refresh_request_query_params field on OAuthAuthenticator.
airbyte_cdk/sources/declarative/models/declarative_component_schema.py Regenerated Pydantic model with the new field.
airbyte_cdk/sources/declarative/parsers/model_to_component_factory.py Wires new field into both declarative OAuth authenticator constructions.
unit_tests/sources/streams/http/requests_native_auth/test_requests_native_auth.py Tests query-param emission, default-None behavior, and body de-duplication; updates mock_request signature.
unit_tests/sources/declarative/auth/test_oauth.py Equivalent declarative tests and mock_request signature update.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Previously `build_refresh_request_query_params()` returned the static mapping
provided at construction time. For `SingleUseRefreshTokenOauth2Authenticator`
this meant the rotated refresh_token was persisted to config and emitted as a
control message, but the next refresh request still carried the original
(now-invalidated) token in the URL query string, breaking long-running syncs.

`build_refresh_request_body()` already avoids this trap by reading from the
dynamic getters on every call. Apply the same pattern here for the refresh
token field specifically — other custom params (e.g. user-overridden
`grant_type`) are still passed through unchanged.

Regression test exercises two back-to-back refreshes against
`SingleUseRefreshTokenOauth2Authenticator` and asserts the URL on the second
request carries the rotated refresh token.

Co-Authored-By: bot_apk <apk@cognition.ai>
@sophiecuiy

Copy link
Copy Markdown
Contributor

One naming nit worth addressing before merge: the declarative manifest field is refresh_request_query_params, but the Python-level constructor kwarg on Oauth2Authenticator / SingleUseRefreshTokenOauth2Authenticator is refresh_request_params. The factory bridges them at model_to_component_factory.py:2875 by passing refresh_request_params=... even though the source field is refresh_request_query_params.

Since it's a new kwarg with a None default, renaming the Python kwarg to refresh_request_query_params is trivially non-breaking and removes a small but real source of confusion (there's no non-query refresh_request_params concept to disambiguate against).

…y_params

Per review feedback, rename the Python-level kwarg on the OAuth
authenticators to match the manifest field name. The kwarg was added
in this PR with a None default, so this is a non-breaking rename:

- Oauth2Authenticator and SingleUseRefreshTokenOauth2Authenticator
  constructor kwarg refresh_request_params -> refresh_request_query_params
- get_refresh_request_params() -> get_refresh_request_query_params()
- Instance attribute _refresh_request_params -> _refresh_request_query_params
- Factory and tests updated to match

Co-Authored-By: bot_apk <apk@cognition.ai>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

sophiecuiy — addressed the naming nit in bc78486.

Renamed throughout: kwarg refresh_request_paramsrefresh_request_query_params, instance attribute _refresh_request_params_refresh_request_query_params, and getter get_refresh_request_params()get_refresh_request_query_params(). The factory bridge at model_to_component_factory.py:2875 now reads refresh_request_query_params=oauth_config.get("refresh_request_query_params"), so the Python and manifest names match.

Non-breaking: the kwarg was added in this PR with a None default and has no external callers yet. Full OAuth test suite (75 tests across test_requests_native_auth.py + declarative test_oauth.py) plus the relevant factory tests pass locally.


Devin session

…h_request_as_query_params boolean

Per maintainer feedback (#1041),
collapse the user API to a single boolean toggle:

- DROP `refresh_request_query_params: Mapping[str, Any]` (kwarg + declarative
  field). The Mapping forced users to manually re-author standard OAuth refresh
  args with interpolation strings and made wire shape data-driven instead of
  policy-driven.
- ADD `send_refresh_request_as_query_params: bool = False`. When `True`, the
  standard refresh args (`grant_type`, `refresh_token`, client credentials
  unless an `Authorization` header is set, scopes, plus any
  `refresh_request_body` extras) are emitted on the URL query string and the
  request body is empty. When `False` (default), behavior is unchanged.

Implementation: shared `_build_standard_refresh_args()` helper used by both
`build_refresh_request_body()` and `build_refresh_request_query_params()`,
gated by `should_send_refresh_request_as_query_params()`.

Rotation persistence (the bug the previous design exposed) is now automatic
because the body builder already pulls from live getters
(`get_refresh_token()`, `get_grant_type()`, etc.) — no separate fix needed.

Co-Authored-By: bot_apk <apk@cognition.ai>
devin-ai-integration Bot added a commit to airbytehq/airbyte that referenced this pull request Jun 8, 2026
Replaces the explicit `refresh_request_query_params` mapping with a single
boolean toggle that opts the OAuth refresh into the URL-query-string shape
required by Gong's documented refresh endpoint. Behavior on the wire is
unchanged. Tracks the upstream CDK API redesign in
airbytehq/airbyte-python-cdk#1041.

Co-Authored-By: bot_apk <apk@cognition.ai>
Co-Authored-By: bot_apk <apk@cognition.ai>
@devin-ai-integration devin-ai-integration Bot changed the title feat: add refresh_request_query_params to OAuth authenticators feat: add send_refresh_request_as_query_params toggle to OAuth authenticators Jun 8, 2026
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

sophiecuiy — redesigned the API per your suggestion. Single boolean toggle, no Mapping. Pushed in 53cbd43 + 63b1650.

API change

# before
refresh_request_query_params:
  grant_type: refresh_token
  refresh_token: "{{ config['credentials']['refresh_token'] }}"

# after
send_refresh_request_as_query_params: true

When true, the standard refresh args (grant_type, refresh_token, client credentials when not in an Authorization header, scopes, plus any refresh_request_body extras) are emitted on the URL query string and the body is empty. When false (default), behavior is unchanged.

Implementation: shared helper, gated by the boolean

def _build_standard_refresh_args(self) -> MutableMapping[str, Any]:
    args = {self.get_grant_type_name(): self.get_grant_type()}
    if not credentials_in_header(self):
        args[self.get_client_id_name()]     = self.get_client_id()
        args[self.get_client_secret_name()] = self.get_client_secret()
    args[self.get_refresh_token_name()] = self.get_refresh_token()
    if self.get_scopes(): args["scopes"] = self.get_scopes()
    for k, v in (self.get_refresh_request_body() or {}).items():
        if k not in args: args[k] = v
    return args

def build_refresh_request_body(self) -> Mapping[str, Any]:
    return {} if self.should_send_refresh_request_as_query_params() \
              else self._build_standard_refresh_args()

def build_refresh_request_query_params(self) -> Mapping[str, Any] | None:
    return self._build_standard_refresh_args() \
           if self.should_send_refresh_request_as_query_params() else None

Because _build_standard_refresh_args() pulls live values from get_refresh_token(), the rotation-persistence behavior is now structural — no separate fix needed.

Live-tenant verification (production Gong, this PR's branch + the matching source-gong manifest in airbytehq/airbyte#78521)

# op input refresh_token (jti) rotated to (jti) result
1 check sFIkaCoMTE/4 (bootstrap) 5m9iFtGDqndS CONNECTION_STATUS: SUCCEEDED
2 check 5m9iFtGDqndS (rotated) f0rMIv0V0eRJ CONNECTION_STATUS: SUCCEEDED
3 discover f0rMIv0V0eRJ (no refresh — static catalog) CATALOG with 6 streams
4 read users (backdated expiry) f0rMIv0V0eRJ → forced refresh aHePkxlMUgq1 10 RECORD, 1 STATE, 0 errors

check #2 is the smoking gun for rotation persistence: Gong invalidates the previous refresh token on rotation, so a stale-token implementation would have 401'd.

PR title and description updated to reflect the new design.


Devin session

Reverts an incidental variable rename (payload -> args) so the diff
against main stays minimal in the body-building logic. The helper
itself is named _build_standard_refresh_args, but the local variable
that historically held the body dict stays named payload, matching
the convention in the rest of this file.

Co-Authored-By: bot_apk <apk@cognition.ai>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

sophiecuiy — fair nit on the payloadargs rename. It was incidental to the extraction (when I pulled the body-building logic into _build_standard_refresh_args(), I matched the local var to the helper name on the theory that the dict now feeds both body and query string, so "payload" felt body-specific). But it's diff noise in a function that was already well-understood, so reverted in 947e145 — local var inside the helper is back to payload, matching the pre-PR convention. Helper signature/docstring still call out "args" since that's accurate at the helper level, but the body of the helper reads identically to the original body-building code.

75 OAuth unit tests still pass; ruff format / lint / mypy clean.


Devin session

@sophiecuiy sophiecuiy merged commit fd95ecf into main Jun 10, 2026
31 of 32 checks passed
@sophiecuiy sophiecuiy deleted the devin/1780092927-oauth-refresh-request-query-params branch June 10, 2026 20:24
devin-ai-integration Bot added a commit to airbytehq/airbyte that referenced this pull request Jun 10, 2026
CDK PR airbytehq/airbyte-python-cdk#1041 has been merged and released as
v7.22.0, which publishes airbyte/source-declarative-manifest:7.22.0
(linux/amd64+linux/arm64 manifest list digest
sha256:02df50154030837bc6de45d9d5362d7f038a507040fb0049d57a9a64a6ac4906).

Reverts the temporary prerelease pin
(7.21.0.post6.dev27226613720@sha256:525e53...) used during pre-merge
verification and points at the stable released SDM that contains the
`send_refresh_request_as_query_params` field this manifest depends on.

Co-Authored-By: bot_apk <apk@cognition.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants