Skip to content

feat: add overlay to remove connector-specific models from generated SDK#185

Open
Aaron ("AJ") Steers (aaronsteers) wants to merge 5 commits into
mainfrom
devin/1782192010-remove-connector-models
Open

feat: add overlay to remove connector-specific models from generated SDK#185
Aaron ("AJ") Steers (aaronsteers) wants to merge 5 commits into
mainfrom
devin/1782192010-remove-connector-models

Conversation

@aaronsteers

@aaronsteers Aaron ("AJ") Steers (aaronsteers) commented Jun 23, 2026

Copy link
Copy Markdown
Member

Summary

Adds a Speakeasy overlay that replaces the SourceConfiguration, DestinationConfiguration, and OAuthCredentialsConfiguration oneOf unions with generic object schemas requiring sourceType/destinationType discriminator fields. After regeneration, this eliminates ~600 connector-specific Python model files and ~1,800 doc files (~126K lines / ~14MB).

Verified result: Generation succeeded and all CI checks pass. The diff is 2705 files changed, 273 insertions(+), 126,086 deletions(-). Zero source/destination connector model files remain; 96 core model files are retained.

What the overlay does (4 remove + 4 update actions):

  1. Removes the oneOf arrays from SourceConfiguration, DestinationConfiguration, OAuthCredentialsConfiguration
  2. Removes the enum array from OAuthActorNames
  3. Replaces each with a generic typed schema:
    • SourceConfiguration{type: object, required: [sourceType], additionalProperties: true}
    • DestinationConfiguration{type: object, required: [destinationType], additionalProperties: true}
    • OAuthCredentialsConfiguration{type: object, additionalProperties: true}
    • OAuthActorNames{type: string} (forward-compatible)

Usage change:

# Before: must import specific connector model
from airbyte_api.models import SourcePostgres
config = SourcePostgres(host="localhost", port=5432, sourceType="postgres", ...)

# After: pass a plain dict with required discriminator
config = {"sourceType": "postgres", "host": "localhost", "port": 5432, ...}

Breaking change: Typed connector model imports (SourcePostgres, DestinationBigquery, etc.) no longer exist. Should accompany a major version bump.

Requested by Aaron ("AJ") Steers (@aaronsteers).

Link to Devin session: https://app.devin.ai/sessions/1a9172f7c32f4aa2852da2423f44101a

Replace SourceConfiguration, DestinationConfiguration, and
OAuthCredentialsConfiguration oneOf unions with generic typed objects
that require sourceType/destinationType discriminator fields.

This removes ~600 individual connector config schemas from generation,
shrinking the SDK by ~600 Python model files and ~1,800 doc files (~14MB).
Users pass configuration as a plain dict; the API validates server-side.

Co-Authored-By: AJ Steers <aj@airbyte.io>
@devin-ai-integration

Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@octavia-bot

octavia-bot Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Generate SDK Job Info

Running Speakeasy SDK generation.

Job started... Check job output.
SDK generation failed. Check the job output for details.

@aaronsteers Aaron ("AJ") Steers (aaronsteers) marked this pull request as ready for review June 23, 2026 05:22
Copilot AI review requested due to automatic review settings June 23, 2026 05:22

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

Open in Devin Review

Comment thread overlays/python_speakeasy.yaml
Speakeasy validates that oneOf and enum must be arrays, not null.
Split the overlay into separate remove actions (to delete the oneOf/enum
arrays) followed by update actions (to set the replacement types).

Co-Authored-By: AJ Steers <aj@airbyte.io>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a Speakeasy overlay (and enables it in the Speakeasy workflow) intended to dramatically reduce SDK bloat by replacing connector-specific oneOf unions for connector configurations with generic object schemas that require discriminator fields (sourceType / destinationType).

Changes:

  • Add overlay actions to replace SourceConfiguration / DestinationConfiguration oneOf unions with generic object schemas requiring discriminator fields.
  • Replace OAuthActorNames from a closed enum to a plain string for forward compatibility.
  • Enable the overlay in .speakeasy/workflow.yaml so it is applied during generation.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
overlays/python_speakeasy.yaml Adds overlay actions to simplify connector config schemas and relax OAuthActorNames.
.speakeasy/workflow.yaml Enables the overlay during Speakeasy generation.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread overlays/python_speakeasy.yaml Outdated
Comment thread .speakeasy/workflow.yaml
@octavia-bot

octavia-bot Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Generate SDK Job Info

Running Speakeasy SDK generation.

Job started... Check job output.
SDK generation completed successfully.

@aaronsteers

Copy link
Copy Markdown
Member Author

Devin, update from main

Resolve merge conflicts:
- overlays/python_speakeasy.yaml: include both connector model removal
  actions and the RowFilteringOperationNot circular-ref fix from main
- src/airbyte_api/models/__init__.py: keep ours (generated, will be
  regenerated by /generate)

Co-Authored-By: AJ Steers <aj@airbyte.io>
@octavia-bot

octavia-bot Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Generate SDK Job Info

Running Speakeasy SDK generation.

Job started... Check job output.
SDK generation completed successfully.

@devin-ai-integration

Copy link
Copy Markdown
Contributor

Aaron ("AJ") Steers (@aaronsteers) Done — merged main, resolved conflicts (overlay now includes both the connector model removal actions and the RowFilteringOperationNot circular-ref fix from main), and re-triggered /generate to regenerate the SDK. Waiting for the generation workflow to push updated code.


Devin session

@github-actions

Copy link
Copy Markdown
Contributor

Generated Code Drift Detected

The committed code does not match what the generation pipeline produces.

To fix: Comment /generate on this PR to regenerate.

 src/airbyte_api/models/__init__.py | 25 +++++++++++++++----------
 1 file changed, 15 insertions(+), 10 deletions(-)

@octavia-bot

octavia-bot Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Pre-Release Job Info

Building pre-release 1.1.0rc1 from ref main.

Job started... Check job output.

@octavia-bot

octavia-bot Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Pre-Release Result: Published

Version: 1.1.0rc1
Ref: devin/1782192010-remove-connector-models
View run
Install: pip install airbyte-api==1.1.0rc1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants