feat: add overlay to remove connector-specific models from generated SDK#185
feat: add overlay to remove connector-specific models from generated SDK#185Aaron ("AJ") Steers (aaronsteers) wants to merge 5 commits into
Conversation
Replace SourceConfiguration, DestinationConfiguration, and OAuthCredentialsConfiguration oneOf unions with generic typed objects that require sourceType/destinationType discriminator fields. This removes ~600 individual connector config schemas from generation, shrinking the SDK by ~600 Python model files and ~1,800 doc files (~14MB). Users pass configuration as a plain dict; the API validates server-side. Co-Authored-By: AJ Steers <aj@airbyte.io>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
|
Speakeasy validates that oneOf and enum must be arrays, not null. Split the overlay into separate remove actions (to delete the oneOf/enum arrays) followed by update actions (to set the replacement types). Co-Authored-By: AJ Steers <aj@airbyte.io>
There was a problem hiding this comment.
Pull request overview
This PR introduces a Speakeasy overlay (and enables it in the Speakeasy workflow) intended to dramatically reduce SDK bloat by replacing connector-specific oneOf unions for connector configurations with generic object schemas that require discriminator fields (sourceType / destinationType).
Changes:
- Add overlay actions to replace
SourceConfiguration/DestinationConfigurationoneOfunions with genericobjectschemas requiring discriminator fields. - Replace
OAuthActorNamesfrom a closed enum to a plainstringfor forward compatibility. - Enable the overlay in
.speakeasy/workflow.yamlso it is applied during generation.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
overlays/python_speakeasy.yaml |
Adds overlay actions to simplify connector config schemas and relax OAuthActorNames. |
.speakeasy/workflow.yaml |
Enables the overlay during Speakeasy generation. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
Devin, update from main |
Resolve merge conflicts: - overlays/python_speakeasy.yaml: include both connector model removal actions and the RowFilteringOperationNot circular-ref fix from main - src/airbyte_api/models/__init__.py: keep ours (generated, will be regenerated by /generate) Co-Authored-By: AJ Steers <aj@airbyte.io>
|
|
Aaron ("AJ") Steers (@aaronsteers) Done — merged main, resolved conflicts (overlay now includes both the connector model removal actions and the |
|
Generated Code Drift Detected The committed code does not match what the generation pipeline produces. To fix: Comment |
|
|
Summary
Adds a Speakeasy overlay that replaces the
SourceConfiguration,DestinationConfiguration, andOAuthCredentialsConfigurationoneOfunions with genericobjectschemas requiringsourceType/destinationTypediscriminator fields. After regeneration, this eliminates ~600 connector-specific Python model files and ~1,800 doc files (~126K lines / ~14MB).Verified result: Generation succeeded and all CI checks pass. The diff is
2705 files changed, 273 insertions(+), 126,086 deletions(-). Zero source/destination connector model files remain; 96 core model files are retained.What the overlay does (4 remove + 4 update actions):
oneOfarrays fromSourceConfiguration,DestinationConfiguration,OAuthCredentialsConfigurationenumarray fromOAuthActorNamesSourceConfiguration→{type: object, required: [sourceType], additionalProperties: true}DestinationConfiguration→{type: object, required: [destinationType], additionalProperties: true}OAuthCredentialsConfiguration→{type: object, additionalProperties: true}OAuthActorNames→{type: string}(forward-compatible)Usage change:
Breaking change: Typed connector model imports (
SourcePostgres,DestinationBigquery, etc.) no longer exist. Should accompany a major version bump.Requested by Aaron ("AJ") Steers (@aaronsteers).
Link to Devin session: https://app.devin.ai/sessions/1a9172f7c32f4aa2852da2423f44101a