fix(build): Remove dagger dependencies and replace with uv scripts#702
Conversation
- Replace dagger containerized execution with uv scripts using inline dependencies - Use subprocess calls instead of async dagger operations - Maintain identical datamodel-codegen command line arguments - Keep same post-processing logic for pydantic imports and deprecated fields - Update shell script to use 'uv run' instead of dagger installation Co-Authored-By: AJ Steers <aj@airbyte.io>
Co-Authored-By: AJ Steers <aj@airbyte.io>
Original prompt from AJ Steers |
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
There was a problem hiding this comment.
Pull Request Overview
This PR removes the dagger-based containerized build system and replaces it with simpler uv scripts to address /poe build slash command failures in GitHub Actions. The change maintains identical functionality for generating pydantic models and JSON schemas from YAML files while eliminating complex containerization dependencies.
- Replaces dagger containerization with uv scripts using inline dependencies
- Converts async dagger operations to synchronous subprocess calls
- Updates shell script to use
uv runinstead of pip installing dagger-io
Comments suppressed due to low confidence (1)
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. Testing This CDK VersionYou can test this version of the CDK using the following: # Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1754527091-remove-dagger-dependencies#egg=airbyte-python-cdk[dev]' --help
# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1754527091-remove-dagger-dependenciesHelpful ResourcesPR Slash CommandsAirbyte Maintainers can execute the following slash commands on your PR:
|
|
/poe build
❌ Poe command |
- Restore base_model_with_deprecations.py file that was accidentally deleted - Update pyproject.toml assemble task to use uv run instead of deleted shell script - Regenerate models and JSON schema with working build process - Resolves ModuleNotFoundError that was causing pytest failures Co-Authored-By: AJ Steers <aj@airbyte.io>
- Recreate the missing base_model_with_deprecations.py file that was lost during formatting - This resolves the ModuleNotFoundError in CI that was preventing imports - File contains the BaseModelWithDeprecations class needed by generated models - All local tests now pass (3694 passed) and formatting checks pass Co-Authored-By: AJ Steers <aj@airbyte.io>
Co-Authored-By: AJ Steers <aj@airbyte.io>
There was a problem hiding this comment.
This file was just a shim and is deleted. Replaced in poe-tasks assemble reference with simply uv run bin/generate_component_manifest_files.py.
8b9171c
into
devin/1754519222-add-json-schema-generation
Remove dagger dependencies and replace with uv scripts
This PR targets the following PR:
Summary
Completely replaces the dagger-based containerized build system with uv scripts using inline dependencies. This change resolves the
/poe buildslash command failures in GitHub Actions caused by missing dagger dependencies, while maintaining identical functionality for generating pydantic models and JSON schemas from YAML files.Key Changes:
datamodel-code-generator==0.26.3,PyYAML>=6.0.1)subprocess.run()uv runinstead ofpip install dagger-io==0.13.3Review & Testing Checklist for Human
airbyte_cdk/sources/declarative/models/declarative_component_schema.pybefore/after this change to ensure byte-for-byte identical output/poe buildslash command now works in GitHub Actions without dagger dependency errorspoetry run poe buildlocally and confirm it completes successfully with same outputairbyte_cdk/sources/declarative/generated/declarative_component_schema.jsonis still generated correctlyRecommended Test Plan:
Diagram
%%{ init : { "theme" : "default" }}%% graph TB subgraph Legend L1[Major Edit]:::major-edit L2[Minor Edit]:::minor-edit L3[Context/No Edit]:::context end YamlSchema["airbyte_cdk/sources/declarative/<br/>declarative_component_schema.yaml"]:::context BuildScript["bin/generate_component_manifest_files.py"]:::major-edit ShellScript["bin/generate-component-manifest-dagger.sh"]:::major-edit PyprojectToml["pyproject.toml<br/>(poe assemble task)"]:::context GeneratedModels["airbyte_cdk/sources/declarative/models/<br/>declarative_component_schema.py"]:::context GeneratedJson["airbyte_cdk/sources/declarative/generated/<br/>declarative_component_schema.json"]:::context YamlSchema --> BuildScript BuildScript --> GeneratedModels BuildScript --> GeneratedJson ShellScript --> BuildScript PyprojectToml --> ShellScript classDef major-edit fill:#90EE90 classDef minor-edit fill:#87CEEB classDef context fill:#FFFFFFNotes
ModuleNotFoundError: No module named 'dagger'that was causing/poe buildslash command failuresSession Info: Requested by AJ Steers (Aaron ("AJ") Steers (@aaronsteers)) - Session: https://app.devin.ai/sessions/5fc8aa08db104607acbe970add5f1002