Skip to content

Custom components in published manifest-only connectors are rejected in Cloud since 7.23.7 #1082

Description

@devin-ai-integration

Summary

CDK 7.23.7 broke every published manifest-only connector that bundles a components.py, in any environment that does not set AIRBYTE_ENABLE_UNSAFE_CODE=true — including Airbyte Cloud. spec still works, but check, discover, and read all fail immediately with:

airbyte_cdk.sources.declarative.parsers.custom_code_compiler.AirbyteCustomCodeNotPermittedError:
Custom connector code is not permitted in this environment.

Root cause

#1076 added an unconditional gate in ModelToComponentFactory.create_custom_component:

if not custom_code_execution_permitted():
    raise AirbyteCustomCodeNotPermittedError

Before that change, the AIRBYTE_ENABLE_UNSAFE_CODE gate applied only to components.py code injected through the config (__injected_components_py), i.e. untrusted, user-supplied code from Connector Builder / a remote manifest. Code baked into a published connector image was trusted.

The new gate does not distinguish provenance, so a manifest-only connector shipping its own manifest.yaml + components.py inside its own image is now rejected too.

Impact

Any manifest-only connector whose manifest references a Custom* component. In the monorepo, 56 connectors bundle a components.py; 4 are already published on base image source-declarative-manifest:7.23.7 and are broken in Cloud today:

  • source-chargebee (0.10.41)
  • source-bing-ads
  • source-greenhouse
  • source-apify-dataset

27 more are on 7.23.6 and will break on their next automated dependency bump.

Example failure (source-chargebee 0.10.41, whose manifest uses CustomTransformation -> source_declarative_manifest.components.CustomFieldTransformation on ~27 streams):

File ".../parsers/model_to_component_factory.py", line 2108, in create_default_stream
    self._create_component_from_model(model=transformation_model, config=config)
File ".../parsers/model_to_component_factory.py", line 1822, in create_custom_component
    raise AirbyteCustomCodeNotPermittedError

Downstream destinations then fail with Input was fully read, but some streams did not receive a terminal stream status message, which obscures the real cause.

Proposed fix

Restore the provenance distinction: apply the gate only when the manifest itself came from the config (__injected_declarative_manifest present), which is the untrusted case that PR #1076 set out to protect. Image-bundled manifests keep working without AIRBYTE_ENABLE_UNSAFE_CODE.

Affected connectors additionally need a rebuild onto the fixed CDK release once it ships.


Filed by Devin at the request of Syed Khadeer (syed.khadeer@airbyte.io) while triaging a broken production Chargebee -> BigQuery connection.

Devin session

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions