Summary
CDK 7.23.7 broke every published manifest-only connector that bundles a components.py, in any environment that does not set AIRBYTE_ENABLE_UNSAFE_CODE=true — including Airbyte Cloud. spec still works, but check, discover, and read all fail immediately with:
airbyte_cdk.sources.declarative.parsers.custom_code_compiler.AirbyteCustomCodeNotPermittedError:
Custom connector code is not permitted in this environment.
Root cause
#1076 added an unconditional gate in ModelToComponentFactory.create_custom_component:
if not custom_code_execution_permitted():
raise AirbyteCustomCodeNotPermittedError
Before that change, the AIRBYTE_ENABLE_UNSAFE_CODE gate applied only to components.py code injected through the config (__injected_components_py), i.e. untrusted, user-supplied code from Connector Builder / a remote manifest. Code baked into a published connector image was trusted.
The new gate does not distinguish provenance, so a manifest-only connector shipping its own manifest.yaml + components.py inside its own image is now rejected too.
Impact
Any manifest-only connector whose manifest references a Custom* component. In the monorepo, 56 connectors bundle a components.py; 4 are already published on base image source-declarative-manifest:7.23.7 and are broken in Cloud today:
source-chargebee (0.10.41)
source-bing-ads
source-greenhouse
source-apify-dataset
27 more are on 7.23.6 and will break on their next automated dependency bump.
Example failure (source-chargebee 0.10.41, whose manifest uses CustomTransformation -> source_declarative_manifest.components.CustomFieldTransformation on ~27 streams):
File ".../parsers/model_to_component_factory.py", line 2108, in create_default_stream
self._create_component_from_model(model=transformation_model, config=config)
File ".../parsers/model_to_component_factory.py", line 1822, in create_custom_component
raise AirbyteCustomCodeNotPermittedError
Downstream destinations then fail with Input was fully read, but some streams did not receive a terminal stream status message, which obscures the real cause.
Proposed fix
Restore the provenance distinction: apply the gate only when the manifest itself came from the config (__injected_declarative_manifest present), which is the untrusted case that PR #1076 set out to protect. Image-bundled manifests keep working without AIRBYTE_ENABLE_UNSAFE_CODE.
Affected connectors additionally need a rebuild onto the fixed CDK release once it ships.
Filed by Devin at the request of Syed Khadeer (syed.khadeer@airbyte.io) while triaging a broken production Chargebee -> BigQuery connection.
Devin session
Summary
CDK 7.23.7 broke every published manifest-only connector that bundles a
components.py, in any environment that does not setAIRBYTE_ENABLE_UNSAFE_CODE=true— including Airbyte Cloud.specstill works, butcheck,discover, andreadall fail immediately with:Root cause
#1076 added an unconditional gate in
ModelToComponentFactory.create_custom_component:Before that change, the
AIRBYTE_ENABLE_UNSAFE_CODEgate applied only tocomponents.pycode injected through the config (__injected_components_py), i.e. untrusted, user-supplied code from Connector Builder / a remote manifest. Code baked into a published connector image was trusted.The new gate does not distinguish provenance, so a manifest-only connector shipping its own
manifest.yaml+components.pyinside its own image is now rejected too.Impact
Any manifest-only connector whose manifest references a
Custom*component. In the monorepo, 56 connectors bundle acomponents.py; 4 are already published on base imagesource-declarative-manifest:7.23.7and are broken in Cloud today:source-chargebee(0.10.41)source-bing-adssource-greenhousesource-apify-dataset27 more are on 7.23.6 and will break on their next automated dependency bump.
Example failure (
source-chargebee0.10.41, whose manifest usesCustomTransformation->source_declarative_manifest.components.CustomFieldTransformationon ~27 streams):Downstream destinations then fail with
Input was fully read, but some streams did not receive a terminal stream status message, which obscures the real cause.Proposed fix
Restore the provenance distinction: apply the gate only when the manifest itself came from the config (
__injected_declarative_manifestpresent), which is the untrusted case that PR #1076 set out to protect. Image-bundled manifests keep working withoutAIRBYTE_ENABLE_UNSAFE_CODE.Affected connectors additionally need a rebuild onto the fixed CDK release once it ships.
Filed by Devin at the request of Syed Khadeer (syed.khadeer@airbyte.io) while triaging a broken production Chargebee -> BigQuery connection.
Devin session