Skip to content

Commit ac5f720

Browse files
authored
Exclude Cassandra for Python 3.12 (#38050)
We need to exclude Cassandra separately, in order to make sure that cassandra drivers are not installed in the Python 3.12 image when preparing it from the main branch. Generally speaking the dependencies are cached from the main branch when image is built, which means that the resulting image will still have cassandra driver even if we remove it in final pyproject.toml. The way how to get rid of it is two fold: 1) add the exclusion to main 2) after it is merged, increase the EPOCH number in the Dockerfile.ci in the PR that adds Python 3.12 compatibility
1 parent 27807f4 commit ac5f720

3 files changed

Lines changed: 24 additions & 2 deletions

File tree

airflow/providers/apache/cassandra/provider.yaml

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,26 @@ dependencies:
4646
- apache-airflow>=2.6.0
4747
- cassandra-driver>=3.13.0
4848

49+
# Cassandra provider is not yet compatible with Python 3.12
50+
# The main issue is that python cassandra driver by default uses asyncore which has been deprecated since
51+
# Python 3.6 and removed in Python 3.12 (https://docs.python.org/3.11/library/asyncore.html)
52+
#
53+
# Currently the "wheel" package 3.29.0 distributed for manylinux platform is build without libev support (
54+
# which could be a viable asyncore replacement), and cassandra driver works in Python 3.12 if it is built
55+
# with libev support (using sdist, having gcc, libev4 and libev-dev installed). But it would be too much to
56+
# expect our users to build the driver from sources to use it with Python 3.12, so we are waiting for the
57+
# next release of cassandra-driver which will have libev support in the wheel package.
58+
# The issue is tracked here https://datastax-oss.atlassian.net/browse/PYTHON-1378
59+
#
60+
# Another option to get cassandra drive back is to have asyncio support in the driver instead of asyncore:
61+
# The issue is tracked here: https://datastax-oss.atlassian.net/browse/PYTHON-1375 and is scheduled
62+
# to be fixed in cassandra-driver 3.30.0.
63+
#
64+
# All Cassandra tests are automatically skipped if cassandra package is not present, so once you remove the
65+
# exclusion, they will start running for Python 3.12.
66+
#
67+
excluded-python-versions: ['3.12']
68+
4969
integrations:
5070
- integration-name: Apache Cassandra
5171
external-doc-url: https://cassandra.apache.org/

generated/provider_dependencies.json

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -90,7 +90,9 @@
9090
],
9191
"devel-deps": [],
9292
"cross-providers-deps": [],
93-
"excluded-python-versions": [],
93+
"excluded-python-versions": [
94+
"3.12"
95+
],
9496
"state": "ready"
9597
},
9698
"apache.drill": {

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -573,7 +573,7 @@ apache-beam = [ # source: airflow/providers/apache/beam/provider.yaml
573573
"pyarrow>=14.0.1;python_version != \"3.12\"",
574574
]
575575
apache-cassandra = [ # source: airflow/providers/apache/cassandra/provider.yaml
576-
"cassandra-driver>=3.13.0",
576+
"cassandra-driver>=3.13.0;python_version != \"3.12\"",
577577
]
578578
apache-drill = [ # source: airflow/providers/apache/drill/provider.yaml
579579
"apache-airflow[common_sql]",

0 commit comments

Comments
 (0)