Skip to content

Commit a54f702

Browse files
authored
chore(airflow): bump Airflow example to 2.11.x, fix cached_property imports (#77)
Bumps airflow/requirements-dev.txt to the latest Airflow 2.x series: - apache-airflow ~=2.2.2 -> ~=2.11.0 - apache-airflow-providers-amazon ~=4.0 -> ~=9.22 - pytest ~=7.1.2 -> ~=9.0 Also fixes long-standing import errors in the custom EMR Serverless operator/hook: airflow.compat.functools was removed in Airflow 2.4, so those imports are replaced with stdlib functools.cached_property. The dead Python 3.7 fallback block in operators/emr.py is also removed (Airflow 2.9+ requires Python 3.8+, 2.11 requires 3.9+). Clears Dependabot alerts: #47 #48 #49 #52 #53 #55 #56 #57 #58 #59 #60 #61 #62 #63 #64 #65 #68 #75 #76 #77 #78 #79 #81 #86 (24 alerts total). Alerts #87 #88 #89 require Airflow 3.2.0 which is incompatible with custom operator APIs and out of scope here. Verified: - apache-airflow 2.11.0 + providers-amazon 9.26.0 + pytest 9.0.3 install cleanly under Python 3.11 - All custom hooks/operators/sensors import without error - test_dag_integrity passes - test_simple_submit has pre-existing issues using removed DAG.run() and schedule_interval APIs; those are integration tests against live EMR-S, out of scope for this PR
1 parent 0e296dc commit a54f702

3 files changed

Lines changed: 6 additions & 13 deletions

File tree

airflow/emr_serverless/hooks/emr.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,12 +17,12 @@
1717
# under the License.
1818
from __future__ import annotations
1919

20+
from functools import cached_property
2021
from time import sleep
2122
from typing import Any, Callable
2223

2324
from botocore.config import Config
2425

25-
from airflow.compat.functools import cached_property
2626
from airflow.exceptions import AirflowException
2727
from airflow.providers.amazon.aws.hooks.base_aws import AwsBaseHook
2828

airflow/emr_serverless/operators/emr.py

Lines changed: 1 addition & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@
1717
# under the License.
1818
from __future__ import annotations
1919

20-
import sys
20+
from functools import cached_property
2121
from typing import TYPE_CHECKING, Dict, Optional, Sequence
2222
from uuid import uuid4
2323

@@ -29,15 +29,8 @@
2929
if TYPE_CHECKING:
3030
from airflow.utils.context import Context
3131

32-
from airflow.compat.functools import cached_property
33-
3432
DEFAULT_CONN_ID = "aws_default"
3533

36-
if sys.version_info >= (3, 8):
37-
from functools import cached_property
38-
else:
39-
from cached_property import cached_property
40-
4134

4235
class EmrServerlessCreateApplicationOperator(BaseOperator):
4336
"""

airflow/requirements-dev.txt

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
apache-airflow~=2.2.2
2-
apache-airflow-providers-amazon~=4.0
3-
pytest~=7.1.2
4-
boto3>=1.23.9,~=1.23
1+
apache-airflow~=2.11.0
2+
apache-airflow-providers-amazon~=9.22
3+
pytest~=9.0
4+
boto3>=1.23.9,~=1.23

0 commit comments

Comments
 (0)