What happened?
When using Python 3.12, installing the Python SDK v2.66.0 and the GCP extras using PDM , the package google-apitools is not installed:
$ pdm add 'apache-beam[gcp]'
Adding packages to default dependencies: apache-beam
0:00:45 🔒 Lock successful.
Changes are written to pyproject.toml.
Synchronizing working set with resolved packages: 99 to add, 0 to update, 0 to remove
0:01:20 🎉 All complete! 2/2
$ pdm list
╭───────────────────────────────────────┬─────────────┬──────────╮
│ name │ version │ location │
├───────────────────────────────────────┼─────────────┼──────────┤
│ annotated-types │ 0.7.0 │ │
│ anyio │ 4.9.0 │ │
│ apache-beam │ 2.66.0 │ │
│ attrs │ 25.3.0 │ │
│ cachetools │ 5.5.2 │ │
│ certifi │ 2025.7.14 │ │
│ charset-normalizer │ 3.4.2 │ │
│ crcmod │ 1.7 │ │
│ dill │ 0.3.1.1 │ │
│ dnspython │ 2.7.0 │ │
│ docopt │ 0.6.2 │ │
│ docstring_parser │ 0.16 │ │
│ fastavro │ 1.11.1 │ │
│ fasteners │ 0.19 │ │
│ google-api-core │ 2.25.1 │ │
│ google-auth │ 2.40.3 │ │
│ google-auth-httplib2 │ 0.2.0 │ │
│ google-cloud-aiplatform │ 1.103.0 │ │
│ google-cloud-bigquery │ 3.34.0 │ │
│ google-cloud-bigquery-storage │ 2.32.0 │ │
│ google-cloud-bigtable │ 2.31.0 │ │
│ google-cloud-core │ 2.4.3 │ │
│ google-cloud-datastore │ 2.21.0 │ │
│ google-cloud-dlp │ 3.31.0 │ │
│ google-cloud-language │ 2.17.2 │ │
│ google-cloud-pubsub │ 2.31.0 │ │
│ google-cloud-pubsublite │ 1.12.0 │ │
│ google-cloud-recommendations-ai │ 0.10.18 │ │
│ google-cloud-resource-manager │ 1.14.2 │ │
│ google-cloud-spanner │ 3.55.0 │ │
│ google-cloud-storage │ 2.19.0 │ │
│ google-cloud-videointelligence │ 2.16.2 │ │
│ google-cloud-vision │ 3.10.2 │ │
│ google-crc32c │ 1.7.1 │ │
│ google-genai │ 1.25.0 │ │
│ google-resumable-media │ 2.7.2 │ │
│ googleapis-common-protos │ 1.70.0 │ │
│ grpc-google-iam-v1 │ 0.14.2 │ │
│ grpc-interceptor │ 0.15.4 │ │
│ grpcio │ 1.65.5 │ │
│ grpcio-status │ 1.65.5 │ │
│ h11 │ 0.16.0 │ │
│ hdfs │ 2.7.3 │ │
│ httpcore │ 1.0.9 │ │
│ httplib2 │ 0.22.0 │ │
│ httpx │ 0.28.1 │ │
│ idna │ 3.10 │ │
│ importlib_metadata │ 8.7.0 │ │
│ jaraco.classes │ 3.4.0 │ │
│ jaraco.context │ 6.0.1 │ │
│ jaraco.functools │ 4.2.1 │ │
│ jsonpickle │ 3.4.2 │ │
│ jsonschema │ 4.24.0 │ │
│ jsonschema-specifications │ 2025.4.1 │ │
│ keyring │ 25.6.0 │ │
│ keyrings.google-artifactregistry-auth │ 1.1.2 │ │
│ more-itertools │ 10.7.0 │ │
│ numpy │ 2.2.6 │ │
│ objsize │ 0.7.1 │ │
│ opentelemetry-api │ 1.35.0 │ │
│ opentelemetry-sdk │ 1.35.0 │ │
│ opentelemetry-semantic-conventions │ 0.56b0 │ │
│ orjson │ 3.10.18 │ │
│ overrides │ 7.7.0 │ │
│ packaging │ 25.0 │ │
│ pluggy │ 1.6.0 │ │
│ proto-plus │ 1.26.1 │ │
│ protobuf │ 5.29.5 │ │
│ pyarrow │ 18.1.0 │ │
│ pyarrow-hotfix │ 0.7 │ │
│ pyasn1 │ 0.6.1 │ │
│ pyasn1_modules │ 0.4.2 │ │
│ pydantic │ 2.11.7 │ │
│ pydantic_core │ 2.33.2 │ │
│ pydot │ 1.4.2 │ │
│ PyJWT │ 2.9.0 │ │
│ pymongo │ 4.13.2 │ │
│ pyparsing │ 3.2.3 │ │
│ python-dateutil │ 2.9.0.post0 │ │
│ pytz │ 2025.2 │ │
│ PyYAML │ 6.0.2 │ │
│ redis │ 5.3.0 │ │
│ referencing │ 0.36.2 │ │
│ regex │ 2024.11.6 │ │
│ requests │ 2.32.4 │ │
│ rpds-py │ 0.26.0 │ │
│ rsa │ 4.9.1 │ │
│ shapely │ 2.1.1 │ │
│ six │ 1.17.0 │ │
│ sniffio │ 1.3.1 │ │
│ sortedcontainers │ 2.4.0 │ │
│ sqlparse │ 0.5.3 │ │
│ tenacity │ 8.5.0 │ │
│ typing-inspection │ 0.4.1 │ │
│ typing_extensions │ 4.14.1 │ │
│ urllib3 │ 2.5.0 │ │
│ websockets │ 15.0.1 │ │
│ zipp │ 3.23.0 │ │
│ zstandard │ 0.23.0 │ │
╰───────────────────────────────────────┴─────────────┴──────────╯
This leads to errors such as:
Python 3.12.10 (main, Apr 9 2025, 03:49:38) [Clang 20.1.0 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from apache_beam.io.gcp import bigquery_tools
>>> bigquery_tools.parse_table_reference("project.dataset.table")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/private/tmp/test/.venv/lib/python3.12/site-packages/apache_beam/io/gcp/bigquery_tools.py", line 264, in parse_table_reference
if isinstance(table, TableReference):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: isinstance() arg 2 must be a type, a tuple of types, or a unionThis looks to be caused by these recent changes:
|
'google-apitools>=0.5.31,<0.5.32; python_version <= "3.12"', |
|
'google-apitools>=0.5.32,<0.5.33; python_version >= "3.13"', |
PDM is unable to handle the <= "3.12" marker, see here.
Is it possible to switch to non-inclusive markers?
Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
Issue Components
What happened?
When using Python 3.12, installing the Python SDK v2.66.0 and the GCP extras using PDM , the package
google-apitoolsis not installed:This leads to errors such as:
This looks to be caused by these recent changes:
beam/sdks/python/setup.py
Lines 453 to 454 in def8664
PDM is unable to handle the
<= "3.12"marker, see here.Is it possible to switch to non-inclusive markers?
Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
Issue Components