Sourced from apache-beam's releases.

Beam 2.75.0 release

We are happy to present the new 2.75.0 release of Beam. This release includes both improvements and new functionality. See the download page for this release.

For more information on changes in 2.75.0, check out the detailed release notes.

Highlights

• Python SDK now supports memory profiling with Memray (#38853).
• (Python) Added Qdrant VectorDatabaseWriteConfig implementation (#38141).

I/Os

• Support for reading from Delta Lake added (Java) (#38551).
• ClickHouseIO: support writing DateTime64(precision[, 'timezone']) columns with sub-second precision (Java) (#38466).
• Upgraded IO Expansion Service to Java 17 (#38974).

New Features / Improvements

• Dataflow Runner v2 has been renamed to Dataflow Portable Runner. Please refer to Dataflow public documentation on when to enable Portable Runner.(#39000).
• (Java) Enabled state tag encoding v2 by default for new Dataflow Streaming Engine jobs. It can be disabled by passing --experiments=disable_streaming_engine_state_tag_encoding_v2 or --updateCompatibilityVersion=2.74.0 pipeline option. Note that the tag encoding version cannot change during a job update. Jobs using tag encoding v2 (enabled by default for new jobs on 2.75.0+) cannot be downgraded to Beam versions prior to 2.73.0, as only versions 2.73.0 and later support tag encoding v2. (#38705).
• (Python) Added instrumentation to support off-the-shelf profiling agents when launching Python SDK Harness (#38853).
• (Java) Added support to the FnApi Data stream protocol allowing runners to isolate bundles slowly processing input from other bundles. (#39001).
• (YAML) Switched js2py library to Quickjs (#38473).
• (YAML) Added HuggingFaceModelHandler for YAML usage (#38696).
• (YAML) Added WriteToMongoDB transform (#38376).
• (YAML) Added WriteToDatadog transform (#38362).
• (Java) Flink 2.1 and 2.2 support is added (#38947) (#38978); Flink 1.17 and 1.18 support is dropped.
• (Python) MqttIO is now supported in Python via cross-language (#21060).

Breaking Changes

• (Python) Typehints of dataclass fields are honored during type inferences. To restore the behavior of fallback-to-any, use pipeline option --exclude_infer_dataclass_field_type (#38797). However fixing forward is recommended.

Bugfixes

• Fixed GCS filesystem glob matching to correctly handle / in object names and support ** for recursive matching (Go) (#38059).
• Fixed BigQueryEnrichmentHandler batch mode dropping earlier requests when multiple requests share the same enrichment key (Python) (#38035).
• Fixed IcebergIO writing manifest column bounds padded with trailing 0x00 bytes, which broke equality predicate pushdown in some query engines (Java) (#38580).

Known Issues

• (Java) Projects using the Flink runner with Flink 2.1 or later alongside libraries requiring org.lz4:lz4-java (e.g., Kafka clients) may encounter a Gradle capability conflict, because Flink 2.1+ ships at.yawk.lz4:lz4-java which declares the same capability. To resolve, add a capabilitiesResolution rule to your build.gradle that selects at.yawk.lz4:lz4-java (#38947).

According to git shortlog, the following people contributed to the 2.75.0 release. Thank you to all contributors!

... (truncated)

Sourced from apache-beam's changelog.

[2.75.0] - 2026-07-08

Highlights

• Python SDK now supports memory profiling with Memray (#38853).
• (Python) Added Qdrant VectorDatabaseWriteConfig implementation (#38141).

I/Os

• Support for reading from Delta Lake added (Java) (#38551).
• ClickHouseIO: support writing DateTime64(precision[, 'timezone']) columns with sub-second precision (Java) (#38466).
• Upgraded IO Expansion Service to Java 17 (#38974).

New Features / Improvements

• Dataflow Runner v2 has been renamed to Dataflow Portable Runner. Please refer to Dataflow public documentation on when to enable Portable Runner.(#39000).
• (Java) Enabled state tag encoding v2 by default for new Dataflow Streaming Engine jobs. It can be disabled by passing --experiments=disable_streaming_engine_state_tag_encoding_v2 or --updateCompatibilityVersion=2.74.0 pipeline option. Note that the tag encoding version cannot change during a job update. Jobs using tag encoding v2 (enabled by default for new jobs on 2.75.0+) cannot be downgraded to Beam versions prior to 2.73.0, as only versions 2.73.0 and later support tag encoding v2. (#38705).
• (Python) Added instrumentation to support off-the-shelf profiling agents when launching Python SDK Harness (#38853).
• (Java) Added support to the FnApi Data stream protocol allowing runners to isolate bundles slowly processing input from other bundles. (#39001).
• (YAML) Switched js2py library to Quickjs (#38473).
• (YAML) Added HuggingFaceModelHandler for YAML usage (#38696).
• (YAML) Added WriteToMongoDB transform (#38376).
• (YAML) Added WriteToDatadog transform (#38362).
• (Java) Flink 2.1 and 2.2 support is added (#38947) (#38978); Flink 1.17 and 1.18 support is dropped.
• (Python) MqttIO is now supported in Python via cross-language (#21060).

Breaking Changes

• (Python) Typehints of dataclass fields are honored during type inferences. To restore the behavior of fallback-to-any, use pipeline option --exclude_infer_dataclass_field_type (#38797). However fixing forward is recommended.

Bugfixes

• Fixed GCS filesystem glob matching to correctly handle / in object names and support ** for recursive matching (Go) (#38059).
• Fixed BigQueryEnrichmentHandler batch mode dropping earlier requests when multiple requests share the same enrichment key (Python) (#38035).
• Fixed IcebergIO writing manifest column bounds padded with trailing 0x00 bytes, which broke equality predicate pushdown in some query engines (Java) (#38580).

Known Issues

• (Java) Projects using the Flink runner with Flink 2.1 or later alongside libraries requiring org.lz4:lz4-java (e.g., Kafka clients) may encounter a Gradle capability conflict, because Flink 2.1+ ships at.yawk.lz4:lz4-java which declares the same capability. To resolve, add a capabilitiesResolution rule to your build.gradle that selects at.yawk.lz4:lz4-java (#38947).

[2.74.0] - 2026-06-02

Highlights

• Spark 4 runner support for Java SDK (#38255).

I/Os

... (truncated)

• b52f04d Set version for 2.75.0 RC1
• 346604f Merge pull request #39153 from derrickaw/20260629_fixpostcommitYamlTest
• c65cda7 fix post commit yaml test (#39142)
• eea560f Write append tables can be asynchronous (#39120) (#39148)
• ccbfb33 Merge pull request #39115 from shunping/cp-39096
• cf9f578 Merge pull request #39117 from shunping/cp-39100
• a85f20c Revert "[Python] Optimize BigQuery copy jobs in file loads using multi-source...
• 9b70197 Trigger failed postcommit tests.
• 43be58f Update beam-master container
• d76047a Fix typo
• Additional commits viewable in compare view

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)