Patched DF 52.5.0 (revision b)#98
Draft
erratic-pattern wants to merge 10 commits into
Draft
Conversation
Includes fix for FixedSizeBinary LEFT JOIN bug - apache/arrow-rs#8981 Cherry-picked test and API updates from - apache#19355
…ons (apache#19078)" This reverts commit ada0923.
…e#19904) ## Which issue does this PR close? <!-- We generally require a GitHub issue to be filed for all bug fixes and enhancements and this helps us generate change logs for our releases. You can link an issue to this PR using the GitHub syntax. For example `Closes apache#123` indicates that this PR will close issue apache#123. --> - Part of apache#19798 ## Rationale for this change <!-- Why are you proposing this change? If this is already explained clearly in the issue then this section is not needed. Explaining clearly why changes are proposed helps reviewers understand your changes and offer better suggestions for fixes. --> This is an edge case, would prefer to fix upstream in arrow-rs instead of having handling code here, so just disable test for now. - arrow-rs issue: apache/arrow-rs#9227 ## What changes are included in this PR? <!-- There is no need to duplicate the description in the issue here but it is sometimes worth providing a summary of the individual changes in this PR. --> Disable edge-case array_union SLT ## Are these changes tested? <!-- We typically require tests for all PRs in order to: 1. Prevent the code from being accidentally broken by subsequent changes 2. Serve as another way to document the expected behavior of the code If tests are not included in your PR, please explain why (for example, are they covered by existing tests)? --> Test related change ## Are there any user-facing changes? <!-- If there are user-facing changes then we may require documentation to be updated before approving the PR. --> No. <!-- If there are any breaking changes to public APIs, please add the `api change` label. -->
Hot-fix for security_audit CI on upgrade-df-ver5250-a. Addresses 5 RUSTSEC advisories that landed between 2026-04-14 and 2026-04-27, after upstream DataFusion 52.5.0 was tagged: - RUSTSEC-2026-0098 — rustls-webpki: name constraints for URI names - RUSTSEC-2026-0099 — rustls-webpki: name constraints accepted for wildcard certs - RUSTSEC-2026-0104 — rustls-webpki: reachable panic in CRL parsing - RUSTSEC-2026-0112 — astral-tokio-tar: PAX header desync - RUSTSEC-2026-0113 — astral-tokio-tar: unpack_in chmod via symlink Bumps Cargo.lock only (no Cargo.toml dep range changes): - astral-tokio-tar 0.6.0 -> 0.6.1 - rustls-webpki 0.103.10 -> 0.103.13 When upstream backports these (or the next 52.x release ships them), this commit will auto-drop on rebase.
github-actions
Bot
added
documentation
erratic-pattern
commented
May 17, 2026
| } | ||
|
|
||
| #[test] | ||
| fn test_one_of_drops_internal_error_and_combines_type_errors() { |
Author
There was a problem hiding this comment.
Carry note (notes 1 + 2 consolidated): These test_one_of_* tests come from upstream apache#21032. Their assertions were written against the friendlier Display output from apache#20605, which landed after 52.5.0 and isn't picked here. I adapted them to 52.5's verbose form (TypeSignatureClass::Decimal, TypeSignatureClass::Native(LogicalType(Native(String), String)), etc.).
The verbose TypeSignatureClass:: prefix in the user-facing error message is the visible consequence of the same gap — when we adopt apache#20605 (or apache#21032 lands upstream and we drop this carry), both the assertions and the message wording clean up together.
erratic-pattern
commented
May 17, 2026
| }; | ||
| use datafusion_common::{HashMap, Result, ScalarValue, exec_err, not_impl_err}; | ||
| use datafusion_common::{ | ||
| DataFusionError, HashMap, Result, ScalarValue, exec_err, not_impl_err, |
Author
There was a problem hiding this comment.
Carry note (note 3): apache#21032 also added use datafusion_expr::expr::AggregateFunction; and use datafusion_expr::expr_fn::cast; here, as context lines — those imports are used elsewhere on upstream main but not in 52.5.0. Trimmed them to keep the build warning-free. They come back automatically when we drop this carry.
Collaborator
|
I think we should clean up the CI failures |
erratic-pattern
force-pushed
the
upgrade-df-ver5250-b
branch
from
65bb475 to
09efa4f
Compare
Cherry-pick of apache#21032. Adapted for 52.5.0: - Two unused imports in datafusion-expr's test module (`IntervalUnit`, `logical_int64`) compile-fail under -D warnings; dropped. - Two sql-integration tests and several sqllogictest expected-error patterns assumed the friendlier Display impl from apache#20605 was already on 52.5.0 (it isn't); updated those patterns to the pre-apache#20605 verbose `TypeSignatureClass::Native(LogicalType(...))` shape and to the post-apache#21032 `failed with: ...` (no quoted inner message) shape. - Also includes the author's own follow-up CI fix (was a separate commit upstream, folded here). Signed-off-by: yaommen <myanstu@163.com>
erratic-pattern
force-pushed
the
upgrade-df-ver5250-b
branch
from
09efa4f to
f478cdc
Compare
Patches RUSTSEC-2026-0145 (PAX header desynchronization, GHSA-3cv2-h65g-fgmm). Transitive dev-dep of `datafusion-cli` via `testcontainers` / `testcontainers-modules`; Cargo.lock-only update.
erratic-pattern
force-pushed
the
upgrade-df-ver5250-b
branch
from
86385bb to
3c068f4
Compare
Collaborator
|
😍 -- looking good to me |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds two commits on top of revision a to fix an InfluxQL signature-mismatch error regression discovered during the DF 52.5.0 upgrade.
influxdb_prothat pins the fork to this branchNew commits on top of revision a
improve OneOf signature diagnostics— cherry-pick of fix: avoid internal errors for OneOf signature mismatches apache/datafusion#21032, squashed to a single commit and adapted for 52.5.0. Replaces theinternal_err!path forTypeSignature::OneOfmismatches with aplan_err!path that carries actionable diagnostics, and restores function-specificSum not supported for Boolean/Avg not supported for Booleansemantic errors.The 52.5.0 adaptation (folded into this commit) covers fallout from the cherry-pick assuming the friendlier
Displayimpl from Improve formatting of datatypes apache/datafusion#20605, which is not on 52.5.0:datafusion-expr's test module (IntervalUnit,logical_int64) that fail-D warnings;sql_integrationtests and thetype_coercion::functionsunit tests whose assertions were written against Improve formatting of datatypes apache/datafusion#20605's output;array.slt,arrow_typeof.slt,datetime/timestamps.slt,encoding.slt,errors.slt,expr.slt,functions.slt,named_arguments.slt,scalar.slt,spark/math/hex.slt, andstring/string_literal.slt.Drop when fix: avoid internal errors for OneOf signature mismatches apache/datafusion#21032 lands upstream and reaches a 52.x point release.
chore(deps): bump astral-tokio-tar from 0.6.1 to 0.6.2— standalone, unrelated to fix: avoid internal errors for OneOf signature mismatches apache/datafusion#21032. Patches RUSTSEC-2026-0145 (PAX header desynchronization), a newly-published advisory that surfaced in this branch's CI. Transitive dev-dependency; Cargo.lock-only.Carry-forward from revision a
Unchanged from #97 — the IOx-specific patch series plus the rev-a dependency bump (
chore(deps): bump rustls-webpki and astral-tokio-tar for cargo audit).Verification
Fork CI green (64/64).