Skip to content

chore!(rust): remove DataFusion driver#4455

Open
lidavidm wants to merge 2 commits into
apache:mainfrom
lidavidm:remove-datafusion
Open

chore!(rust): remove DataFusion driver#4455
lidavidm wants to merge 2 commits into
apache:mainfrom
lidavidm:remove-datafusion

Conversation

@lidavidm

@lidavidm lidavidm commented Jul 2, 2026

Copy link
Copy Markdown
Member

Maintenance of this driver has moved to https://github.com/adbc-drivers/datafusion. Additionally, this makes it easier to deal with dependency upgrades (DataFusion tends to lag behind a bit).

@lidavidm lidavidm requested a review from amoeba July 2, 2026 03:20
@lidavidm

lidavidm commented Jul 2, 2026

Copy link
Copy Markdown
Member Author

CC @eitsupi

@lidavidm lidavidm marked this pull request as ready for review July 2, 2026 03:22
Comment thread .github/workflows/rust.yml Outdated
Comment on lines 219 to 233
@@ -244,7 +227,7 @@ jobs:
-p arrow-schema \
-p arrow-select

# If the lock file was updated, run the tests except for adbc_datafusion
# If the lock file was updated, run the tests
if ! git diff --quiet Cargo.lock; then
cargo test --all-targets --all-features --workspace --exclude adbc_datafusion
cargo test --all-targets --all-features --workspace
fi

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The purpose of this test is to test the case where Cargo.lock does not point to the latest arrow due to constraints imposed by Datafusion, so it can probably be removed.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, ideally we can still test with an older version of Arrow though

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, but you have that test already with minimal-versions. Got it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course, there's still the advantage of testing for the latest arrow immediately before Dependabot updating the lock file, so it might be worth keeping (although it could be annoying if it suddenly starts failing).

@lidavidm lidavidm force-pushed the remove-datafusion branch from c69965f to 7751791 Compare July 5, 2026 23:15

.. _DataFusion: https://datafusion.apache.org/

.. note:: The DataFusion driver is maintained by a third party. We're using it here since we don't maintain any Rust-based drivers directly.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
.. note:: The DataFusion driver is maintained by a third party. We're using it here since we don't maintain any Rust-based drivers directly.
.. note:: The DataFusion driver is maintained by a third party, the ADBC Driver Foundry (https://adbc-drivers.org). We're using it here since we don't maintain any Rust-based drivers directly.

@amoeba

amoeba commented Jul 7, 2026

Copy link
Copy Markdown
Member

I updated the PR body to link to the new repo instead of just saying "a third party".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants