Skip to content

ci: add swap during build, use tpchgen-cli #1443

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
timsaucer merged 12 commits into from
Mar 26, 2026
Merged

ci: add swap during build, use tpchgen-cli #1443

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
f106c2b
restrict number of rustc jobs during build stage
timsaucer Mar 25, 2026
14f6b0f
temporarily run release build on PR
timsaucer Mar 25, 2026
3ee76a8
Change optimization setting for substrait
timsaucer Mar 25, 2026
f36b1e6
Add swap during release build
timsaucer Mar 26, 2026
cdfff1e
Remove temporary checks to build in PR
timsaucer Mar 26, 2026
ea02fba
Try using tpchgen-cli for test files. commit answers
timsaucer Mar 26, 2026
0904924
taplo fmt
timsaucer Mar 26, 2026
f9a3a70
do not run rat on data files
timsaucer Mar 26, 2026
63e998e
ci needs ./ in path
timsaucer Mar 26, 2026
f933819
add no-project to uv run
timsaucer Mar 26, 2026
3f1d9a3
Temporary debug lines to figure out what is happening in CI
timsaucer Mar 26, 2026
7f9c17a
filter null value during aggregation instead now that https://github.…
timsaucer Mar 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -228,7 +228,10 @@ jobs:

- name: Build (release mode)
uses: PyO3/maturin-action@v1
if: inputs.build_mode == 'release'
# temporarily comment out to verify it works in the PR
# if: inputs.build_mode == 'release'
env:
CARGO_BUILD_JOBS: 2
Copy link
Copy Markdown
Contributor

@kevinjqliu kevinjqliu Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should help, we can even do CARGO_BUILD_JOBS=1 and also add it to the "debug" mode below on L241

with:
target: aarch64-unknown-linux-gnu
manylinux: "2_28"
Expand Down
4 changes: 4 additions & 0 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,10 @@ datafusion-python-util = { path = "crates/util" }
lto = true
codegen-units = 1

[profile.release.package.substrait]
opt-level = 1
codegen-units = 16

Copy link
Copy Markdown
Contributor

@kevinjqliu kevinjqliu Mar 25, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets see if scoping it to substrait will help here.

I think we might need to bite the bullet and do this:

[profile.release]
lto = "thin"
codegen-units = 4

or override the options using env var just for that one job.

but note that this affects the final artifact

# We cannot publish to crates.io with any patches in the below section. Developers
# must remove any entries in this section before creating a release candidate.
[patch.crates-io]
Loading