[FLINK] CI refine#12430
Open
zhanglistar wants to merge 5 commits into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This pull request refines the Flink GitHub Actions CI job to run inside a different, pre-built Docker container image, aiming to speed up the pipeline by relying on an Ubuntu-based dependency image instead of installing build deps at runtime.
Changes:
- Switch the
flink-testjob container fromapache/gluten:centos-8-jdk17to an Ubuntu 20-based image and configure compiler/JAVA env vars. - Update ccache cache key naming from
centos8toubuntu20and increase ccache max size. - Remove CentOS-specific dependency installation/setup steps and add
JAVA_HOME/PATHexporting for Maven runs.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
61
to
+66
| runs-on: ubuntu-22.04 | ||
| container: apache/gluten:centos-8-jdk17 | ||
| container: listar1111/velox-deps:ubuntu20 | ||
| env: | ||
| CC: /usr/bin/gcc-11 | ||
| CXX: /usr/bin/g++-11 | ||
| JAVA_HOME: /usr/lib/jvm/java-11-openjdk-amd64 |
Comment on lines
61
to
+66
| runs-on: ubuntu-22.04 | ||
| container: apache/gluten:centos-8-jdk17 | ||
| container: listar1111/velox-deps:ubuntu20 | ||
| env: | ||
| CC: /usr/bin/gcc-11 | ||
| CXX: /usr/bin/g++-11 | ||
| JAVA_HOME: /usr/lib/jvm/java-11-openjdk-amd64 |
Comment on lines
84
to
86
| git clone -b gluten-0530 https://github.com/bigo-sg/velox4j.git | ||
| cd velox4j && git reset --hard 9811c337d5e37ab8d16d5bf0791b48836fcd2afd | ||
| cd velox4j && git reset --hard 5090dfdb713f53436368c2e177ce85580f31b0e8 | ||
| git apply $GITHUB_WORKSPACE/gluten-flink/patches/fix-velox4j.patch |
Comment on lines
79
to
+81
| export CCACHE_DIR=${CCACHE_DIR} | ||
| export CCACHE_MAXSIZE=2G | ||
| export CCACHE_MAXSIZE=5G | ||
| export simdjson_SOURCE=BUNDLED |
zhouyuan
reviewed
Jul 2, 2026
zhouyuan
left a comment
zhouyuan
left a comment
Member
There was a problem hiding this comment.
Thanks. may need to add a job to build cache in main branch so each PR can benefit from it
| export PATH=$JAVA_HOME/bin:$PATH | ||
| export CCACHE_DIR=${CCACHE_DIR} | ||
| export CCACHE_MAXSIZE=2G | ||
| export CCACHE_MAXSIZE=5G |
| key: ccache-flink-ubuntu20 | ||
| restore-keys: | | ||
| ccache-flink-centos8 | ||
| ccache-flink-ubuntu20 |
Member
There was a problem hiding this comment.
may need to store these changes in the main branch
https://github.com/apache/gluten/blob/main/.github/workflows/velox_backend_cache.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes are proposed in this pull request?
Refine CI based on docker to speed up.
How was this patch tested?
CI
Was this patch authored or co-authored using generative AI tooling?