Skip to content

fix ci_yq_donwload #4748

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
zjjlivein wants to merge 9 commits into from
Closed

fix ci_yq_donwload #4748

Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
a383529
fix ci_yq_donwload
zjjlivein Jul 7, 2026
4cec311
fix download
zjjlivein Jul 7, 2026
37d9be6
fix formes downlaod
zjjlivein Jul 7, 2026
c0467e9
fix
zjjlivein Jul 7, 2026
15efbd4
fix proxy
zjjlivein Jul 7, 2026
b8a1cb0
fix
zjjlivein Jul 7, 2026
47333d1
fix retry
zjjlivein Jul 7, 2026
dd6f0c1
fix
zjjlivein Jul 7, 2026
c3efa4a
fix
zjjlivein Jul 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
105 changes: 75 additions & 30 deletions .github/workflows/fleet-model-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -145,33 +145,49 @@ jobs:
docker exec -t ${{ env.container_name }} /bin/bash -ce '
rm -rf * .[^.]*
echo $PR_USER
source /root/proxy
mkdir -p /home/.cache/pip
pip cache dir
pip install --upgrade pip
git clone https://github.com/PaddlePaddle/PaddleFormers.git -b ${BRANCH}
wget -q --no-proxy https://paddle-qa.bj.bcebos.com/CodeSync/develop/PaddleFormers.tar --no-check-certificate
echo "Extracting PaddleFormers.tar"
rm -rf PaddleFormers
tar xf PaddleFormers.tar && rm -rf PaddleFormers.tar
cd PaddleFormers
git status
git config --global --add safe.directory /workspace/PaddleFormers
git config user.name "PaddleCI"
git config user.email "paddle_ci@example.com"
git config pull.rebase false
git pull --no-edit origin pull/${PR_ID}/head
wget -q --no-proxy https://paddle-qa.bj.bcebos.com/CodeSync/develop/PaddleFleet.tar
tar xf PaddleFleet.tar
source /root/proxy
git config --global user.name "PaddleCI"
git config --global user.email "paddle_ci@example.com"
base_commit=$(git rev-parse HEAD)
echo "base_commit(来自 BOS 快照)=$base_commit"
git submodule update --init --recursive --force
if [ -n "$PR_ID" ] && [ "$PR_ID" != "0" ]; then
git fetch origin pull/${PR_ID}/head
git checkout -b PR_${PR_ID} FETCH_HEAD

echo "Merging local base ${base_commit} into PR_${PR_ID}..."
git merge "$base_commit" --no-edit
git diff --numstat "$base_commit" -- | awk "{print \$NF}"
else
echo "Not in a pull_request event. Skipping PR-specific operations."
fi
git log --pretty=oneline -10
export UV_SKIP_WHEEL_FILENAME_CHECK=1
fleet_commit=$(python scripts/ci_utils/get_fleet_commit.py)
git clone https://github.com/PaddlePaddle/PaddleFleet.git
cd PaddleFleet
git config --global --add safe.directory /workspace/PaddleFormers
git config user.name "PaddleCI"
git config user.email "paddle_ci@example.com"
git config pull.rebase false
# git checkout release/0.3
git fetch origin
git reset --hard $fleet_commit
git reset --hard $fleet_commit
CUDA_VERSION=CUDA129 scripts/install_ops_wheel.sh
cd ..
# python -m pip install --pre paddlefleet-ops --index-url https://www.paddlepaddle.org.cn/packages/nightly/cu129/ --extra-index-url https://www.paddlepaddle.org.cn/packages/stable/cu129/ --extra-index-url https://pypi.tuna.tsinghua.edu.cn/simple
pip install -e ".[paddlefleet]" --extra-index-url=https://www.paddlepaddle.org.cn/packages/stable/cu129/ --extra-index-url=https://www.paddlepaddle.org.cn/packages/nightly/cu129/
wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O /usr/local/bin/yq
wget -q --no-proxy https://paddle-qa.bj.bcebos.com/paddleformers/yq_linux_amd64 -O /usr/local/bin/yq
chmod +x /usr/local/bin/yq
echo "paddle commit:"
python -c "import paddle; print(paddle.version.commit)"
Expand Down Expand Up @@ -304,27 +320,41 @@ jobs:
run: |
docker exec -t ${{ env.container_name }} /bin/bash -ce '
rm -rf * .[^.]*
source /root/proxy
mkdir -p /home/.cache/pip
pip cache dir
pip install --upgrade pip
git clone https://github.com/PaddlePaddle/PaddleFormers.git -b ${BRANCH}
wget -q --no-proxy https://paddle-qa.bj.bcebos.com/CodeSync/develop/PaddleFormers.tar --no-check-certificate
echo "Extracting PaddleFormers.tar"
tar xf PaddleFormers.tar && rm -rf PaddleFormers.tar
cd PaddleFormers
git status
git config --global --add safe.directory /workspace/PaddleFormers
git config user.name "PaddleCI"
git config user.email "paddle_ci@example.com"
git config pull.rebase false
git pull --no-edit origin pull/${PR_ID}/head
wget -q --no-proxy https://paddle-qa.bj.bcebos.com/CodeSync/develop/PaddleFleet.tar
tar xf PaddleFleet.tar
source /root/proxy
git config --global user.name "PaddleCI"
git config --global user.email "paddle_ci@example.com"
base_commit=$(git rev-parse HEAD)
echo "base_commit(来自 BOS 快照)=$base_commit"

@risemeup1111 risemeup1111 Jul 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 优先级：P1
这里会让 multi-card 安装阶段失败。这个 docker exec /bin/bash -ce 是独立 shell，前一个 job 里设置的 base_commit=$(git rev-parse HEAD) 不会带到这里；当前脚本只打印 $base_commit，随后 git merge "$base_commit" 会拿空字符串执行，报 “not something we can merge”。请像 single-card block 一样先在当前 PaddleFormers 目录内赋值。

Suggested change
echo "base_commit(来自 BOS 快照)=$base_commit"
base_commit=$(git rev-parse HEAD)
echo "base_commit(来自 BOS 快照)=$base_commit"

git submodule update --init --recursive --force
if [ -n "$PR_ID" ] && [ "$PR_ID" != "0" ]; then
git fetch origin pull/${PR_ID}/head
git checkout -b PR_${PR_ID} FETCH_HEAD

echo "Merging local base ${base_commit} into PR_${PR_ID}..."
git merge "$base_commit" --no-edit
git diff --numstat "$base_commit" -- | awk "{print \$NF}"
else
echo "Not in a pull_request event. Skipping PR-specific operations."
fi
git log --pretty=oneline -10
export UV_SKIP_WHEEL_FILENAME_CHECK=1
fleet_commit=$(python scripts/ci_utils/get_fleet_commit.py)
git clone https://github.com/PaddlePaddle/PaddleFleet.git
cd PaddleFleet
git config --global --add safe.directory /workspace/PaddleFormers
git config user.name "PaddleCI"
git config user.email "paddle_ci@example.com"
git config pull.rebase false
# git checkout release/0.3
git fetch origin
git reset --hard $fleet_commit
CUDA_VERSION=CUDA129 scripts/install_ops_wheel.sh
cd ..
Expand All @@ -336,7 +366,7 @@ jobs:
# wget -q --no-proxy --no-check-certificate https://paddle-qa.bj.bcebos.com/paddle-pipeline/Release-TagBuild-Training-Linux-Gpu-Cuda12.9-Cudnn9.9-Trt10.5-Mkl-Avx-Gcc11-SelfBuiltPypiUse/latest/paddlepaddle_gpu-0.0.0-cp310-cp310-linux_x86_64.whl
# pip uninstall paddlepaddle-gpu -y
# pip install paddlepaddle_gpu-0.0.0-cp310-cp310-linux_x86_64.whl --index-url=https://www.paddlepaddle.org.cn/packages/nightly/cu129/
wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O /usr/local/bin/yq
wget -q --no-proxy https://paddle-qa.bj.bcebos.com/paddleformers/yq_linux_amd64 -O /usr/local/bin/yq
chmod +x /usr/local/bin/yq
echo "paddle commit:"
python -c "import paddle; print(paddle.version.commit)"
Expand Down Expand Up @@ -597,27 +627,42 @@ jobs:
run: |
docker exec -t ${{ env.container_name }} /bin/bash -ce '
rm -rf * .[^.]*
source /root/proxy
mkdir -p /home/.cache/pip
pip cache dir
pip install --upgrade pip
git clone https://github.com/PaddlePaddle/PaddleFormers.git -b ${BRANCH}
wget -q --no-proxy https://paddle-qa.bj.bcebos.com/CodeSync/develop/PaddleFormers.tar --no-check-certificate
echo "Extracting PaddleFormers.tar"
rm -rf PaddleFormers
tar xf PaddleFormers.tar && rm -rf PaddleFormers.tar
cd PaddleFormers
git status
git config --global --add safe.directory /workspace/PaddleFormers
git config user.name "PaddleCI"
git config user.email "paddle_ci@example.com"
git config pull.rebase false
git pull --no-edit origin pull/${PR_ID}/head
wget -q --no-proxy https://paddle-qa.bj.bcebos.com/CodeSync/develop/PaddleFleet.tar
tar xf PaddleFleet.tar
source /root/proxy
git config --global user.name "PaddleCI"
git config --global user.email "paddle_ci@example.com"
base_commit=$(git rev-parse HEAD)
echo "base_commit(来自 BOS 快照)=$base_commit"

@risemeup1111 risemeup1111 Jul 7, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 优先级：P1
A100 安装脚本同样没有给 base_commit 赋值。每个 docker exec /bin/bash -ce 都是独立 shell，single-card block 中的赋值不会继承到这里；执行到后面的 git merge "$base_commit" 时会尝试 merge 空字符串并失败。请先在当前 PaddleFormers 目录内取 BOS 快照的 HEAD。

Suggested change
echo "base_commit(来自 BOS 快照)=$base_commit"
base_commit=$(git rev-parse HEAD)
echo "base_commit(来自 BOS 快照)=$base_commit"

git submodule update --init --recursive --force
if [ -n "$PR_ID" ] && [ "$PR_ID" != "0" ]; then
git fetch origin pull/${PR_ID}/head
git checkout -b PR_${PR_ID} FETCH_HEAD

echo "Merging local base ${base_commit} into PR_${PR_ID}..."
git merge "$base_commit" --no-edit
git diff --numstat "$base_commit" -- | awk "{print \$NF}"
else
echo "Not in a pull_request event. Skipping PR-specific operations."
fi
git log --pretty=oneline -10
export UV_SKIP_WHEEL_FILENAME_CHECK=1
fleet_commit=$(python scripts/ci_utils/get_fleet_commit.py)
git clone https://github.com/PaddlePaddle/PaddleFleet.git
cd PaddleFleet
git config --global --add safe.directory /workspace/PaddleFormers
git config user.name "PaddleCI"
git config user.email "paddle_ci@example.com"
git config pull.rebase false
# git checkout release/0.3
git fetch origin
git reset --hard $fleet_commit
CUDA_VERSION=CUDA129 scripts/install_ops_wheel.sh
cd ..
Expand All @@ -640,7 +685,7 @@ jobs:
pip install bce-python-sdk==0.8.74
pip install coverage==7.6.1
pip install librosa==0.11.0
wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O /usr/local/bin/yq
wget -q --no-proxy https://paddle-qa.bj.bcebos.com/paddleformers/yq_linux_amd64 -O /usr/local/bin/yq
chmod +x /usr/local/bin/yq
'

Expand Down
Loading