Skip to content

fix:Unstable ci #3201

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
chejinge merged 14 commits into from
Dec 3, 2025
Merged

fix:Unstable ci #3201

Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
0752ca1
fix:unstable_CI
brother-jin Dec 2, 2025
2b1aae9
fix:unstable_CI
brother-jin Dec 2, 2025
5ef35c2
fix:unstable_CI
brother-jin Dec 2, 2025
3b43023
fix:unstable_CI
brother-jin Dec 2, 2025
1cb1bfe
fix:unstable_CI
brother-jin Dec 2, 2025
4de3e65
fix
brother-jin Dec 2, 2025
9d09def
fix
brother-jin Dec 2, 2025
ad1af35
fix
brother-jin Dec 2, 2025
7a44a34
fix
brother-jin Dec 2, 2025
2e68271
fix
brother-jin Dec 2, 2025
15e0bcf
fix
brother-jin Dec 2, 2025
573d530
fix
brother-jin Dec 3, 2025
46c2530
fix
brother-jin Dec 3, 2025
d9591c5
fix
brother-jin Dec 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 91 additions & 2 deletions .github/workflows/pika.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,14 @@ jobs:
runs-on: ubuntu-latest

steps:
- name: Free Disk Space
run: |
sudo rm -rf /usr/share/dotnet
sudo rm -rf /opt/ghc
sudo rm -rf /usr/local/share/boost
sudo rm -rf "$AGENT_TOOLSDIRECTORY"
df -h

- uses: actions/checkout@v4

- name: Set up Go
Expand All @@ -45,9 +53,11 @@ jobs:
# Build your program with the given configuration
run: cmake --build build --config ${{ env.BUILD_TYPE }}

- name: Cleanup
- name: Cleanup Build Trees
run: |
rm -rf ./buildtrees
rm -rf ./deps
df -h

- uses: actions/upload-artifact@v4
with:
Expand All @@ -64,6 +74,14 @@ jobs:
working-directory: ${{ github.workspace }}
run: ./pikatests.sh all clean

- name: Cleanup Build Artifacts
run: |
find ./build -name "*.o" -type f -delete
find ./build -name "*.a" -type f -delete
rm -rf ./build/CMakeFiles
rm -rf ./build/_deps
df -h

# master on port 9221, slave on port 9231, all with 2 db
- name: Start codis, pika master and pika slave
working-directory: ${{ github.workspace }}/build
Expand All @@ -83,13 +101,35 @@ jobs:
chmod +x integrate_test.sh
sh integrate_test.sh

- name: Cleanup Test Data
if: always()
working-directory: ${{ github.workspace }}/build
run: |
pkill -9 pika || true
pkill -9 codis || true
rm -rf master_data slave_data rename_data acl1_data acl2_data acl3_data
rm -rf codis_data_1 codis_data_2
rm -rf *.conf *.conf.bak
df -h


build_on_centos:
runs-on: ubuntu-latest
container:
image: cheniujh/pika-centos7-ci:v5

steps:
- name: Free Disk Space
run: |
rm -rf /usr/share/dotnet
rm -rf /opt/ghc
rm -rf /usr/local/share/boost
find / -type f -name "*.log" -delete 2>/dev/null || true
find / -type f -name "*.tmp" -delete 2>/dev/null || true
find / -name '*cache*' -type d -exec rm -rf {} + 2>/dev/null || true
find / -name '*.bak' -type f -delete 2>/dev/null || true
df -h

- name: Checkout
uses: actions/checkout@v1
with:
Expand All @@ -105,9 +145,14 @@ jobs:
source /opt/rh/devtoolset-10/enable
cmake --build build --config ${{ env.BUILD_TYPE }}

- name: Cleanup
- name: Cleanup Build Trees
run: |
rm -rf ./buildtrees
rm -rf ./deps
# Clean build intermediate files but keep binaries
find ./build -name "*.o" -type f -delete || true
find ./build -name "*.a" -type f -delete || true
df -h

- name: Test
working-directory: ${{ github.workspace }}/build
Expand All @@ -117,13 +162,48 @@ jobs:
working-directory: ${{ github.workspace }}
run: ./pikatests.sh all clean

- name: Cleanup After Unit Test
run: |
# Clean up test data to free space before integration tests
rm -rf ./log* ./db* ./dump* ./dbsync* || true
df -h

- name: Extreme Disk Cleanup
run: |

rm -rf /usr/local/share/* || true
rm -rf /usr/share/doc/* || true
rm -rf /usr/share/man/* || true
rm -rf /var/cache/* || true

find ${{ github.workspace }} -name "*.o" -type f -delete || true
find ${{ github.workspace }} -name "*.a" -type f -delete || true
find ${{ github.workspace }} -name "*.la" -type f -delete || true
find ${{ github.workspace }} -name "*.so" -type f -delete || true
find ${{ github.workspace }} -name "*.pyc" -type f -delete || true

rm -rf ${{ github.workspace }}/.git || true

df -h

echo "Largest directories:"
du -h --max-depth=2 / 2>/dev/null | sort -hr | head -20
Comment on lines +171 to +190

@coderabbitai coderabbitai Bot Dec 3, 2025

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Extreme Disk Cleanup is overly aggressive and risks breaking the build.

This step removes files that may be needed during the build or for binary execution:

  • Line 182 (find ... *.so ... -delete): Removing shared libraries could break the pika binary if it depends on runtime .so files in the build directory or system paths.
  • Line 185 (rm -rf ... .git): Removing the .git directory in CI workflows is risky; some build steps may reference git metadata or history.
  • Lines 174–177: System-level cleanup (/usr/local/share, /usr/share/doc, /var/cache) may interfere with dependencies or system tools needed by the build.

Consider replacing this with targeted, phase-specific cleanup:

  • Clean only after successful builds and before the next phase.
  • Preserve .git, shared libraries, and system dependencies.
  • Target only known-safe directories (build artifacts, test data, caches).

Suggested replacement (conservative approach):

  - name: Extreme Disk Cleanup
    run: |
-     
-     rm -rf /usr/local/share/* || true
-     rm -rf /usr/share/doc/* || true
-     rm -rf /usr/share/man/* || true
-     rm -rf /var/cache/* || true
-     
-     find ${{ github.workspace }} -name "*.o" -type f -delete || true
-     find ${{ github.workspace }} -name "*.a" -type f -delete || true
-     find ${{ github.workspace }} -name "*.la" -type f -delete || true
-     find ${{ github.workspace }} -name "*.so" -type f -delete || true
-     find ${{ github.workspace }} -name "*.pyc" -type f -delete || true
-     
-     rm -rf ${{ github.workspace }}/.git || true
-     
-     df -h
-     
-     echo "Largest directories:"
-     du -h --max-depth=2 / 2>/dev/null | sort -hr | head -20
+     # Clean only CMake intermediate files and test artifacts
+     find ${{ github.workspace }}/build -name "*.o" -type f -delete || true
+     find ${{ github.workspace }}/build -name "*.a" -type f -delete || true
+     rm -rf ${{ github.workspace }}/build/CMakeFiles || true
+     rm -rf ${{ github.workspace }}/build/_deps || true
+     
+     # Do NOT remove:
+     # - .so files (may be runtime dependencies)
+     # - .git (git metadata may be needed)
+     # - System files (could break build tools)
+     
+     df -h
🤖 Prompt for AI Agents 
.github/workflows/pika.yml around lines 171 to 190: the "Extreme Disk Cleanup"
step is overly aggressive (removing system dirs, .so files and the repo .git)
and risks breaking later build steps; replace it with a conservative,
phase-targeted cleanup that only removes known build artifacts and temporary
files (e.g., specific build output directories and compiled object/temp files
inside the workspace), never rm -rf system folders or the workspace .git, avoid
deleting *.so runtime libraries, and run this cleanup only after successful
build/test phases (or scope it to explicit safe paths and rely on actions/cache
for dependency cleanup).


- name: Create Log Directories
run: |
mkdir -p /__w/pikiwidb/pikiwidb/codis/admin/../log
mkdir -p /__w/pikiwidb/pikiwidb/log
mkdir -p ./bin || true
df -h
Comment on lines +192 to +197

@coderabbitai coderabbitai Bot Dec 3, 2025

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Hardcoded absolute path in Create Log Directories step may not work across environments.

Line 195 uses a hardcoded absolute path:

mkdir -p /__w/pikiwidb/pikiwidb/codis/admin/../log

This path is specific to the GitHub Actions runner layout and will fail if:

  • The runner uses a different workspace path.
  • The step runs in a different environment or local CI setup.
  • The directory structure changes.

Replace with a workspace-relative path using the provided variable:

  - name: Create Log Directories
    run: |
-     mkdir -p /__w/pikiwidb/pikiwidb/codis/admin/../log
-     mkdir -p /__w/pikiwidb/pikiwidb/log
+     mkdir -p ${{ github.workspace }}/codis/log
+     mkdir -p ${{ github.workspace }}/log
      mkdir -p ./bin || true
      df -h
🤖 Prompt for AI Agents 
.github/workflows/pika.yml around lines 193 to 198: the Create Log Directories
step uses a hardcoded absolute path (/__w/pikiwidb/...) which will break on
different runners; change those mkdir calls to use the workspace-relative
variable (use $GITHUB_WORKSPACE or the Actions expression ${{ github.workspace
}}) so the directories are created relative to the current workspace (e.g.,
mkdir -p "$GITHUB_WORKSPACE/pikiwidb/codis/admin/../log" and similarly for the
other paths), ensuring portability across environments.


- name: Start codis, pika master and pika slave
working-directory: ${{ github.workspace }}/build
run: |
chmod +x ../tests/integration/start_master_and_slave.sh
../tests/integration/start_master_and_slave.sh
chmod +x ../tests/integration/start_codis.sh
../tests/integration/start_codis.sh

- name: Run Go E2E Tests
working-directory: ${{ github.workspace }}/build
run: |
Expand All @@ -133,6 +213,15 @@ jobs:
chmod +x integrate_test.sh
sh integrate_test.sh

- name: Cleanup Test Data
if: always()
working-directory: ${{ github.workspace }}/build
run: |
rm -rf master_data slave_data rename_data acl1_data acl2_data acl3_data
rm -rf codis_data_1 codis_data_2
rm -rf *.conf *.conf.bak
df -h
Comment on lines +216 to +223

@coderabbitai coderabbitai Bot Dec 2, 2025

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion | 🟠 Major

Add process termination to CentOS Cleanup Test Data for consistency.

The ubuntu Cleanup Test Data (lines 108–109) explicitly terminates pika and codis processes, but the centos equivalent (lines 175–182) omits pkill commands. To ensure consistent cleanup and prevent stale processes from affecting subsequent runs, align centos with ubuntu:

 - name: Cleanup Test Data
   if: always()
   working-directory: ${{ github.workspace }}/build
   run: |
+    pkill -9 pika || true
+    pkill -9 codis || true
     rm -rf master_data slave_data rename_data acl1_data acl2_data acl3_data
     rm -rf codis_data_1 codis_data_2
     rm -rf *.conf *.conf.bak
     df -h
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
- name: Cleanup Test Data
if: always()
working-directory: ${{ github.workspace }}/build
run: |
rm -rf master_data slave_data rename_data acl1_data acl2_data acl3_data
rm -rf codis_data_1 codis_data_2
rm -rf *.conf *.conf.bak
df -h
- name: Cleanup Test Data
if: always()
working-directory: ${{ github.workspace }}/build
run: |
pkill -9 pika || true
pkill -9 codis || true
rm -rf master_data slave_data rename_data acl1_data acl2_data acl3_data
rm -rf codis_data_1 codis_data_2
rm -rf *.conf *.conf.bak
df -h
🤖 Prompt for AI Agents 
.github/workflows/pika.yml around lines 175 to 182: the CentOS "Cleanup Test
Data" step removes files but does not terminate pika/codis processes like the
Ubuntu cleanup does; add the same pkill commands used in the Ubuntu cleanup to
kill pika and codis (e.g., pkill -9 -f pika and pkill -9 -f codis) before
removing data files and configs so leftover processes are terminated and cleanup
is consistent between runners.



build_on_macos:

Expand Down
Loading