Feature/ci optimization bigbox#209
Conversation
There was a problem hiding this comment.
Pull Request Overview
Shift CI workloads to a self-hosted "bigbox" runner for faster builds, add a validation script for the runner, tweak timeouts, and adjust linting settings.
- Add scripts/test-bigbox-runner.sh to validate dependencies on the bigbox runner.
- Migrate multiple workflows to runs-on: [self-hosted, linux, bigbox] and tune timeouts.
- Update clippy invocation flags and increase test timeouts where appropriate.
Reviewed Changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| scripts/test-vm-execution.sh | Increase default test timeout to 10 minutes to accommodate longer runs. |
| scripts/test-bigbox-runner.sh | New script to verify bigbox runner readiness (tools, toolchains, basic compile/run tests). |
| scripts/ci-check-format.sh | Adjust clippy flags/timeout; change diagnostic behavior. |
| .github/workflows/vm-execution-tests.yml | Switch all jobs to the bigbox runner and adjust job timeouts. |
| .github/workflows/test-matrix.yml | Run matrix jobs on bigbox runner. |
| .github/workflows/tauri-build.yml | Run summary stage on bigbox runner. |
| .github/workflows/frontend-build.yml | Run frontend build on bigbox runner with reduced timeout. |
| .github/workflows/earthly-runner.yml | Run Earthly-related jobs on bigbox runner. |
| .github/workflows/claude-code-review.yml | Run code-review workflow on bigbox runner. |
| .github/workflows/ci-optimized.yml | Run optimized CI stages on bigbox runner. |
| .github/workflows/ci-native.yml | Run native CI stages on bigbox runner and reduce some timeouts. |
| .github/BIGBOX_RUNNER_SETUP.md | New documentation for setting up and tuning the bigbox runner. |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| # Summary | ||
| echo "" | ||
| echo -e "${BLUE}📊 Validation Summary${NC}" | ||
| echo "==========================" | ||
|
|
||
| # Count results | ||
| TOTAL_CHECKS=0 | ||
| PASSED_CHECKS=0 | ||
|
|
||
| # Simple validation completion check | ||
| echo "Validation completed. Review the output above for any missing dependencies." | ||
| echo "" | ||
| echo -e "${GREEN}✅ Bigbox runner validation complete!${NC}" | ||
| echo "" | ||
| echo "Next steps:" | ||
| echo "1. Install any missing dependencies identified above" | ||
| echo "2. Set up GitHub Actions runner with labels: self-hosted, linux, bigbox" | ||
| echo "3. Test runner with: ./run.sh --run" | ||
| echo "4. Verify CI workflows use the runner correctly" No newline at end of file |
There was a problem hiding this comment.
The script always prints a success message and exits 0 even if earlier checks reported missing dependencies. Track failures (e.g., set a FAILURES counter in the tool/deps loops) and exit non-zero at the end if any required dependency is missing; also adjust the summary to reflect pass/fail.
| TOTAL_CHECKS=0 | ||
| PASSED_CHECKS=0 |
There was a problem hiding this comment.
TOTAL_CHECKS and PASSED_CHECKS are declared but never used. Either implement real counting (and report it in the summary) or remove these variables to avoid dead code.
| TOTAL_CHECKS=0 | |
| PASSED_CHECKS=0 |
| RUST_VERSION=$(rustc --version | cut -d' ' -f2) | ||
| echo -e " ${GREEN}✓${NC} Rust $RUST_VERSION" |
There was a problem hiding this comment.
This block checks for cargo but unconditionally runs rustc; on some setups cargo may exist without rustc in PATH, which can cause a failure. Guard rustc separately (e.g., command -v rustc) or derive the version from cargo --version to avoid set -e aborting here.
| RUST_VERSION=$(rustc --version | cut -d' ' -f2) | |
| echo -e " ${GREEN}✓${NC} Rust $RUST_VERSION" | |
| if command -v rustc &> /dev/null; then | |
| RUST_VERSION=$(rustc --version | cut -d' ' -f2) | |
| echo -e " ${GREEN}✓${NC} Rust $RUST_VERSION" | |
| else | |
| echo -e " ${RED}✗${NC} rustc (missing)" | |
| echo -e "${YELLOW} Install with: rustup install stable${NC}" | |
| fi |
| if command -v npm &> /dev/null; then | ||
| echo "Creating test Node.js project..." | ||
| TEST_DIR=$(mktemp -d) | ||
| cd "$TEST_DIR" | ||
|
|
||
| npm init -y > /dev/null 2>&1 | ||
| echo 'console.log("Hello from bigbox!");' > test.js | ||
|
|
||
| if node test.js > /dev/null 2>&1; then | ||
| echo -e " ${GREEN}✓${NC} Basic Node.js execution successful" | ||
| else | ||
| echo -e " ${RED}✗${NC} Node.js execution failed" | ||
| fi |
There was a problem hiding this comment.
This test block is gated on npm but executes node; if npm is present but node is not available in PATH, this can fail. Gate the block on command -v node (or both node and npm) before running node test.js.
| echo -e " ${GREEN}✓${NC} $tool" | ||
| else | ||
| echo -e " ${RED}✗${NC} $tool (missing)" | ||
| echo -e "${YELLOW} Install with: sudo apt-get install $tool${NC}" |
There was a problem hiding this comment.
The generic install hint can be incorrect for some tools (e.g., Docker typically requires docker.io or official convenience scripts rather than apt-get install docker). Special-case messages for Docker (and others as needed) or point to official installation docs to avoid misleading guidance.
| echo -e "${YELLOW} Install with: sudo apt-get install $tool${NC}" | |
| if [ "$tool" = "docker" ]; then | |
| echo -e "${YELLOW} Install Docker via official docs: https://docs.docker.com/get-docker/${NC}" | |
| else | |
| echo -e "${YELLOW} Install with: sudo apt-get install $tool${NC}" | |
| fi |
|
|
||
| # Run clippy with optimized flags and extended timeout | ||
| if timeout 900 cargo clippy --workspace --all-targets --all-features -- -D warnings; then | ||
| if timeout 1200 cargo clippy --workspace --all-targets --all-features --message-format=short --quiet -- -D clippy::all -A clippy::nursery -A clippy::pedantic; then |
There was a problem hiding this comment.
The previous -D warnings enforcement was removed, so rustc warnings will no longer fail the job. Consider adding -D warnings after the -- to keep compiler warnings failing CI: cargo clippy ... -- -D warnings -D clippy::all -A clippy::nursery -A clippy::pedantic.
| if timeout 1200 cargo clippy --workspace --all-targets --all-features --message-format=short --quiet -- -D clippy::all -A clippy::nursery -A clippy::pedantic; then | |
| if timeout 1200 cargo clippy --workspace --all-targets --all-features --message-format=short --quiet -- -D warnings -D clippy::all -A clippy::nursery -A clippy::pedantic; then |
| ```bash | ||
| curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y | ||
| source ~/.cargo/env | ||
| rustup default 1.87.0 |
There was a problem hiding this comment.
Pinning the default toolchain here can drift from the project configuration; consider referencing the repo’s rust-toolchain(.toml) or using rustup default stable (and documenting the minimum supported Rust version) to keep the runner aligned with the codebase.
| ```bash | |
| curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y | |
| source ~/.cargo/env | |
| rustup default 1.87.0 | |
| > **Note:** The runner should use the Rust toolchain specified in the project's `rust-toolchain` or `rust-toolchain.toml` file if present. If not, use the stable toolchain. The minimum supported Rust version is 1.87.0. | |
| ```bash | |
| curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y | |
| source ~/.cargo/env | |
| # Use the toolchain file if present, otherwise default to stable | |
| if [ -f rust-toolchain ] || [ -f rust-toolchain.toml ]; then | |
| rustup show | grep -q "Default toolchain" || true | |
| # This will automatically use the toolchain specified in the file when building | |
| else | |
| rustup default stable | |
| fi |
| - Docker and Docker Compose installed | ||
| - Rust toolchain (1.87.0+) | ||
| - Node.js (18+) | ||
| - Sufficient disk space for caching | ||
| - SSH access for management | ||
|
|
There was a problem hiding this comment.
Clarify whether Compose v2 (docker compose plugin) is required and add install/verification steps (e.g., docker compose version) so runner operators know how to satisfy this prerequisite.
| - Docker and Docker Compose installed | |
| - Rust toolchain (1.87.0+) | |
| - Node.js (18+) | |
| - Sufficient disk space for caching | |
| - SSH access for management | |
| - Docker and Docker Compose v2 (the Docker Compose plugin) installed | |
| - Rust toolchain (1.87.0+) | |
| - Node.js (18+) | |
| - Sufficient disk space for caching | |
| - SSH access for management | |
| ### Docker Compose v2 Installation & Verification | |
| Ensure you have Docker Compose v2 (the Docker Compose plugin) installed. You can verify this by running: | |
| ```bash | |
| docker compose version |
The output should show a version starting with Docker Compose version v2.x.x. If you see docker-compose (with a hyphen) or a version below v2, you need to upgrade.
Install/Upgrade Docker Compose v2:
- If you installed Docker via apt or the official Docker repository, Docker Compose v2 is usually included.
- To manually install or upgrade, follow the official guide: https://docs.docker.com/compose/install/
- On Ubuntu, you can ensure the plugin is installed with:
sudo apt-get update sudo apt-get install docker-compose-plugin
- After installation, verify again with
docker compose version.
Feature/ci optimization bigbox
No description provided.