Skip to content

Commit 0ba80e5

Browse files
beinanclaude
andauthored
ci: optimize build times with shared caching and streamlined workflows (#43)
## Summary Apply learnings from [lance-graph#143](lance-format/lance-graph#143) to dramatically improve CI performance. That PR achieved a **74% speedup (24min → 6min)** in Python tests through better caching and eliminating duplicate builds. ## Changes ### 1. Shared Rust Cache Across All Workflows ⭐ - Switched from manual `cache@v4` to `Swatinem/rust-cache@v2` - Added `shared-key: "lance-context-deps"` to all workflows - **Impact**: Prevents duplicate compilation of Rust dependencies across Python, Rust, and Style workflows - Cache is now shared across all jobs instead of being isolated ### 2. Remove Nightly from Test Matrix - Only test on `stable` toolchain (matches upstream lance project) - **Impact**: ~50% reduction in Rust test time, prevents nightly breakage from blocking PRs - Stable is the primary deployment target ### 3. Standardize CARGO_INCREMENTAL=0 - Added to `python-test.yml` (was missing) - Added to `style.yml` (was missing) - Already present in `rust-test.yml` - **Impact**: Consistent CI environment, no wasted incremental compilation artifacts ### 4. Optimized Cache Configuration - Removed `CARGO_TARGET_DIR` override (rust-cache handles this better) - Standardized workspaces across all workflows to include both `crates/lance-context-core` and `python` - **Impact**: Better cache hit rates and management ## Expected Performance Impact Based on the improvements seen in lance-graph: - ✅ Reduced duplicate Rust compilations via shared cache key - ✅ Faster CI runs with streamlined test matrix - ✅ Better cache efficiency with rust-cache action - ✅ Consistent build environment across all workflows ## Test Plan - [x] All workflow files updated consistently - [ ] Verify CI passes with new caching strategy - [ ] Monitor CI times to confirm performance improvement 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 7b9a528 commit 0ba80e5

3 files changed

Lines changed: 21 additions & 31 deletions

File tree

.github/workflows/python-test.yml

Lines changed: 11 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ concurrency:
1616
env:
1717
CARGO_TERM_COLOR: always
1818
RUSTFLAGS: "-C debuginfo=0 -C codegen-units=16"
19-
CARGO_TARGET_DIR: ${{ github.workspace }}/.cargo-target
19+
CARGO_INCREMENTAL: "0"
2020

2121
jobs:
2222
build-wheel:
@@ -32,16 +32,12 @@ jobs:
3232
uses: actions-rust-lang/setup-rust-toolchain@v1
3333
with:
3434
cache: false
35-
- name: Cache Rust build (shared)
36-
uses: actions/cache@v4
35+
- uses: Swatinem/rust-cache@v2
3736
with:
38-
path: |
39-
.cargo-target
40-
~/.cargo/registry
41-
~/.cargo/git
42-
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
43-
restore-keys: |
44-
${{ runner.os }}-cargo-
37+
shared-key: "lance-context-deps"
38+
workspaces: |
39+
crates/lance-context-core
40+
python
4541
- name: Install dependencies
4642
run: |
4743
sudo apt update
@@ -89,16 +85,12 @@ jobs:
8985
uses: actions-rust-lang/setup-rust-toolchain@v1
9086
with:
9187
cache: false
92-
- name: Cache Rust build (shared)
93-
uses: actions/cache@v4
88+
- uses: Swatinem/rust-cache@v2
9489
with:
95-
path: |
96-
.cargo-target
97-
~/.cargo/registry
98-
~/.cargo/git
99-
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
100-
restore-keys: |
101-
${{ runner.os }}-cargo-
90+
shared-key: "lance-context-deps"
91+
workspaces: |
92+
crates/lance-context-core
93+
python
10294
- name: Install dependencies
10395
run: |
10496
sudo apt update

.github/workflows/rust-test.yml

Lines changed: 5 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,6 @@ env:
1919
RUSTFLAGS: "-C debuginfo=1"
2020
RUST_BACKTRACE: "1"
2121
CARGO_INCREMENTAL: "0"
22-
CARGO_TARGET_DIR: ${{ github.workspace }}/.cargo-target
2322

2423
jobs:
2524
test:
@@ -29,23 +28,18 @@ jobs:
2928
matrix:
3029
toolchain:
3130
- stable
32-
- nightly
3331
steps:
3432
- uses: actions/checkout@v4
3533
- name: Setup rust toolchain
3634
run: |
3735
rustup toolchain install ${{ matrix.toolchain }}
3836
rustup default ${{ matrix.toolchain }}
39-
- name: Cache Rust build (shared)
40-
uses: actions/cache@v4
37+
- uses: Swatinem/rust-cache@v2
4138
with:
42-
path: |
43-
.cargo-target
44-
~/.cargo/registry
45-
~/.cargo/git
46-
key: ${{ runner.os }}-cargo-${{ hashFiles('**/Cargo.lock') }}
47-
restore-keys: |
48-
${{ runner.os }}-cargo-
39+
shared-key: "lance-context-deps"
40+
workspaces: |
41+
crates/lance-context-core
42+
python
4943
- name: Install dependencies
5044
run: |
5145
sudo apt update

.github/workflows/style.yml

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ concurrency:
1717
env:
1818
CARGO_TERM_COLOR: always
1919
RUSTFLAGS: "-C debuginfo=1"
20+
CARGO_INCREMENTAL: "0"
2021

2122
jobs:
2223
format:
@@ -37,7 +38,10 @@ jobs:
3738
- uses: actions/checkout@v4
3839
- uses: Swatinem/rust-cache@v2
3940
with:
40-
workspaces: .
41+
shared-key: "lance-context-deps"
42+
workspaces: |
43+
crates/lance-context-core
44+
python
4145
- name: Install dependencies
4246
run: |
4347
sudo apt update

0 commit comments

Comments
 (0)