-
-
Notifications
You must be signed in to change notification settings - Fork 9
386 lines (351 loc) · 16.4 KB
/
Copy pathdelta-sync-integration.yml
File metadata and controls
386 lines (351 loc) · 16.4 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
name: Delta Sync Integration
# Validates the rsync-over-SSH stack end-to-end against a local docker
# fixture. Cheap to run, scoped to paths that actually affect the feature
# so unrelated PRs don't pay for it.
on:
push:
branches: [main]
paths:
- 'src-tauri/src/lib.rs'
- 'src-tauri/src/settings.rs'
- 'src-tauri/src/ssh_exec.rs'
- 'src-tauri/src/rsync_output.rs'
- 'src-tauri/src/rsync_over_ssh.rs'
- 'src-tauri/src/delta_transport.rs'
- 'src-tauri/src/delta_sync_rsync.rs'
# The native rsync wire implementation lives here and is the code
# this job actually exercises end-to-end. Omitting it meant a
# native-rsync change did not trigger this job at its own commit;
# the regression only surfaced later when an unrelated filtered
# path (Cargo.lock, lib.rs) triggered the job, producing
# misleading "clean bisects" that blamed the triggering commit.
- 'src-tauri/src/aerorsync/**'
- 'src-tauri/src/providers/sftp.rs'
- 'src-tauri/Cargo.lock'
- 'src-tauri/tests/fixtures/sftp-rsync/**'
- 'src-tauri/tests/integration_delta_sync.rs'
- '.github/workflows/delta-sync-integration.yml'
pull_request:
branches: [main]
paths:
- 'src-tauri/src/lib.rs'
- 'src-tauri/src/settings.rs'
- 'src-tauri/src/ssh_exec.rs'
- 'src-tauri/src/rsync_output.rs'
- 'src-tauri/src/rsync_over_ssh.rs'
- 'src-tauri/src/delta_transport.rs'
- 'src-tauri/src/delta_sync_rsync.rs'
- 'src-tauri/src/aerorsync/**'
- 'src-tauri/src/providers/sftp.rs'
- 'src-tauri/Cargo.lock'
- 'src-tauri/tests/fixtures/sftp-rsync/**'
- 'src-tauri/tests/integration_delta_sync.rs'
- '.github/workflows/delta-sync-integration.yml'
workflow_dispatch:
permissions:
contents: read
jobs:
integration:
name: SFTP rsync fixture + live test (key-auth)
runs-on: ubuntu-latest
# 15 minutes was under the job's own runtime, so it died wherever the wall
# happened to fall: on the integration test at 4df30affc and 45880d0a2, on
# the post-job cache save at 8b39ed171, always reported as "cancelled"
# rather than a failure. The sibling fallback-fixture job does comparable
# work, is allowed 25, and takes about 16. Match it.
timeout-minutes: 25
steps:
- name: Checkout
uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5
# The aeroftp lib in test profile plus the docker fixture no longer fit
# in the runner's free disk: this lane started dying with "No space left
# on device" (or wedging until the job timeout) once the crate outgrew
# the runner headroom (first red at d744affc). Same reclaim step
# cli-smoke.yml and build.yml already use.
- name: Free disk space
run: |
echo "Before:"; df -h /
sudo rm -rf /usr/share/dotnet /opt/ghc /usr/local/lib/android \
/opt/hostedtoolcache/CodeQL /usr/local/share/boost || true
sudo docker image prune --all --force >/dev/null 2>&1 || true
echo "After:"; df -h /
- name: Install Rust
uses: dtolnay/rust-toolchain@29eef336d9b2848a0b548edc03f92a220660cdb8 # stable
with:
toolchain: stable
- name: Rust cache
uses: swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2
with:
workspaces: './src-tauri -> target'
- name: Install system dependencies
run: |
sudo apt-get update
# rsync + ssh client for the host side of the integration test.
# Webkit/GTK stack is needed because compiling the aeroftp
# library pulls Tauri deps even when we only run one test binary.
sudo apt-get install -y \
rsync \
openssh-client \
libwebkit2gtk-4.1-dev \
libappindicator3-dev \
librsvg2-dev
- name: Generate fixture SSH key
working-directory: src-tauri/tests/fixtures/sftp-rsync
run: bash setup.sh
- name: Build & start docker fixture
working-directory: src-tauri/tests/fixtures/sftp-rsync
run: docker compose up -d --build
- name: Wait for fixture SSH
run: |
for i in $(seq 1 20); do
if ssh -i src-tauri/tests/fixtures/sftp-rsync/ssh_key \
-o StrictHostKeyChecking=no \
-o UserKnownHostsFile=/dev/null \
-o BatchMode=yes \
-o ConnectTimeout=2 \
-p 2222 testuser@127.0.0.1 'true' 2>/dev/null; then
echo "Fixture SSH ready after $((i * 2))s"
exit 0
fi
echo "Waiting for fixture SSH... ($i/20)"
sleep 2
done
echo "Fixture did not come up within 40s; dumping logs" >&2
docker logs aeroftp-delta-sync-fixture >&2 || true
exit 1
- name: "Unit tests: delta-sync modules"
working-directory: src-tauri
run: |
cargo test --lib --no-fail-fast -- \
rsync_output:: \
rsync_over_ssh:: \
ssh_exec:: \
delta_transport:: \
delta_sync_rsync::
- name: "Integration test: delta sync over SSH"
working-directory: src-tauri
run: cargo test --test integration_delta_sync -- --ignored --nocapture
- name: Dump fixture logs on failure
if: failure()
working-directory: src-tauri/tests/fixtures/sftp-rsync
run: |
docker logs aeroftp-delta-sync-fixture || true
docker compose ps
- name: Teardown fixture
if: always()
working-directory: src-tauri/tests/fixtures/sftp-rsync
run: docker compose down -v
fallback-fixture:
# PR-T02: covers the silent-fallback branch when the SFTP session
# is password-only (delta path returns `None`, classic upload takes
# over) and the offline hard-rejection contract. Runs in parallel
# with the key-auth integration job so a regression in either lane
# blocks the PR independently.
name: SFTP password-only fixture + fallback contract
runs-on: ubuntu-latest
# Cold-cache PRs (Cargo.lock churn from deps bumps) compile the full
# workspace + Tauri + aeroftp lib before the test binary links. The
# sibling key-auth job needs ~11min on a warm shared cache; this lane
# uses an isolated shared-key so the first cache miss can take 18-22
# minutes. 25min gives enough headroom without parking PRs forever.
timeout-minutes: 25
steps:
- name: Checkout
uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5
# Same disk reclaim as the key-auth lane above (this lane hit the same
# "No space left on device" while rebuilding the lib for the second
# cargo test invocation).
- name: Free disk space
run: |
echo "Before:"; df -h /
sudo rm -rf /usr/share/dotnet /opt/ghc /usr/local/lib/android \
/opt/hostedtoolcache/CodeQL /usr/local/share/boost || true
sudo docker image prune --all --force >/dev/null 2>&1 || true
echo "After:"; df -h /
- name: Install Rust
uses: dtolnay/rust-toolchain@29eef336d9b2848a0b548edc03f92a220660cdb8 # stable
with:
toolchain: stable
- name: Rust cache
uses: swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2
with:
workspaces: './src-tauri -> target'
shared-key: delta-sync-fallback
- name: Install system dependencies
# sshpass needed to poll the password-auth fixture readiness
# (the integration test itself uses russh via SftpProvider -
# sshpass is just a scripting convenience).
run: |
sudo apt-get update
sudo apt-get install -y \
rsync \
openssh-client \
sshpass \
libwebkit2gtk-4.1-dev \
libappindicator3-dev \
librsvg2-dev
- name: Build & start password-auth docker fixture
uses: nick-fields/retry@ad984534de44a9489a53aefd81eb77f87c70dc60 # v3
with:
timeout_minutes: 5
max_attempts: 2
retry_wait_seconds: 10
command: |
cd src-tauri/tests/fixtures/sftp-rsync
docker compose -f docker-compose.password.yml up -d --build
- name: Wait for password fixture SSH
run: |
for i in $(seq 1 20); do
if sshpass -p testpass ssh \
-o StrictHostKeyChecking=no \
-o UserKnownHostsFile=/dev/null \
-o BatchMode=no \
-o PreferredAuthentications=password \
-o PubkeyAuthentication=no \
-o ConnectTimeout=2 \
-p 2223 testuser@127.0.0.1 'true' 2>/dev/null; then
echo "Password fixture SSH ready after $((i * 2))s"
exit 0
fi
echo "Waiting for password fixture SSH... ($i/20)"
sleep 2
done
echo "Password fixture did not come up within 40s; dumping logs" >&2
docker logs aeroftp-delta-sync-fixture-password >&2 || true
exit 1
- name: Offline hard-rejection contract
working-directory: src-tauri
# Non-ignored offline pins: no fixture needed but we run them
# here too so a change that breaks the string contract without
# touching the key-auth lane's paths is still caught.
run: |
cargo test --test integration_delta_sync -- \
hard_rejection_string_contract_is_pinned_offline \
hard_error_branch_runs_before_classic_fallback_in_bivio \
--nocapture
- name: "Fallback integration test: password-only fixture"
working-directory: src-tauri
run: |
cargo test --test integration_delta_sync -- \
--ignored \
product_path_falls_through_silently_when_session_not_eligible \
--nocapture
- name: Dump fixture logs on failure
if: failure()
working-directory: src-tauri/tests/fixtures/sftp-rsync
run: |
docker logs aeroftp-delta-sync-fixture-password || true
docker compose -f docker-compose.password.yml ps
- name: Teardown fixture
if: always()
working-directory: src-tauri/tests/fixtures/sftp-rsync
run: docker compose -f docker-compose.password.yml down -v
windows-native:
# PR-T11: validate the cross-OS surgical gate on Windows.
# Since v3.6.1 `aerorsync` is in the default feature set,
# so the "default features" matrix entry is the same as
# "--features aerorsync". The two meaningful matrix entries
# are now:
# - default (native prototype on): the shipping configuration
# - `--no-default-features`: the classic-only fallback path; on
# Windows this must compile to a `delta_transport() -> None`
# surface so the consumer drops silently to plain SFTP.
# No live fixture on Windows: the CI image cannot reach a Linux rsync
# server and the native prototype's live probe is covered by the
# aerorsync-protocol.yml lane against the Linux fixture.
name: Windows cross-OS build + native unit tests
# Pin to windows-2022 while cmake-rs (crate `cmake` 0.1.58 used
# transitively by whisper-rs-sys 0.15.0) hardcodes the generator
# string `Visual Studio 18 2026` whenever it detects MSVC v18 on
# the runner, but CMake itself does not yet know that generator.
# Same root cause that hit cli-smoke and was patched in v3.7.2.
# windows-2022 still ships VS 17 2022, which both cmake-rs and
# CMake support.
runs-on: windows-2022
# MSVC links roughly 2x slower than ubuntu-latest on the large
# lib target; the single-config run used to fit in 20 min but the
# v3.6.1 default-feature change doubled the amount of code the
# test binary pulls in. 40 min is the comfortable ceiling.
timeout-minutes: 40
steps:
- name: Checkout
uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5
- name: Install Rust (stable, msvc)
uses: dtolnay/rust-toolchain@29eef336d9b2848a0b548edc03f92a220660cdb8 # stable
with:
toolchain: stable
targets: x86_64-pc-windows-msvc
- name: Rust cache
uses: swatinem/rust-cache@c19371144df3bb44fab255c43d04cbc2ab54d1c4 # v2
with:
workspaces: './src-tauri -> target'
shared-key: delta-sync-windows
# Mirror the prefix bumped in build.yml / cli-smoke.yml on
# 2026-05-06 to invalidate any cached whisper-rs-sys
# CMakeCache.txt referencing Visual Studio 17 2022 after the
# windows-latest image rolled to VS 18 2026. windows-2022 is
# also pinned above as belt-and-suspenders, but if the pin is
# ever lifted we still want the cache invalidation in place.
prefix-key: 'v2-whisper-vs18'
# whisper-rs-sys vendors whisper.cpp and runs CMake at build time.
# The generator picked by cmake-rs is whichever Visual Studio
# CMake finds on the runner. When GitHub rolls the image to a
# newer VS, the rust-cache restored CMakeCache.txt still
# references the previous generator and CMake aborts. Belt-and-
# suspenders: drop the whole `whisper-rs-sys-*` tree (debug and
# release) so cmake-rs reconfigures cleanly. Removing only `out/`
# used to leave `out/build/CMakeCache.txt` behind on some runners
# depending on cmake-rs internal layout.
- name: Drop stale whisper-rs-sys CMake artifacts (Windows)
shell: pwsh
run: |
foreach ($profile in @('debug', 'release')) {
$base = "src-tauri/target/$profile/build"
$purged = 0
if (Test-Path $base) {
Get-ChildItem $base -Directory -ErrorAction SilentlyContinue |
Where-Object { $_.Name -like 'whisper-rs-sys-*' } |
ForEach-Object {
Write-Host "Removing whole whisper-rs-sys build dir: $($_.FullName)"
Remove-Item -Recurse -Force $_.FullName -ErrorAction SilentlyContinue
$purged++
}
Write-Host "[$profile] Purged $purged whisper-rs-sys build dir(s)"
}
}
# Compile-surface gate for the classic-only fallback path:
# `--no-default-features` turns `aerorsync` off, and on
# Windows the binary-rsync transport is also gated out. The only
# thing that should survive on this path is
# `SftpProvider::delta_transport() -> None`, which the consumer
# treats as "drop to plain SFTP silently". Regression here means
# the surgical `#[cfg(unix)]` gate has leaked a Unix-only symbol
# back into the cross-OS surface.
- name: cargo check (--no-default-features, classic-only fallback)
working-directory: src-tauri
run: cargo check --locked --no-default-features
# Compile-surface gate for the shipping configuration (default
# features, which include `aerorsync`). This confirms
# the native rsync driver, the type layer
# (`RsyncCapability`/`RsyncConfig`/`RsyncError`/`RsyncStats`),
# the cross-platform adapter, and `SftpProvider::delta_transport()`
# all build cleanly under MSVC with the feature on.
- name: "cargo check (default features: shipping configuration)"
working-directory: src-tauri
run: cargo check --locked --all-targets
# Note on the test binary: a full `cargo test --lib` on Windows
# requires the whole lib-test link closure, which pulls in
# `whisper-rs-sys` (STT for Linux/Windows, excluded on macOS).
# `whisper-rs-sys` link-time symbols against the MSVC import
# library stubs (`__imp_rand`, `__imp__aligned_malloc`,
# `__imp_expm1f`, `__imp_fmaxf`, `__imp_erff`, etc.) do not
# resolve cleanly in the `cargo test` profile today (a known
# issue with the vendored ggml sources' CRT import descriptors
# under recent MSVC toolchains). The cross-platform unit tests
# we actually care about for PR-T11 (delta_sync_rsync
# eligibility probes, rsync_output parsing, native prototype
# encoders, ndx round-trips) are executed by the
# aerorsync-protocol.yml lane and by the Linux integration
# job above. Adding a Windows-side test runner is tracked as a
# follow-up to harden whisper-rs-sys MSVC linkage separately
# from the release gate.