Commit 0729c7d
authored
Add support for using pre-built images (#6737)
* api: add InitialSync type to LiveUpdateSpec
Signed-off-by: arnas dundulis <arnas@vinted.com>
* tiltfile: add initial_sync() live update step
Signed-off-by: arnas dundulis <arnas@vinted.com>
* reconciler: implement initial_sync file collection
Signed-off-by: arnas dundulis <arnas@vinted.com>
* fix: enter maybeSync when initial_sync is configured
Without this, maybeSync is gated behind hasChangesToSync which requires
file change events or container state changes. On a fresh tilt up, the
container starts with no file changes detected, so maybeSync is never
entered and isInitialSync is never checked. This means initial_sync
silently does nothing.
Fix: when InitialSync is configured on the spec, always set
hasChangesToSync=true so the reconciler enters maybeSync. The
isInitialSync check inside maybeSync still correctly gates on whether
the container is new (lastFileTimeSynced is zero).
Signed-off-by: arnas dundulis <arnas@vinted.com>
* fix: initial_sync tests match wrong container and miss auto-reconciles
Two pre-existing test bugs:
1. Test containers use Image "frontend-image" which doesn't match the
ImageMap selector (expects "local-registry:12345/frontend-image:my-tag").
Tests were actually syncing to the setupFrontend fixture's "main-id"
container, not the test's "container-1". Assertions only checked call
count so this was hidden.
2. f.Update() and f.kdUpdateStatus() auto-reconcile via MustReconcile,
so initial sync fires during setup before the test's explicit
MustReconcile call. Tests now clear f.cu.Calls before the assertion
window and account for the correct reconcile sequence.
Signed-off-by: arnas dundulis <arnas@vinted.com>
* optimize: tar batching for initial_sync
During initial sync, instead of collecting all individual file paths,
converting them to PathMappings via FilesToPathMappings (O(files x syncs)),
statting each for existence via MissingLocalPaths, and then creating a tar
from individual file mappings — build the tar archive directly from
directory-level sync mappings with the ignore filter passed to
TarArchiveForPaths.
This means a single directory walk produces the tar stream, skipping
the redundant FilesToPathMappings conversion and MissingLocalPaths stat
calls. collectAllSyncedFiles still runs for BoilRuns trigger matching.
Benchmarks at 14k files:
old (individual files): 189ms, 33MB allocs
new (tar batched): 164ms, 21MB allocs (14% faster, 37% less memory)
Also adds:
- buildInitialSyncFilter: composes per-sync-path ignore matchers into
a single filter for TarArchiveForPaths
- Warnings when sync paths don't exist or dockerignore path has no
.dockerignore file during initial sync
Signed-off-by: arnas dundulis <arnas@vinted.com>
* test: verify tar archive contents during initial_sync
Add 6 tests that inspect the actual tar archive produced by initial_sync:
- TarBatching_IgnoresFilteredFiles: IgnorePaths exclude files from tar
- TarBatching_DockerignoreExcludesFromTar: .dockerignore patterns applied
- TarBatching_MultipleSyncPathsInSingleTar: files from all syncs in one tar
- TarBatching_NoDeletesDuringInitialSync: ToDelete is always empty
- TarBatching_GlobPatternExcludesFromTar: **/testdata glob works
- TarBatching_LargeFileTree: 1000 files with 100 ignored, correct count
Signed-off-by: arnas dundulis <arnas@vinted.com>
* cleanup: rename noMoreInitialSync, update IgnorePaths docs
- Rename noMoreInitialSync → seenInitialSync in liveUpdateFromSteps
for clarity (the variable tracks whether we've seen an initial_sync
step, not whether more are allowed)
- Update IgnorePaths field comment to accurately describe dockerignore
glob syntax support (**/ patterns, *.ext wildcards) instead of just
"exact matches and directory prefixes"
Signed-off-by: arnas dundulis <arnas@vinted.com>
* generate openapi
Signed-off-by: arnas dundulis <arnas@vinted.com>
* fix: only force maybeSync when containers still need initial sync
The previous approach set hasChangesToSync=true on every reconcile when
InitialSync was configured, causing unnecessary work entering and
exiting maybeSync after all containers have already been synced.
Now uses monitor.needsInitialSync() which returns true only when no
containers are tracked yet (first reconcile) or when any tracked
container has never been synced (lastFileTimeSynced is zero). Once all
containers complete their initial sync, the normal hasChangesToSync
gate applies.
Signed-off-by: arnas dundulis <arnas@vinted.com>
* cleanup: remove unnecessary err2 variable in applyInternal
The err2 rename was an artifact of the tar batching refactor. Since err
is not used in the outer scope at that point, use err directly with var.
Signed-off-by: arnas dundulis <arnas@vinted.com>
* fix: use path.IsAbs for ignore path validation (Windows compat)
filepath.IsAbs("/absolute/path") returns false on Windows since it's
not a Windows absolute path (no drive letter). Use path.IsAbs (POSIX
semantics) instead so the validation rejects Unix-style absolute paths
on all platforms.
Fixes TestLiveUpdate_InitialSync_AbsolutePathError on Windows.
Signed-off-by: arnas dundulis <arnas@vinted.com>
* ci: retry
Signed-off-by: arnas dundulis <arnas@vinted.com>
* remove explicit ignore, reuse live_update ignore set
Signed-off-by: arnas dundulis <arnas@vinted.com>
---------
Signed-off-by: arnas dundulis <arnas@vinted.com>1 parent c3400c3 commit 0729c7d
11 files changed
Lines changed: 1151 additions & 30 deletions
File tree
- internal
- controllers/core/liveupdate
- tiltfile
- pkg
- apis/core/v1alpha1
- openapi
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
| 6 | + | |
6 | 7 | | |
7 | 8 | | |
8 | 9 | | |
| |||
18 | 19 | | |
19 | 20 | | |
20 | 21 | | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
21 | 30 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
| 37 | + | |
37 | 38 | | |
38 | 39 | | |
39 | 40 | | |
| |||
44 | 45 | | |
45 | 46 | | |
46 | 47 | | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
47 | 64 | | |
48 | 65 | | |
49 | 66 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
7 | 10 | | |
8 | 11 | | |
9 | 12 | | |
| |||
27 | 30 | | |
28 | 31 | | |
29 | 32 | | |
| 33 | + | |
30 | 34 | | |
31 | 35 | | |
32 | 36 | | |
| |||
181 | 185 | | |
182 | 186 | | |
183 | 187 | | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
184 | 198 | | |
185 | 199 | | |
186 | 200 | | |
| |||
307 | 321 | | |
308 | 322 | | |
309 | 323 | | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
310 | 336 | | |
311 | 337 | | |
312 | 338 | | |
313 | 339 | | |
314 | 340 | | |
315 | 341 | | |
| 342 | + | |
316 | 343 | | |
317 | 344 | | |
318 | 345 | | |
| |||
328 | 355 | | |
329 | 356 | | |
330 | 357 | | |
331 | | - | |
332 | | - | |
333 | | - | |
334 | | - | |
335 | | - | |
336 | | - | |
337 | | - | |
338 | | - | |
339 | 358 | | |
340 | 359 | | |
341 | 360 | | |
| |||
697 | 716 | | |
698 | 717 | | |
699 | 718 | | |
| 719 | + | |
| 720 | + | |
| 721 | + | |
| 722 | + | |
| 723 | + | |
| 724 | + | |
| 725 | + | |
| 726 | + | |
| 727 | + | |
| 728 | + | |
| 729 | + | |
| 730 | + | |
| 731 | + | |
| 732 | + | |
| 733 | + | |
| 734 | + | |
| 735 | + | |
| 736 | + | |
| 737 | + | |
| 738 | + | |
| 739 | + | |
| 740 | + | |
| 741 | + | |
700 | 742 | | |
701 | 743 | | |
702 | 744 | | |
| |||
778 | 820 | | |
779 | 821 | | |
780 | 822 | | |
| 823 | + | |
| 824 | + | |
781 | 825 | | |
782 | 826 | | |
783 | 827 | | |
| |||
906 | 950 | | |
907 | 951 | | |
908 | 952 | | |
909 | | - | |
910 | | - | |
911 | | - | |
912 | | - | |
913 | | - | |
914 | | - | |
| 953 | + | |
| 954 | + | |
| 955 | + | |
| 956 | + | |
| 957 | + | |
| 958 | + | |
| 959 | + | |
| 960 | + | |
| 961 | + | |
| 962 | + | |
| 963 | + | |
| 964 | + | |
| 965 | + | |
| 966 | + | |
| 967 | + | |
| 968 | + | |
| 969 | + | |
| 970 | + | |
| 971 | + | |
| 972 | + | |
| 973 | + | |
| 974 | + | |
| 975 | + | |
| 976 | + | |
| 977 | + | |
915 | 978 | | |
916 | | - | |
917 | | - | |
918 | 979 | | |
919 | | - | |
920 | | - | |
921 | | - | |
922 | | - | |
| 980 | + | |
| 981 | + | |
| 982 | + | |
| 983 | + | |
| 984 | + | |
923 | 985 | | |
924 | | - | |
925 | 986 | | |
926 | | - | |
927 | | - | |
928 | | - | |
929 | | - | |
| 987 | + | |
| 988 | + | |
| 989 | + | |
| 990 | + | |
| 991 | + | |
930 | 992 | | |
931 | 993 | | |
932 | 994 | | |
| |||
935 | 997 | | |
936 | 998 | | |
937 | 999 | | |
938 | | - | |
| 1000 | + | |
939 | 1001 | | |
940 | 1002 | | |
941 | 1003 | | |
| |||
1134 | 1196 | | |
1135 | 1197 | | |
1136 | 1198 | | |
| 1199 | + | |
| 1200 | + | |
| 1201 | + | |
| 1202 | + | |
| 1203 | + | |
| 1204 | + | |
| 1205 | + | |
| 1206 | + | |
| 1207 | + | |
| 1208 | + | |
| 1209 | + | |
| 1210 | + | |
| 1211 | + | |
| 1212 | + | |
| 1213 | + | |
| 1214 | + | |
| 1215 | + | |
| 1216 | + | |
| 1217 | + | |
| 1218 | + | |
| 1219 | + | |
| 1220 | + | |
| 1221 | + | |
| 1222 | + | |
| 1223 | + | |
| 1224 | + | |
| 1225 | + | |
| 1226 | + | |
| 1227 | + | |
| 1228 | + | |
| 1229 | + | |
| 1230 | + | |
| 1231 | + | |
| 1232 | + | |
| 1233 | + | |
| 1234 | + | |
| 1235 | + | |
| 1236 | + | |
| 1237 | + | |
| 1238 | + | |
| 1239 | + | |
| 1240 | + | |
| 1241 | + | |
| 1242 | + | |
| 1243 | + | |
| 1244 | + | |
| 1245 | + | |
| 1246 | + | |
| 1247 | + | |
| 1248 | + | |
| 1249 | + | |
| 1250 | + | |
| 1251 | + | |
| 1252 | + | |
| 1253 | + | |
| 1254 | + | |
| 1255 | + | |
| 1256 | + | |
| 1257 | + | |
| 1258 | + | |
| 1259 | + | |
| 1260 | + | |
| 1261 | + | |
| 1262 | + | |
| 1263 | + | |
| 1264 | + | |
0 commit comments