Skip to content

Add Applier for upgrading workloads in place#5410

Merged
JAORMX merged 1 commit into
mainfrom
upgrade-4-applier
Jun 2, 2026
Merged

Add Applier for upgrading workloads in place#5410
JAORMX merged 1 commit into
mainfrom
upgrade-4-applier

Conversation

@JAORMX
Copy link
Copy Markdown
Collaborator

@JAORMX JAORMX commented Jun 1, 2026

Summary

Detecting an available upgrade is only useful if users can apply it while keeping their configuration. This adds the apply core that the CLI and API drive in the next PR. Part of RFC THV-0068, local scope.

  • Add upgrade.Applier: it reloads the workload's saved config, re-runs the check on fresh state (so a stale result can never drive an apply — closes the TOCTOU window), resolves the candidate from the registry, and rebuilds the run config preserving the full user configuration (auth, authz, audit, telemetry, tools filters, volumes, secrets, ports, permission profile, …), changing only the image, merged env/secrets, and re-resolved registry URLs. Newly required env vars surface through the injected validator.
  • Security-critical ordering: the candidate image is verified and pulled (and the policy gate runs) before the destructive stop/delete/start, so a missing or unverifiable image leaves the running workload untouched — there is no rollback once UpdateWorkload begins. Verification uses the same path as thv run.
  • Apply awaits UpdateWorkload's completion so the upgraded config is durably persisted before returning (both CLI and API are synchronous).

Type of change

  • New feature

Test plan

  • Unit tests (task test) — no-op when up-to-date; resolve/verify/pull failure leaves the workload untouched (UpdateWorkload never called); ordering assertion (verify+pull before update, completion awaited); the exact gated+pulled config is the deployed one; full-config preservation; copy-before-mutate; completion-failure surfaces as an error.
  • Linting (task lint-fix)

Changes

File Change
pkg/workloads/upgrade/applier.go Applier, Apply, config merge/preservation
pkg/workloads/upgrade/applier_test.go Unit tests (manager + retriever seams mocked)

Does this introduce a user-facing change?

No — internal apply path. The thv upgrade apply command and POST .../upgrade endpoint that drive it land in PR 5.

Large PR Justification

This PR is ~1,015 lines, but 576 are unit tests and 439 are a single source file (applier.go).

  • The apply path must be atomic to be reviewable. Apply is one sequence — load → re-check → resolve+verify → rebuild (preserving every user-owned field) → pull → recreate → await completion. Splitting it would create half-built intermediate states that neither compile meaningfully nor convey the security-critical ordering that is the whole point of the change.
  • This is the highest-risk code in the feature, so its tests are intentionally exhaustive (the 576 lines cover every failure point, the verify-before-destroy ordering, full-config preservation, and copy-before-mutate). They must ship with the code under review.
  • No generated code; the size is genuinely source+tests for one cohesive unit.

Special notes for reviewers

PR 4 of 6 in the RFC THV-0068 stack; based on #5409. This is the highest-risk change — the destructive recreate path. The verify-then-pull-before-destroy ordering and full-config preservation were the focus of the security/Go review (an earlier draft silently dropped auth/audit/tools-filter config on upgrade; now every user-owned field is preserved). On Kubernetes the byte-level pull is delegated to the kubelet, but verification + policy gate still precede destruction; local runtime is the RFC's scope.

🤖 Generated with Claude Code

@JAORMX JAORMX mentioned this pull request Jun 1, 2026
3 tasks
@github-actions github-actions Bot added the size/L Large PR: 600-999 lines changed label Jun 1, 2026
@JAORMX JAORMX force-pushed the upgrade-3-cli-check branch from 2af4157 to 53115aa Compare June 1, 2026 09:31
@JAORMX JAORMX force-pushed the upgrade-4-applier branch from 391531e to ba2195c Compare June 1, 2026 09:32
@github-actions github-actions Bot added size/L Large PR: 600-999 lines changed and removed size/L Large PR: 600-999 lines changed labels Jun 1, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented Jun 1, 2026

Codecov Report

❌ Patch coverage is 89.86486% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.83%. Comparing base (a49b4fa) to head (01fbc6f).

Files with missing lines Patch % Lines
pkg/workloads/upgrade/applier.go 89.86% 8 Missing and 7 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5410      +/-   ##
==========================================
+ Coverage   68.75%   68.83%   +0.07%     
==========================================
  Files         634      635       +1     
  Lines       64183    64331     +148     
==========================================
+ Hits        44132    44281     +149     
+ Misses      16782    16774       -8     
- Partials     3269     3276       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@JAORMX JAORMX force-pushed the upgrade-4-applier branch from ba2195c to a93ff85 Compare June 1, 2026 10:12
@github-actions github-actions Bot added size/XL Extra large PR: 1000+ lines changed and removed size/L Large PR: 600-999 lines changed labels Jun 1, 2026
Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Large PR Detected

This PR exceeds 1000 lines of changes and requires justification before it can be reviewed.

How to unblock this PR:

Add a section to your PR description with the following format:

## Large PR Justification

[Explain why this PR must be large, such as:]
- Generated code that cannot be split
- Large refactoring that must be atomic
- Multiple related changes that would break if separated
- Migration or data transformation

Alternative:

Consider splitting this PR into smaller, focused changes (< 1000 lines each) for easier review and reduced risk.

See our Contributing Guidelines for more details.


This review will be automatically dismissed once you add the justification section.

@JAORMX JAORMX marked this pull request as ready for review June 1, 2026 10:21
@JAORMX JAORMX requested review from amirejaz and lujunsan as code owners June 1, 2026 10:21
@github-actions github-actions Bot added size/XL Extra large PR: 1000+ lines changed and removed size/XL Extra large PR: 1000+ lines changed labels Jun 1, 2026
@github-actions github-actions Bot dismissed their stale review June 1, 2026 10:26

Large PR justification has been provided. Thank you!

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 1, 2026

✅ Large PR justification has been provided. The size review has been dismissed and this PR can now proceed with normal review.

@JAORMX JAORMX force-pushed the upgrade-3-cli-check branch from 53115aa to a6b9a1d Compare June 1, 2026 13:16
@JAORMX JAORMX force-pushed the upgrade-4-applier branch from a93ff85 to 4ad3554 Compare June 1, 2026 13:16
@github-actions github-actions Bot added size/XL Extra large PR: 1000+ lines changed and removed size/XL Extra large PR: 1000+ lines changed labels Jun 1, 2026
@JAORMX JAORMX force-pushed the upgrade-3-cli-check branch from a6b9a1d to 1029c5c Compare June 1, 2026 13:27
@JAORMX JAORMX force-pushed the upgrade-4-applier branch from 4ad3554 to a87510f Compare June 1, 2026 13:27
@github-actions github-actions Bot removed the size/XL Extra large PR: 1000+ lines changed label Jun 1, 2026
@github-actions github-actions Bot added the size/XL Extra large PR: 1000+ lines changed label Jun 1, 2026
Copy link
Copy Markdown
Contributor

@jhrozek jhrozek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two notes from re-reviewing the Applier (the rest of my findings I'm tracking elsewhere, or they belong to the stacked check PRs). Both are non-blocking — the verify-then-pull ordering and TOCTOU guard look solid.

Comment thread pkg/workloads/upgrade/applier.go
Comment thread pkg/workloads/upgrade/applier.go Outdated
Base automatically changed from upgrade-3-cli-check to main June 2, 2026 12:01
@github-actions github-actions Bot added size/XL Extra large PR: 1000+ lines changed and removed size/XL Extra large PR: 1000+ lines changed labels Jun 2, 2026
Detecting an available upgrade is only useful if users can apply it
while keeping their configuration. Add the apply path that the CLI and
API will drive.

Add upgrade.Applier: it reloads the workload's saved config, re-runs the
check on fresh state (so a stale result can never drive an apply),
resolves the candidate from the registry, and rebuilds the run config
preserving the full user configuration — auth, authz, audit, telemetry,
tools filters, volumes, secrets, ports, permission profile, and more —
changing only the image, merged env/secrets, and re-resolved registry
URLs. New required env vars surface through the injected validator.

Crucially, the candidate image is verified and pulled (and the policy
gate runs) before the destructive stop/delete/start, so a missing or
unverifiable image leaves the running workload untouched — there is no
rollback once UpdateWorkload begins. Verification uses the same path as
thv run.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@JAORMX JAORMX force-pushed the upgrade-4-applier branch from a87510f to 01fbc6f Compare June 2, 2026 12:20
@github-actions github-actions Bot added size/XL Extra large PR: 1000+ lines changed and removed size/XL Extra large PR: 1000+ lines changed labels Jun 2, 2026
@JAORMX JAORMX merged commit 0181749 into main Jun 2, 2026
49 checks passed
@JAORMX JAORMX deleted the upgrade-4-applier branch June 2, 2026 13:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/XL Extra large PR: 1000+ lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants