Ce cos status redesign rfc#7
Open
perdasilva wants to merge 43 commits into
Open
Conversation
perdasilva
force-pushed
the
ce-cos-status-redesign-rfc
branch
from
c43d3f2 to
d138002
Compare
Proposes improvements to CE and COS status APIs: - Fix Progressing condition semantics (True/Succeeded → False/Succeeded) - Add Ready condition to CE for dedicated health signal - Add status.rollout field with type (Install/Upgrade/Reconfigure) and target bundle - Add per-phase status to COS for rollout progress visibility - Rename COS Available → Ready, convert Succeeded condition to succeededAt field - Stop mirroring COS conditions on CE — translate to CE-native conditions instead - Update print columns: CE gets Ready/Progressing/Version/Rollout/Target/Age, COS gets Revision/Lifecycle/Ready/Progressing/Age - Remove experimental activeRevisions from CE - Complete condition/reason audit with reference tables - 24 scenario examples covering happy paths, CE errors, COS errors, and edge cases Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Updates the Ready condition derivation model: during multi-revision transitions, Ready reflects the latest rolling-out COS's probe state rather than the old installed revision's state. This prevents misleading Ready=True when objects are being replaced phase by phase and the old COS's view is stale for handed-off objects. Also fixes scenarios 3.14/3.15/3.18 to surface COS retrying errors on the CE (collision, validation, probe failures), removes Migrated from COS doc comments, and consolidates ClusterObjectSetReasonRetrying to shared ReasonRetrying. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the Progressing True/False print column with the Progressing condition reason. Same column count, strictly more informative: Succeeded (done), RollingOut (active), Retrying (error, retrying), Blocked (terminal), InvalidConfiguration (fix spec), ProgressDeadlineExceeded (timed out). Users can now triage at a glance without kubectl describe: Ready=True + Succeeded = all good Ready=False + RollingOut = normal upgrade, wait Ready=False + Blocked = needs attention Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The column shows the Progressing condition's reason (Succeeded, RollingOut, Blocked, etc.), not a boolean. STATUS reads naturally with every value — "Status: Succeeded", "Status: Blocked". Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Keep PROGRESSING as the standard Kubernetes True/False boolean and add a REASON column showing the Progressing condition's reason. Applied to both CE and COS tables for consistency. CE columns: Ready, Progressing, Reason, Version, Rollout, Target, Age COS columns: Revision, Lifecycle, Ready, Progressing, Reason, Age Fixed column alignment across all 24 scenario examples. Fixed COS installed revision reason (Succeeded, not inherited from new revision). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Reformat all kubectl table examples with consistent 3-space column gaps and structure-aware alignment. Fixes misaligned empty columns (Rollout/Target) and ensures all tables read cleanly. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add a 5th phase state 'Transitioning' for old COS revisions during upgrades, covering the state where some objects in a phase have been adopted by a newer revision while others are still owned. This state is derived from boxcutter's ActionProgressed/ActionIdle mix. Add scenario 3.3a showing detailed phase-level view of BOTH revisions during an upgrade — COS-2 shows its rollout progress (Complete/Active/ Pending) while COS-1 shows the handoff progress (Complete/Transitioning). Updated phase derivation rules to distinguish new revision behavior (sequential rollout) from old revision behavior (handoff tracking). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The Reason column already shows 'Archived' for archived revisions, making the Lifecycle column redundant. COS print columns are now: Revision, Ready, Progressing, Reason, Age (5 columns, down from 7). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Make Ready and Progressing orthogonal, following the Kubernetes Deployment pattern where Available and Progressing don't echo each other's errors. Ready answers only "are managed resources healthy?" — uses Absent when nothing is deployed, ProbeFailure for probe issues, RollingOut during transitions. It never carries pipeline error detail. Progressing answers "what's happening?" — carries all pipeline and rollout error detail (resolution failures, config errors, pull errors, COS blocking errors). During upgrade scenarios with COS errors (collision, validation, blocked), Ready stays True/Succeeded when the installed revision is healthy. The error goes only in Progressing. Removes Failed and Reconciling from CE Ready reasons. Simplifies Ready to 4 values: Succeeded, Absent, ProbeFailure, RollingOut (+ Pending for Unknown). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Update Key UX point paragraphs in scenarios 3.5, 3.8, 3.14, 3.16 to match the orthogonal condition model — Ready no longer carries pipeline error detail, that's Progressing's job. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the catch-all Retrying reason with specific error categories so users can triage from kubectl get without kubectl describe: - ResolutionFailed: bundle not found / ambiguous - PullFailed: image pull error (auth, network, missing) - ValidationFailed: CE validation error (SA not found, etc.) - AuthorizationFailed: RBAC pre-authorization failed - ContentFailed: bundle content unsupported - PreflightFailed: CRD safety or other preflight check failed - ProbeFailure: COS probes failing during rollout (was RollingOut) Retrying is kept only for COS-level transient errors (collisions, validation) where the CE translates a COS Retrying state. Also fixes the probe failure gap: scenario 3.14 now shows ProbeFailure instead of RollingOut, making stuck upgrades distinguishable from healthy ones at a glance. Updated §1.1, §1.5, §1.6, §1.7, §4.1, §4.4, §4.5, all affected scenario yaml and print column tables. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Split the COS generic Retrying into two specific reasons for the most common debugging scenarios: - CollisionDetected: object ownership conflict with another controller - ValidationFailed: preflight/dry-run validation failure Retrying is kept for remaining transient errors (secret resolution, watch setup, engine errors). Updated §2.4, §2.6, §4.3, §4.4, §4.5, scenarios 3.15 and 3.18. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move ValidationFailed to shared in §4.1 (CE ✓, COS ✓) and remove duplicate from §4.3 COS-specific. CollisionDetected stays COS-specific. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Make Installed orthogonal with Progressing — it answers ONE question: "is a bundle installed?" Yes (Succeeded) or No (Absent). Removes Failed and Unknown from Installed. Error detail lives exclusively in the Progressing condition. This completes the orthogonality model: Ready (health), Installed (fact), Progressing (what's happening + errors). No duplication between any pair. Updated §1.7, §4.1, §4.4, §4.5, and all affected scenario yaml. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Recommend Kubernetes Events for key state transitions on both ClusterExtension and ClusterObjectSet. Events provide a time-series trail complementing conditions (current state). CE events: rollout started/completed/failed, reason changes, resolution/pull/auth/deadline errors. COS events: phase completion, probe failures, collisions, blocked, archived, succeeded. Includes guidance on event types (Normal vs Warning), deduplication, and transition-only emission to avoid event flooding. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Follow K8s convention: controllers should set conditions on first visit, even if Unknown, to signal awareness. COS now sets Ready=Unknown/Reconciling when errors prevent probe evaluation (secret immutability, phase content, preflight), and Ready=Unknown/Archived for archived revisions. Eliminates <none> from all COS print column examples. Updated §2.4, COS yaml in scenarios 3.16 and 3.17, and all COS print column tables. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add a Message column (priority=1, shown with -o wide) to both CE and COS print columns. Shows the Progressing condition's message inline, giving SREs the specific error detail without kubectl describe. Default table stays clean (7 columns for CE, 5 for COS). Wide view adds the error message for quick debugging. Updated §1.6 with both default and wide examples, §2.5, and implementation plan. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
AGE is always the last column in kubectl output (pods, deployments, nodes, etc.). Move MESSAGE before AGE in the wide view so AGE stays last. Updated §1.6 wide example, §1.6 and §2.5 column tables. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add kubectl get clusterextensions -o wide examples to all 18 error scenarios (3.5-3.23), showing the MESSAGE column with the Progressing condition's message truncated to ~70 chars. Messages are placed before AGE (last column per kubectl convention). Users can now see the full error detail inline without kubectl describe. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Ready=RollingOut message changed from "Revision X.X.X is rolling out" (duplicating Progressing) to "Managed resources are being updated" (Ready's own concern). Ready describes the health state, Progressing describes what's happening. No duplication. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…sionDetected More specific and self-explanatory reason names: - ImagePullFailed aligns with K8s terminology (ImagePullBackOff) - ObjectCollisionDetected clarifies it's about managed object ownership Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- ClusterExtensionRolloutStatus → ClusterExtensionOperationStatus - RolloutType → OperationType - status.rollout → status.operation - ROLLOUT print column → OPERATION - All yaml examples, prose references to the field, and impl plan Generic "rollout" as a concept word (phased rollout, active rollout) is preserved — it's standard Kubernetes terminology. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…xamples Both are standard metav1.Condition fields that SREs and tooling rely on: - observedGeneration: whether the condition reflects the current spec - lastTransitionTime: when the condition last changed CE conditions use observedGeneration matching the CE's generation (1 for first install, 2 for upgrades/reconfigs). COS conditions always use 1 (COS spec is immutable). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
upgrade scenarios During upgrades, conditions that haven't changed since the original install should retain their original observedGeneration and lastTransitionTime: - Installed=True/Succeeded: observedGeneration=1 (set at first install, not re-evaluated), lastTransitionTime=original install - Ready=True/Succeeded (when unchanged): observedGeneration=2 (controller reconciled gen 2), lastTransitionTime=original install (condition didn't transition) - Progressing: observedGeneration=2, lastTransitionTime=upgrade time (condition changed) This accurately models how Kubernetes conditions behave — observedGeneration tracks which generation the controller processed, lastTransitionTime tracks when the condition status last changed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ContentFailed implies broken/corrupt content. UnsupportedContent accurately describes the issue: the bundle uses features OLM doesn't support yet (apiServiceDefinitions, unsupported install modes). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Consistent 3-space column gaps across all scenario examples. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Installed=False/Absent was carrying the validation error message (duplicating Progressing). Changed to "No bundle installed" — Installed only reports the binary fact, error detail is exclusively in Progressing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…osed Current COS controller uses phase index and raw ObjectResult.String(): "revision object collisions in phase 2\nObject Deployment..." Proposed format uses phase name and cleaner single-line structure: "Object collision in phase \"roles\": Deployment.apps/v1 my-ns/deploy owned by ClusterObjectSet/other-ext-1" Added a callout in scenario 3.15 documenting the current format and the proposed improvement. Updated all collision messages in CE, COS, and wide examples to use the proposed format. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The boxcutter ObjectResultCollision.String() produces a verbose multi-line dump including raw OwnerReference.String() and probe details. The COS controller wraps it with "revision object collisions in phase %d\n%s". Document the actual current output and propose constructing a cleaner message using the phase name (from spec) and the structured ConflictingOwner() accessor instead of raw String(). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
CE conditions now use CE-native terminology: - RollingOut → Deploying (CE deploys, COS rolls out phases) - PreflightFailed → SafetyCheckFailed (user-friendly, not boxcutter jargon) COS keeps RollingOut (the COS IS the thing rolling out phases). §4.1 separates ReasonDeploying (CE-only) from ReasonRollingOut (COS-only). Updated §1.1, §1.2, §1.5, §1.6, §1.7, §4.1, §4.4, §4.5, all CE scenario yaml and print columns. COS unchanged. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
and remove duplicate ValidationFailed row from §4.5 The RollingOut→Deploying rename was CE-only but accidentally changed COS print column tables too. COS uses RollingOut (COS-native term). Also removed a duplicate ValidationFailed entry in §4.5 reason semantics table. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Consistent 3-space column gaps across all CE (default + wide) and COS scenario examples. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Archived revisions have their objects torn down — readiness is not applicable. Ready shows <none> instead of Unknown/Archived. The Archived reason on Progressing already conveys the lifecycle state. Ready is always set on active revisions (including Unknown/Reconciling for pre-phase errors), but not on archived ones. Updated §2.4, §2.5, §2.6 doc comment, §4.3, §4.4, and print column examples. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…sing translation rule - Fix 14 missing closing backticks on -o wide code blocks (broken markdown rendering) - Fix §3.5 key UX point: Retrying → ResolutionFailed to match the actual condition - Fix §3.4 COS: Retrying → RollingOut for consistency with other in-progress scenarios - Fix §3.11: populate operation field (SA validation is post-resolution, target is known) - Add §1.5 translation rule for COS Ready=Unknown/Reconciling → CE Deploying - Resolve Open Question 4: no Downgrade type now, can be added non-breakingly later - Fix §5.3 typo: Rollout → Operation to match print column name Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ions Pod, Node, and other core types use STATUS as the column name for "what's going on with this thing." REASON is a condition field name, not a user-facing concept. Rename for consistency with the Kubernetes ecosystem while keeping the same JSONPath source (.reason field). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Document that Available is intentionally not used now but reserved for a potential future stability signal (analogous to Deployment's Available = Ready for minReadySeconds). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- §3.1a: Initial state before first reconcile (Ready=Unknown/Pending) - §3.4a: Failed reconfiguration with probe failure (Operation=Reconfigure) - §3.14a: Probe failure during first install (Installed=False, no fallback) These fill gaps in the §4.4 condition×reason matrix, ensuring every defined state has at least one concrete example in the scenario catalog. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New order: NAME, VERSION, READY, PROGRESSING, STATUS, OPERATION, TARGET, AGE Groups columns by purpose: - Identity (NAME + VERSION): what is this - Health (READY + PROGRESSING + STATUS): is it okay and why - Activity (OPERATION + TARGET): what's happening (empty in steady state) Reordered all 49 CE tables across the scenario catalog. COS tables unchanged. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Shows the unique Ready=False + Progressing=False/Succeeded combination that identifies drift: the system is broken but no rollout is stuck. The COS self-heals within the current revision — no new operation is created because drift recovery is not a revision-level rollout. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The COS is actively re-creating the missing resource, so Progressing=True is correct. The absence of operation (no new revision) distinguishes drift recovery from a stuck upgrade. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…dationFailed CE's ValidationFailed (ServiceAccount not found) and COS's ValidationFailed (admission webhook rejection) are different errors. Split them: - CE: PreflightFailed — CE-level preflight checks (SA not found, etc.) - COS: ValidationFailed — dry-run/admission validation (webhook rejection) This gives each layer a specific, self-describing reason without shared-constant ambiguity. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Show the current CE and COS print column output so reviewers can see the before/after contrast. The CE example highlights the Progressing=True bug (both healthy and unhealthy show True). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
perdasilva
force-pushed
the
ce-cos-status-redesign-rfc
branch
from
473a2c9 to
fe7ef4e
Compare
Add "Current output" blocks showing today's print columns alongside the proposed output for representative scenarios: - §3.1 Steady state: Progressing=True bug most visible - §3.3 Upgrade: indistinguishable from steady state, no target - §3.3a Phase view: COS lacks revision number and phase detail - §3.6 Bundle not found: error completely invisible - §3.14 Probe failure: COS error not surfaced on CE - §3.16 Blocked: current does show Progressing=False but no reason Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposes improvements to CE and COS status APIs:
Description
Reviewer Checklist