fix(cicd): allow skipped Deploy-to-Morpheus-P-Node to satisfy C-Node deploy needs#715
Merged
Merged
Conversation
…deploy needs
Root cause of the first v7 test-branch run (GH Actions run 24796856145):
the drain job ran and deregistered the dev C-Node from both NLB target
groups, then Deploy-to-Morpheus-C-Node was silently skipped because its
implicit success() guard propagated the skip from Deploy-to-Morpheus-P-Node
(which is main-only and intentionally skips on test pushes). That left
dev with the old C-Node task running in ECS but with no load-balancer
membership until we manually re-registered it.
Fix:
- Add an explicit `if` to Deploy-to-Morpheus-C-Node that:
- Requires GHCR build + drain success.
- Accepts Deploy-to-Morpheus-P-Node.result in {success, skipped}.
- Wraps in `!cancelled()` so manual cancels still short-circuit.
- Rewrite the inline comment that previously (incorrectly) claimed
skipped jobs are treated as successful for dependency resolution;
they are not.
No change to the deploy logic itself, the drain job, or any other
workflow sequencing. This is a pure `needs` / guard correctness fix.
Made-with: Cursor
nomadicrogue
added a commit
that referenced
this pull request
Apr 22, 2026
…deploy needs (#716) ## Summary Promotes the #715 hotfix from `dev` → `test` so we can retry the v7 test deploy and actually exercise the new drain / hold / re-register sequence end-to-end. ## What's in this PR Single-commit delta ahead of `test`: - **`2b14c58` — fix(cicd): allow skipped Deploy-to-Morpheus-P-Node to satisfy C-Node deploy needs** (merged via #715) Everything else in `dev` is already on `test` from #714. ## Background First v7 test run ([Actions run #1447 / run id 24796856145](https://github.com/MorpheusAIs/Morpheus-Lumerin-Node/actions/runs/24796856145)) revealed a correctness gap in the workflow from #713/#714: - `Drain-Morpheus-C-Node` ran and successfully drained the dev C-Node targets from both NLB target groups. - `Deploy-to-Morpheus-P-Node` was correctly skipped (main-only). - `Deploy-to-Morpheus-C-Node` was **silently skipped** because the implicit `success()` guard propagated the skip from the P-Node job — GitHub Actions treats skipped `needs` as non-success, contrary to the inline comment in the previous PR. - Dev C-Node remained functional in ECS but without LB membership until manually re-registered. ## Fix (this PR carries) Explicit `if` guard on `Deploy-to-Morpheus-C-Node` that: - Requires real success from `GHCR-Build-and-Push` and `Drain-Morpheus-C-Node`. - Accepts either `success` or `skipped` as the outcome for `Deploy-to-Morpheus-P-Node`. - Wraps everything in `!cancelled()` to short-circuit on manual cancel. The misleading comment is replaced with an accurate explanation referencing this incident. ## Expected behavior after merge to test On push to `test` the sequence becomes: 1. Build + test + GHCR push (~10–20 min). 2. `Drain-Morpheus-C-Node` ✅ deregisters the existing dev C-Node targets. 3. `Deploy-to-Morpheus-P-Node` skipped (expected). 4. `Deploy-to-Morpheus-C-Node` ✅ **runs this time** because the new `if` accepts the P-Node skip: - `update-service` with `--health-check-grace-period-seconds 600`. - Poll new task ENI IP. - Deregister IP from TGs. - Sleep 90s (`cnode_rehydration_wait_secs`). - Re-register IP, `wait target-in-service`. - Public `/healthcheck` version match on `router.dev.mor.org:8082`. 5. `Deploy-to-Titan` + `Deploy-TEE-SecretVM-test` run in parallel to the deploy sequence (unchanged). On dev the rehydration hold is effectively a no-op because EFS-backed BadgerDB is persistent there — perfect for a first validation of the workflow plumbing without depending on rehydration correctness. ## Review focus - The new `if` block on `Deploy-to-Morpheus-C-Node` (lines ~1298–1320 of `build.yml`). - Reasoning captured in the inline comment. - No changes to deploy logic, drain logic, or sequencing — only the guard. ## Test plan (after you merge to test) 1. Merge triggers the deploy workflow against dev. 2. Confirm `Deploy-to-Morpheus-Consumer` enters `in_progress` (not skipped) after the drain completes. 3. Watch the controlled-traffic sequence: - New task gets an ENI IP. - `deregister-targets` runs. - 90s hold. - `register-targets` + `wait target-in-service` succeed. - `/healthcheck` version-match passes on the new image tag. 4. If green, proceed with `test` → `main` for the first prd exercise (existing open PR). ## Related - Previous merges on the new CICD flow: #713, #714, #715. - Companion IAM + planning doc already applied in `Morpheus-Infra`. Made with [Cursor](https://cursor.com)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Hotfix for the first v7
test-branch deployment attempt (Actions run #1447 / run id 24796856145).The previous PR (#714 → #713) introduced a new job sequence:
On push to
test,Deploy-to-Morpheus-P-Nodeis correctly skipped (its ownifrestricts it tomain— no dev P-Node exists). That skip then propagated ontoDeploy-to-Morpheus-C-Nodethrough the implicitsuccess()guard that every job has when no explicitiftolerates a skipped dependency. Net effect:Drain-Morpheus-C-Node✅ removed the running dev C-Node task from both NLB target groups.Deploy-to-Morpheus-C-Node❌ silently skipped — no new task registered, no re-register of the existing task.router.dev.mor.org:8082/:8545) stopped serving traffic until we manually re-registered the old task to the TGs.What this PR changes
.github/workflows/build.yml,Deploy-to-Morpheus-C-Nodejob only:Explicit
ifguard that tolerates the P-Node skipsuccessORskippedfor the P-Node dependency.!cancelled()short-circuits the job if someone cancels the run, so we don't try to redeploy a half-cancelled sequence.Comment rewrite
The previous inline comment claimed skipped jobs are treated as successful for dependency resolution. That's the opposite of how GitHub Actions actually behaves — the note is replaced with an accurate explanation and a reference to this incident so future-us (or other maintainers) won't step on the same rake.
What this PR does NOT change
ifgating.This is a pure guard-correctness fix.
Test plan
yaml.safe_load).needsoutcomes:test: P-Nodeskipped, drain + GHCRsuccess→ C-Node runs. ✅main: all threesuccess→ C-Node runs. ✅dev, promote totest, confirmDeploy-to-Morpheus-Consumeractually runs this time and completes the drain → hold → re-register → /healthcheck cycle end-to-end.test→mainfor first prd exercise.Related
Morpheus-Infra.Made with Cursor