fix(release): harden npm-publish pipeline + document release SOP (#744)

* fix(release): harden npm-publish pipeline + document release SOP

Three real failure modes observed during the v5.2.5/5.2.6 releases, each
fixed and documented so the next release does not re-discover them.

## npm-publish.yml

### Race: publish_legacy_aiox_core ran in parallel with publish

The legacy compat wrapper `aiox-core@X.Y.Z` declares
`@aiox-squads/core@X.Y.Z` as a runtime dependency. The smoke test step
runs `npx --yes aiox-core@X.Y.Z --version`, which transitively installs
the scoped package. With both jobs in parallel, `npx` raced against npm
CDN propagation of the scoped package and timed out after 90s — making
the entire job appear failed even though the publish step inside it
had already printed "✅ Published aiox-core@X.Y.Z".

Fix: `publish_legacy_aiox_core` now `needs: [test, build, publish]`,
serializing the legacy publish behind the scoped publish.

### Smoke test legacy npx: timeout too short, no diagnostics

Bumped retry budget 6×15s (90s) → 16×15s (240s). Added a second
visibility check for the scoped dependency before invoking `npx` so we
distinguish "compat not visible" from "scoped dep not visible". Captured
last `npx` stderr in the timeout message instead of failing silently.

### Notify: aggregated bool hid which step failed

Replaced with a structured summary that distinguishes hard fail (scoped
publish) from soft warnings (workspace, legacy, smoke). Only the scoped
publish is treated as release-blocking; the rest are informational and
can be republished out-of-band.

## installer-smoke-matrix.yml

### Windows path mangling in node -e via Git Bash

`${{ github.workspace }}` on a Windows runner expands to
`D:\a\aiox-core\aiox-core`. Interpolated into a `node -e "..."` script
running on Git Bash, the backslashes were interpreted as escape
sequences (`\a` → `a`), producing `D:aaiox-coreaiox-core` and
`Cannot find module ...`.

Fix: pass workspace path via `WORKSPACE_DIR` env var, read inside the
script as `process.env.WORKSPACE_DIR`. Applied to all 3 smoke install
steps.

## docs/guides/release-procedure.md

New SOP capturing the entire release pipeline:
- Pre-flight checks (lint, tests, tokens)
- 4-site version bump coordination (root, compat, installer, lock)
- Two-system branch protection bypass dance (modern ruleset +
  legacy branch protection both enforce CODEOWNERS independently;
  `gh pr merge --admin` bypasses neither in isolation)
- Atomic bypass→merge→restore as one shell block
- Tag-driven publish trigger
- Post-publish verification (registry sanity + artifact inspection)
- Known CI quirks with mitigations
- Rollback procedure (deprecate, don't unpublish)
- Failure modes seen in production with root causes

The SOP cites real PR numbers and incidents so future operators know
why each step exists.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: address CodeRabbit feedback on PR #744

Two real bugs caught:

## notify step printed "Successfully published" for skipped scoped publish

The previous condition gated the success block on
`PUBLISH_RESULT = success OR skipped` AND pre-seeded the PUBLISHED array
with `@aiox-squads/core@${VERSION}` unconditionally. Result: a workflow
run where `publish` was skipped (e.g. manual dispatch excluding `core`)
would falsely report the scoped package as published.

Fix: PUBLISHED is now built only from jobs that actually succeeded.
A separate branch handles the all-skipped case explicitly. The
success message only renders when something was actually published.

## SOP bypass dance: snapshot reuse could leave protections relaxed

The documented procedure piped the raw `gh api .../rulesets/...`
output back into `gh api -X PUT`. The raw response includes read-only
fields (`id`, `node_id`, `created_at`, `updated_at`, `_links`, `url`)
that GitHub rejects with HTTP 422 on PUT — meaning the restore could
silently fail and leave production in a relaxed state.

Two fixes layered for safety:

1. Snapshot is now separated from restore payload. Raw snapshot stays
   on disk only as a diff baseline. Sanitized restore payload is
   derived via `jq '{name, target, enforcement, conditions,
   bypass_actors, rules}'` — only writable fields.

2. The bypass→merge→restore sequence is wrapped in `set -e` + a
   `trap 'restore_protections' EXIT`. The trap fires on ANY exit
   (success, failure, signal) and idempotently re-applies the
   sanitized restore payloads. If a restore step itself fails, the
   trap emits `::error::` annotations so the failure surfaces in the
   GitHub Actions UI.

The previous documentation worked in practice because `gh api --input`
tolerates read-only fields in some paths, but that behavior is not
guaranteed. The new procedure is defensively correct under network
failures, manual interrupts, and edge-case API rejections.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: rafaelscosta

.github/workflows/installer-smoke-matrix.yml

@@ -135,13 +135,19 @@ jobs:
 
       - name: Smoke install (empty target, no ancestors)
         shell: bash
+        env:
+          # Pass workspace via env to avoid backslash escape mangling on Windows
+          # (interpolating ${{ github.workspace }} inside node -e on Git Bash for
+          # Windows turns `D:\a\aiox-core\aiox-core` into `D:aaiox-coreaiox-core`
+          # because \a, \aiox are interpreted as escape sequences).
+          WORKSPACE_DIR: ${{ github.workspace }}
         run: |
           set -e
           WORKDIR=$(mktemp -d)
           cd "$WORKDIR"
           echo "Smoke target: $WORKDIR"
           node -e "
-            const proSetup = require('${{ github.workspace }}/packages/installer/src/wizard/pro-setup');
+            const proSetup = require(process.env.WORKSPACE_DIR + '/packages/installer/src/wizard/pro-setup');
             (async () => {
               const fs = require('fs');
               const path = require('path');
@@ -167,6 +173,8 @@ jobs:
 
       - name: Smoke install (target nested inside an ancestor workspace)
         shell: bash
+        env:
+          WORKSPACE_DIR: ${{ github.workspace }}
         run: |
           set -e
           WS_ROOT=$(mktemp -d)
@@ -177,7 +185,7 @@ jobs:
           cd "$WS_ROOT/projects/aiox-install"
           echo "Smoke target: $(pwd)"
           node -e "
-            const proSetup = require('${{ github.workspace }}/packages/installer/src/wizard/pro-setup');
+            const proSetup = require(process.env.WORKSPACE_DIR + '/packages/installer/src/wizard/pro-setup');
             (async () => {
               const fs = require('fs');
               const path = require('path');
@@ -209,6 +217,8 @@ jobs:
 
       - name: Smoke install (target nested under plain package.json ancestor)
         shell: bash
+        env:
+          WORKSPACE_DIR: ${{ github.workspace }}
         run: |
           set -e
           PARENT_ROOT=$(mktemp -d)
@@ -219,7 +229,7 @@ jobs:
           cd "$PARENT_ROOT/subdir"
           echo "Smoke target: $(pwd)"
           node -e "
-            const proSetup = require('${{ github.workspace }}/packages/installer/src/wizard/pro-setup');
+            const proSetup = require(process.env.WORKSPACE_DIR + '/packages/installer/src/wizard/pro-setup');
             (async () => {
               const fs = require('fs');
               const path = require('path');

.github/workflows/npm-publish.yml

@@ -354,8 +354,13 @@ jobs:
           exit 1
 
   publish_legacy_aiox_core:
-    needs: [test, build]
-    if: ${{ contains(format(',{0},', needs.build.outputs.packages), ',aiox-core,') }}
+    # MUST wait for `publish` because compat/aiox-core declares
+    # `@aiox-squads/core@<version>` as a runtime dependency. If this job runs
+    # in parallel, the legacy smoke test races against npm propagation of the
+    # scoped package and times out — making the workflow appear failed even
+    # though both packages were published successfully.
+    needs: [test, build, publish]
+    if: ${{ contains(format(',{0},', needs.build.outputs.packages), ',aiox-core,') && (needs.publish.result == 'success' || needs.publish.result == 'skipped') }}
     runs-on: ubuntu-latest
     outputs:
       published: ${{ steps.should-publish.outputs.should_publish }}
@@ -439,21 +444,35 @@ jobs:
         if: steps.should-publish.outputs.should_publish == 'true'
         env:
           PKG_SPEC: aiox-core@${{ steps.pkg.outputs.version }}
+          SCOPED_SPEC: '@aiox-squads/core@${{ steps.pkg.outputs.version }}'
         run: |
-          for i in 1 2 3 4 5 6; do
+          # Two phases: (1) wait for both packages to be visible in npm
+          # registry, (2) actually exercise npx. Total budget: 16 × 15s = 240s
+          # to accommodate slow CDN propagation (the legacy compat wraps the
+          # scoped package, so BOTH must be resolvable before npx works).
+          last_err=""
+          for i in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16; do
             sleep 15
-            if npm view "${PKG_SPEC}" version > /dev/null 2>&1; then
-              SMOKE_DIR=$(mktemp -d)
-              if (cd "${SMOKE_DIR}" && npx --yes "${PKG_SPEC}" --version >/dev/null 2>&1); then
-                rm -rf "${SMOKE_DIR}"
-                echo "✅ Smoke test passed: npx --yes ${PKG_SPEC} --version works"
-                exit 0
-              fi
+            if ! npm view "${PKG_SPEC}" version > /dev/null 2>&1; then
+              echo "⏳ Waiting for ${PKG_SPEC} propagation... (attempt $i/16)"
+              continue
+            fi
+            if ! npm view "${SCOPED_SPEC}" version > /dev/null 2>&1; then
+              echo "⏳ ${PKG_SPEC} visible but ${SCOPED_SPEC} not yet... (attempt $i/16)"
+              continue
+            fi
+            SMOKE_DIR=$(mktemp -d)
+            if last_err=$(cd "${SMOKE_DIR}" && npx --yes "${PKG_SPEC}" --version 2>&1); then
               rm -rf "${SMOKE_DIR}"
+              echo "✅ Smoke test passed: npx --yes ${PKG_SPEC} --version works"
+              exit 0
             fi
-            echo "⏳ Waiting for npm propagation... (attempt $i/6)"
+            rm -rf "${SMOKE_DIR}"
+            echo "⏳ npx attempt failed, retrying... (attempt $i/16)"
+            echo "    last npx stderr: $(echo "$last_err" | tail -3)"
           done
-          echo "❌ Smoke test timeout for ${PKG_SPEC}"
+          echo "❌ Smoke test timeout for ${PKG_SPEC} after 240s"
+          echo "   Last npx stderr: $last_err"
           exit 1
 
   smoke_test_exports:
@@ -519,48 +538,74 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Notify completion
+        env:
+          VERSION: ${{ needs.build.outputs.version }}
+          PUBLISH_RESULT: ${{ needs.publish.result }}
+          WORKSPACE_RESULT: ${{ needs.publish_workspace_packages.result }}
+          LEGACY_RESULT: ${{ needs.publish_legacy_aiox_core.result }}
+          LEGACY_PUBLISHED: ${{ needs.publish_legacy_aiox_core.outputs.published }}
+          EXPORTS_RESULT: ${{ needs.smoke_test_exports.result }}
         run: |
-          if [ "${{ needs.publish.result }}" = "failure" ] || [ "${{ needs.publish_workspace_packages.result }}" = "failure" ] || [ "${{ needs.publish_legacy_aiox_core.result }}" = "failure" ] || [ "${{ needs.smoke_test_exports.result }}" = "failure" ]; then
-            echo "❌ Publishing failed"
+          echo "=== Release pipeline summary for v${VERSION} ==="
+          echo ""
+          echo "Job results:"
+          printf "  %-32s %s\n" "publish (@aiox-squads/core)"   "$PUBLISH_RESULT"
+          printf "  %-32s %s\n" "publish_workspace_packages"    "$WORKSPACE_RESULT"
+          printf "  %-32s %s\n" "publish_legacy_aiox_core"      "$LEGACY_RESULT (published=$LEGACY_PUBLISHED)"
+          printf "  %-32s %s\n" "smoke_test_exports"            "$EXPORTS_RESULT"
+          echo ""
+
+          # Hard failure: the scoped @aiox-squads/core publish IS THE release.
+          # Workspace packages and legacy compat wrapper are advisory — they
+          # can be republished out-of-band. Smoke tests are validation; if they
+          # fail but publish succeeded, log a regression warning but don't
+          # break the release signal (the artifact is still in the registry).
+          if [ "$PUBLISH_RESULT" = "failure" ] || [ "$PUBLISH_RESULT" = "cancelled" ]; then
+            echo "❌ HARD FAIL: scoped @aiox-squads/core@${VERSION} did not publish."
+            echo "   The release did NOT reach npmjs.org."
             exit 1
           fi
 
-          if [ "${{ needs.publish.result }}" = "cancelled" ] || [ "${{ needs.publish_workspace_packages.result }}" = "cancelled" ] || [ "${{ needs.publish_legacy_aiox_core.result }}" = "cancelled" ] || [ "${{ needs.smoke_test_exports.result }}" = "cancelled" ]; then
-            echo "❌ Publishing was cancelled"
-            exit 1
+          # Soft warnings on partial failures.
+          WARNINGS=()
+          if [ "$WORKSPACE_RESULT" = "failure" ] || [ "$WORKSPACE_RESULT" = "cancelled" ]; then
+            WARNINGS+=("workspace packages did not publish — investigate publish_workspace_packages")
           fi
-
-          PUBLISHED_PACKAGES=""
-
-          if [ "${{ needs.publish.result }}" = "success" ]; then
-            PUBLISHED_PACKAGES="@aiox-squads/core"
+          if [ "$LEGACY_RESULT" = "failure" ] || [ "$LEGACY_RESULT" = "cancelled" ]; then
+            WARNINGS+=("legacy aiox-core wrapper did not publish — check NPM_TOKEN with owner access to legacy package")
           fi
-
-          if [ "${{ needs.publish_workspace_packages.result }}" = "success" ]; then
-            if [ -n "$PUBLISHED_PACKAGES" ]; then
-              PUBLISHED_PACKAGES="$PUBLISHED_PACKAGES, selected @aiox-squads workspace packages"
-            else
-              PUBLISHED_PACKAGES="selected @aiox-squads workspace packages"
-            fi
+          if [ "$EXPORTS_RESULT" = "failure" ]; then
+            WARNINGS+=("smoke_test_exports failed — possible regression of issue #734 (ERR_PACKAGE_PATH_NOT_EXPORTED)")
           fi
 
-          if [ "${{ needs.publish_legacy_aiox_core.outputs.published }}" = "true" ]; then
-            if [ -n "$PUBLISHED_PACKAGES" ]; then
-              PUBLISHED_PACKAGES="$PUBLISHED_PACKAGES, aiox-core legacy wrapper"
-            else
-              PUBLISHED_PACKAGES="aiox-core legacy wrapper"
-            fi
+          PUBLISHED=()
+          if [ "$PUBLISH_RESULT" = "success" ]; then
+            PUBLISHED+=("@aiox-squads/core@${VERSION}")
+          fi
+          if [ "$WORKSPACE_RESULT" = "success" ]; then
+            PUBLISHED+=("selected @aiox-squads workspace packages")
+          fi
+          if [ "$LEGACY_PUBLISHED" = "true" ]; then
+            PUBLISHED+=("aiox-core@${VERSION} (legacy wrapper)")
           fi
 
-          if [ -n "$PUBLISHED_PACKAGES" ]; then
-            echo "🎉 Successfully published to NPM:"
-            echo "   Packages: $PUBLISHED_PACKAGES"
-            echo "   Registry: ${{ env.NPM_REGISTRY }}"
-          elif [ "${{ needs.publish.result }}" = "skipped" ] && [ "${{ needs.publish_workspace_packages.result }}" = "skipped" ] && [ "${{ needs.publish_legacy_aiox_core.result }}" = "skipped" ]; then
-            echo "⏭️ No packages needed publishing (already up-to-date)"
+          if [ ${#PUBLISHED[@]} -gt 0 ]; then
+            echo "🎉 Successfully published to npmjs.org:"
+            for pkg in "${PUBLISHED[@]}"; do
+              echo "   - $pkg"
+            done
+          elif [ "$PUBLISH_RESULT" = "skipped" ] && [ "$WORKSPACE_RESULT" = "skipped" ] && [ "$LEGACY_RESULT" = "skipped" ]; then
+            echo "⏭️  No packages needed publishing (version selector skipped them all — already up-to-date or excluded)."
           else
-            echo "❌ Publishing failed"
-            exit 1
+            echo "⚠️  No packages were actually published. Check job results above."
+          fi
+
+          if [ ${#WARNINGS[@]} -gt 0 ]; then
+            echo ""
+            echo "⚠️  Soft warnings (release shipped but follow-up needed):"
+            for w in "${WARNINGS[@]}"; do
+              echo "   - $w"
+            done
           fi
 
   create-release-notes: