Harden production promotion workflow

Author: justin808

.controlplane/docs/testing-cpflow-github-actions.md

@@ -29,8 +29,10 @@ bin/pin-cpflow-github-ref <40-character-control-plane-flow-commit-sha>
 bin/conductor-exec bin/test-cpflow-github-flow ruby /path/to/control-plane-flow/bin/cpflow
 ```
 
-Leave `CPFLOW_VERSION` unset while testing a commit SHA. After the upstream gem
-and tag ship, repin wrappers to the release tag, such as `v5.0.4`.
+Leave `CPFLOW_VERSION` unset while testing a commit SHA. After the upstream PR
+ships in a release tag, repin wrappers to that tag. Use `v5.1.0` only for
+changes already included in that tag; keep this promotion-hardening canary on an
+immutable commit SHA until the upstream hardening PR is released.
 
 ## Review App Canary
 

.controlplane/readme.md

@@ -122,7 +122,16 @@ The matching Control Plane resources are:
 | Production app secret dictionary | `react-webpack-rails-tutorial-production-secrets` |
 
 Bootstrap production the same way before the first promotion, using the
-production org and production-only secret values.
+production org and production-only secret values. After bootstrap or any
+template change, re-apply the persistent production templates so the `rails`
+and `daily-task` workloads keep the same secret-backed env names as staging:
+
+```sh
+cpflow apply-template app postgres redis daily-task rails \
+  -a react-webpack-rails-tutorial-production \
+  --org shakacode-open-source-examples-production \
+  --yes --add-app-identity
+```
 
 All review, staging, and production secret dictionaries need these app runtime
 secrets:
@@ -529,8 +538,10 @@ If staging moves off `master`, update both the `STAGING_APP_BRANCH` repository
 variable and the `branches:` filter in `.github/workflows/cpflow-deploy-staging.yml`;
 GitHub does not allow repository variables in trigger branch filters.
 The production promotion workflow checks that production has all environment
-variable names present in staging; it does not compare secret values, workload
-environment variables, or Control Plane secret references.
+variable names present in staging at both the GVC level and each configured app
+workload's container level. It does not compare secret values. The health check
+waits for Control Plane to report both `status.ready` and `status.readyLatest`
+before probing the public endpoint.
 
 The GitHub settings and Control Plane resources must match the app names in
 `.controlplane/controlplane.yml`. For the standard review-app path, leave
@@ -564,9 +575,11 @@ Keep the reusable-workflow mechanics in the upstream
 For this repo, the update loop is:
 
 1. Generate from the desired `cpflow` release with `--staging-branch master`.
-2. Keep generated refs on a release tag such as `v5.0.4`. Use a full upstream
-   commit SHA only for short-lived downstream testing of an unreleased upstream
-   PR, and leave `CPFLOW_VERSION` unset in that case.
+2. Keep generated refs on a release tag once the upstream hardening changes ship.
+   This branch temporarily pins refs to
+   `9ef104c246670d6c1ea4132dfd22be68ef930a70` to test upstream promotion
+   hardening before the next release tag. Leave `CPFLOW_VERSION` unset while
+   testing a commit SHA.
 3. Keep app names and GitHub settings aligned with `.controlplane/controlplane.yml`.
 4. Validate locally:
 

.controlplane/shakacode-team.md

@@ -102,14 +102,27 @@ cpflow setup-app -a react-webpack-rails-tutorial-production --org shakacode-open
 
 Use `setup-app` for first-time bootstrap because it creates the app secret
 policy and identity binding. Use `cpflow apply-template` for later template
-updates to existing persistent apps.
+updates to existing persistent apps. Production promotion compares both GVC env
+names and app workload container env names against staging before copying the
+image, so keep production `rails` and `daily-task` env references in sync with
+the templates:
+
+```sh
+cpflow apply-template app postgres redis daily-task rails \
+  -a react-webpack-rails-tutorial-production \
+  --org shakacode-open-source-examples-production \
+  --yes --add-app-identity
+```
 
 Advanced optional settings are documented upstream in the
 [`control-plane-flow` CI automation guide](https://github.com/shakacode/control-plane-flow/blob/main/docs/ci-automation.md).
 
-Current workflow wrappers are pinned to the upstream `control-plane-flow`
-release tag `v5.0.4`. Keep release tags as the steady-state configuration; use
-a full commit SHA only for short-lived upstream PR testing.
+Current workflow wrappers are temporarily pinned to upstream
+`control-plane-flow` commit `9ef104c246670d6c1ea4132dfd22be68ef930a70` to test
+promotion hardening before it ships in a release tag. Keep release tags as the
+steady-state configuration once the upstream PR is released; use a full commit
+SHA only for short-lived upstream PR testing and leave `CPFLOW_VERSION` unset in
+that case.
 
 If staging moves off `master`, update both `STAGING_APP_BRANCH` and the branch
 filter in `.github/workflows/cpflow-deploy-staging.yml`.
@@ -119,8 +132,8 @@ filter in `.github/workflows/cpflow-deploy-staging.yml`.
 When the upstream `control-plane-flow` repo changes the generated GitHub Actions
 flow, regenerate from the target `cpflow` version with `--staging-branch master`,
 review the diff, and validate with `bin/test-cpflow-github-flow` plus the normal
-CI checks. Stable automation should use release tags such as `v5.0.4`, not
-`main` or a feature branch.
+CI checks. Stable automation should use a release tag that includes the upstream
+hardening changes, not `main` or a feature branch.
 
 See [readme.md](readme.md) and
 [Testing cpflow GitHub Actions Changes](docs/testing-cpflow-github-actions.md)

.github/cpflow-help.md

@@ -2,7 +2,7 @@
 
 These commands are generated by [cpflow](https://github.com/shakacode/control-plane-flow).
 For full setup, version-pinning, and troubleshooting details, see the upstream
-[CI automation guide](https://github.com/shakacode/control-plane-flow/blob/v5.0.4/docs/ci-automation.md).
+[CI automation guide](https://github.com/shakacode/control-plane-flow/blob/9ef104c246670d6c1ea4132dfd22be68ef930a70/docs/ci-automation.md).
 
 ## Pull Request Commands
 
@@ -23,23 +23,11 @@ For the normal generated review-app path, GitHub needs one repository secret:
 | --- | --- | --- |
 | `CPLN_TOKEN_STAGING` | Repository secret | Control Plane service-account token for the staging/review org. |
 
-For public repositories, use a staging/review token that cannot access
-production Control Plane resources. Generated review-app deploys skip fork PR
-heads because Docker builds use repository secrets. If a forked change needs a
-review app, first move the reviewed change to a trusted branch in this
-repository.
-
 No repository variables are required for the standard review-app path when
 `.controlplane/controlplane.yml` has exactly one review app entry with
 `match_if_app_name_starts_with: true`. cpflow infers the review-app prefix and
 staging org from that config.
 
-Review apps run pull request code. Any value mounted through
-`cpln://secret/...` can be read by that code after the workload starts, so keep
-review-app secret dictionaries limited to disposable databases, review-only
-renderer credentials, and license values that are acceptable for review-app
-exposure.
-
 Optional overrides exist for forks, clones, and unusual apps:
 
 | Name | Notes |
@@ -89,7 +77,7 @@ normal environment-gated job cannot tell which secret scope supplied a nonempty
 value, so a broader secret with the same name can mask a missing environment
 secret.
 
-If promotion fails with
+If the promotion workflow fails with
 `CPLN_TOKEN_PRODUCTION is not set. Add it as a secret on the 'production' GitHub Environment.`,
 the token is missing from the environment scope or the workflow job is no longer
 declaring `environment: production`. Create or verify the environment secret
@@ -98,27 +86,31 @@ You need permission to manage repository environments and secrets to run these
 commands.
 
 ```sh
-gh secret set CPLN_TOKEN_PRODUCTION --repo shakacode/react-webpack-rails-tutorial --env production
-gh secret list --repo shakacode/react-webpack-rails-tutorial --env production
-gh secret list --repo shakacode/react-webpack-rails-tutorial
-gh secret list --org shakacode | grep '^CPLN_TOKEN_PRODUCTION[[:space:]]' || true
+gh secret set CPLN_TOKEN_PRODUCTION --repo OWNER/REPO --env production
+# Paste the token value when prompted.
+gh secret list --repo OWNER/REPO --env production
+gh secret list --repo OWNER/REPO
+gh secret list --org OWNER | grep '^CPLN_TOKEN_PRODUCTION[[:space:]]' || true
 ```
 
 Before the first promotion, bootstrap the production app the same way in the
 production org, using production-only secrets and values.
 
 ## Version Locking
 
-Generated wrappers pin Control Plane Flow with a release tag, for example
-`v5.0.4`. Reusable review-app, staging, cleanup, and helper workflows pin the
-tag in their `uses:` ref. Production promotion pins the same tag in the
-`Checkout control-plane-flow actions` step so the caller-owned job can keep
+Generated wrappers normally pin Control Plane Flow with a release tag, for
+example `v5.1.0`. This branch temporarily pins the wrappers to upstream commit
+`9ef104c246670d6c1ea4132dfd22be68ef930a70` while testing unreleased production
+promotion hardening. Reusable review-app, staging, cleanup, and helper workflows
+pin that ref in their `uses:` entry. Production promotion pins the same ref in
+the `Checkout control-plane-flow actions` step so the caller-owned job can keep
 `environment: production` and receive production environment secrets directly.
 
 Leave `CPFLOW_VERSION` unset so the workflow builds cpflow from the same
 checked-out upstream source. If you set `CPFLOW_VERSION`, it must match the
-release tag, for example `CPFLOW_VERSION=5.0.4` with a wrapper pinned to
-`uses: ...@v5.0.4`.
+release tag your wrappers are pinned to: a `CPFLOW_VERSION=5.1.x` runtime
+override goes with a wrapper pinned to `uses: ...@v5.1.x` (substitute the
+release you pinned above).
 
 After updating the `cpflow` gem in this repo, update the generated wrappers in
 the same PR:
@@ -151,7 +143,7 @@ Most apps do not need these:
 | Name | Notes |
 | --- | --- |
 | `DOCKER_BUILD_EXTRA_ARGS` | Newline-delimited extra Docker build tokens. |
-| `DOCKER_BUILD_SSH_KEY` | Read-only, revocable deploy key for Docker builds that fetch private dependencies. Do not use a personal SSH key. |
+| `DOCKER_BUILD_SSH_KEY` | Private SSH key for Docker builds that fetch private dependencies. |
 | `DOCKER_BUILD_SSH_KNOWN_HOSTS` | SSH known_hosts entries when SSH build hosts are not GitHub.com. |
 | `REVIEW_APP_DEPLOYING_ICON_URL` | Cosmetic custom image URL for the animated deploying icon. Set to `none` to use the text fallback icon. |
 | `STAGING_APP_BRANCH` | Custom staging branch. The branch must also appear in `cpflow-deploy-staging.yml`'s push filter. |

.github/workflows/cpflow-cleanup-stale-review-apps.yml

@@ -12,6 +12,6 @@ jobs:
   cleanup:
     # Cleanup targets the current inferred review-app prefix. If you changed
     # naming conventions, manually delete review apps under the old prefix.
-    uses: shakacode/control-plane-flow/.github/workflows/cpflow-cleanup-stale-review-apps.yml@v5.0.4
+    uses: shakacode/control-plane-flow/.github/workflows/cpflow-cleanup-stale-review-apps.yml@9ef104c246670d6c1ea4132dfd22be68ef930a70
     secrets:
       CPLN_TOKEN_STAGING: ${{ secrets.CPLN_TOKEN_STAGING }}

.github/workflows/cpflow-delete-review-app.yml

@@ -31,6 +31,6 @@ jobs:
       github.event_name == 'workflow_dispatch'
     # This `if:` mirrors the upstream job guard to avoid a billable workflow_call
     # when the event does not match. Keep both conditions in sync.
-    uses: shakacode/control-plane-flow/.github/workflows/cpflow-delete-review-app.yml@v5.0.4
+    uses: shakacode/control-plane-flow/.github/workflows/cpflow-delete-review-app.yml@9ef104c246670d6c1ea4132dfd22be68ef930a70
     secrets:
       CPLN_TOKEN_STAGING: ${{ secrets.CPLN_TOKEN_STAGING }}

.github/workflows/cpflow-deploy-review-app.yml

@@ -30,7 +30,7 @@ jobs:
        github.event.issue.pull_request &&
        contains(fromJson('["+review-app-deploy","+review-app-deploy\n","+review-app-deploy\r\n"]'), github.event.comment.body) &&
        contains(fromJson('["OWNER","MEMBER","COLLABORATOR"]'), github.event.comment.author_association))
-    uses: shakacode/control-plane-flow/.github/workflows/cpflow-deploy-review-app.yml@v5.0.4
+    uses: shakacode/control-plane-flow/.github/workflows/cpflow-deploy-review-app.yml@9ef104c246670d6c1ea4132dfd22be68ef930a70
     secrets:
       CPLN_TOKEN_STAGING: ${{ secrets.CPLN_TOKEN_STAGING }}
       DOCKER_BUILD_SSH_KEY: ${{ secrets.DOCKER_BUILD_SSH_KEY }}

.github/workflows/cpflow-deploy-staging.yml

@@ -16,7 +16,7 @@ permissions:
 
 jobs:
   deploy-staging:
-    uses: shakacode/control-plane-flow/.github/workflows/cpflow-deploy-staging.yml@v5.0.4
+    uses: shakacode/control-plane-flow/.github/workflows/cpflow-deploy-staging.yml@9ef104c246670d6c1ea4132dfd22be68ef930a70
     with:
       staging_app_branch_default: "master"
     secrets:

.github/workflows/cpflow-help-command.yml

@@ -23,4 +23,4 @@ jobs:
        contains(fromJson('["+review-app-help","+review-app-help\n","+review-app-help\r\n"]'), github.event.comment.body) &&
        contains(fromJson('["OWNER","MEMBER","COLLABORATOR"]'), github.event.comment.author_association)) ||
       github.event_name == 'workflow_dispatch'
-    uses: shakacode/control-plane-flow/.github/workflows/cpflow-help-command.yml@v5.0.4
+    uses: shakacode/control-plane-flow/.github/workflows/cpflow-help-command.yml@9ef104c246670d6c1ea4132dfd22be68ef930a70