Skip to content

Commit f2bb55c

Browse files
bhillkeyfactorclaude
bhillkeyfactor
and
claude
committed
Replace update-catalog with validation gate and PR-based flow
The existing update-catalog.yml committed catalog entries directly to the integrations-catalog main branch with no validation, no PR, and no review. Several bugs accumulated: - *-dev / *-test / *-staging / *-poc repos with update_catalog=true leaked entries into the public catalog (~6 known leaks were manually deleted Feb-Mar 2026). - Repos with empty names, missing descriptions, or invalid integration_type values published verbatim — no schema check. - Private repos with link_github=true created broken-link entries on the public catalog. There was no notion of a Coming Soon entry for intentionally-private in-flight integrations. - Every catalog write used the same "Added the manifest for X" commit message regardless of whether it was an add or an update. This replacement adds three things: 1. A validation gate that runs before rendering. Rejects repo names matching the non-production pattern (failsafe — primary control is still update_catalog=false in the manifest), verifies required fields exist (name, integration_type, description), validates integration_type against an enum, and handles the private/link_github combinations correctly (private + link_github=false renders as Coming Soon; private + link_github=true is rejected with a clear error). 2. A branch + PR flow instead of direct-to-main commits. Each integration push lands on a catalog-update/{repo} branch and opens a PR labeled catalog-add, catalog-update, or coming-soon depending on the entry type. Re-pushes force-update the branch and the existing PR in place (no duplicates). 3. Idempotent gh label create calls before gh pr create. Without these, the first run with a fresh integrations-catalog repo fails at the PR-creation step because the labels don't exist yet (same failure mode the cleanup-catalog workflow hit during its initial deployment). Additions vs. the draft in integrations-catalog/docs/proposed-update-catalog.yml: - dns-plugin added to ALLOWED_TYPES (a category page now exists in the catalog repo and an integration of this type has already published). - The label-create block (point 3 above), informed by the cleanup workflow's own first-run failure. Aggressive rollout: this replaces the unversioned reference that every integration repo's caller workflow points at. The next push from any integration repo will use the new flow. The flow fails closed — if validation rejects, no catalog entry is added; the catalog repo's main branch is unaffected. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 9cb0eaa commit f2bb55c

1 file changed

Lines changed: 224 additions & 41 deletions

File tree

‎.github/workflows/update-catalog.yml‎

Lines changed: 224 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -1,41 +1,224 @@
1-
name: Update Keyfactor Integrations Catalog Entry
2-
on:
3-
workflow_call:
4-
secrets:
5-
token:
6-
description: 'Secret token from caller workflow to access SDK repo'
7-
required: true
8-
9-
jobs:
10-
update-catalog-entry:
11-
runs-on: ubuntu-latest
12-
13-
steps:
14-
- name: Checkout project repo
15-
uses: keyfactor/checkout@v4
16-
17-
- name: Checkout catalog repo
18-
uses: keyfactor/checkout@v4
19-
with:
20-
token: ${{ secrets.token }}
21-
path: './catalog-temp/'
22-
repository: 'Keyfactor/integrations-catalog' # Change back to integrations-catalog after testing
23-
24-
- uses: Keyfactor/jinja2-action@v1.2.0-multiple-data-files
25-
with:
26-
template: ./catalog-temp/_integration.md.tpl
27-
output_file: ${{ format('./catalog-temp/_integrations/{0}.md', github.event.repository.name) }}
28-
data_file: integration-manifest.json
29-
variables: |
30-
repository= ${{ format('https://github.com/{0}', github.repository) }}
31-
env:
32-
GITHUB_TOKEN: ${{ secrets.token }}
33-
34-
- uses: Keyfactor/add-and-commit@v9.1.4
35-
with:
36-
author_name: 'Keyfactor'
37-
author_email: 'keyfactor@keyfactor.github.io'
38-
branch: 'main'
39-
message: ${{ format('Added the manifest for {0}', github.event.repository.name) }}
40-
add: ${{ format('_integrations/{0}.md --force', github.event.repository.name) }}
41-
cwd: './catalog-temp/'
1+
# ============================================================================
2+
# PROPOSED replacement for: Keyfactor/actions/.github/workflows/update-catalog.yml
3+
#
4+
# Changes from current version:
5+
# 1. Adds validation gate (repo name filter, manifest field checks, visibility)
6+
# 2. Replaces direct-to-main commit with PR-based flow
7+
# 3. Handles private repos as "Coming Soon" entries (link_github=false)
8+
# 4. Labels PRs by type: catalog-add, catalog-update, coming-soon
9+
# ============================================================================
10+
11+
name: Update Keyfactor Integrations Catalog Entry
12+
on:
13+
workflow_call:
14+
secrets:
15+
token:
16+
description: 'Secret token from caller workflow to access catalog repo'
17+
required: true
18+
19+
jobs:
20+
update-catalog-entry:
21+
runs-on: ubuntu-latest
22+
23+
steps:
24+
- name: Checkout project repo
25+
uses: keyfactor/checkout@v4
26+
27+
- name: Checkout catalog repo
28+
uses: keyfactor/checkout@v4
29+
with:
30+
token: ${{ secrets.token }}
31+
path: './catalog-temp/'
32+
repository: 'Keyfactor/integrations-catalog'
33+
34+
# ----------------------------------------------------------------
35+
# STEP 1: Validation Gate
36+
# ----------------------------------------------------------------
37+
- name: Validate entry
38+
id: validate
39+
env:
40+
GH_TOKEN: ${{ secrets.token }}
41+
REPO_NAME: ${{ github.event.repository.name }}
42+
REPO_FULL: ${{ github.repository }}
43+
run: |
44+
echo "### Validating: $REPO_NAME" >> "$GITHUB_STEP_SUMMARY"
45+
46+
# --- Failsafe: reject -dev, -test, -staging, -poc repo names ---
47+
# NOTE: This is a failsafe only. Developers are responsible for setting
48+
# update_catalog=false in their manifest for non-production repos.
49+
if echo "$REPO_NAME" | grep -qE '-(dev|test|staging|poc)$'; then
50+
echo "::error::Repository name '$REPO_NAME' matches a non-production pattern (-dev, -test, -staging, -poc). Set update_catalog=false in your integration-manifest.json for non-production repos."
51+
echo "rejected=true" >> "$GITHUB_OUTPUT"
52+
echo "reject_reason=Repo name matches non-production pattern. Developers: set \`update_catalog=false\` in your manifest." >> "$GITHUB_OUTPUT"
53+
exit 1
54+
fi
55+
56+
# --- Validate required manifest fields ---
57+
if [ ! -f "integration-manifest.json" ]; then
58+
echo "::error::integration-manifest.json not found in repository root."
59+
exit 1
60+
fi
61+
62+
NAME=$(jq -r '.name // empty' integration-manifest.json)
63+
TYPE=$(jq -r '.integration_type // empty' integration-manifest.json)
64+
DESC=$(jq -r '.description // empty' integration-manifest.json)
65+
LINK_GITHUB=$(jq -r '.link_github // "true"' integration-manifest.json | tr '[:upper:]' '[:lower:]')
66+
67+
ERRORS=""
68+
if [ -z "$NAME" ]; then
69+
ERRORS="$ERRORS\n- Missing required field: \`name\`"
70+
fi
71+
if [ -z "$TYPE" ]; then
72+
ERRORS="$ERRORS\n- Missing required field: \`integration_type\`"
73+
fi
74+
if [ -z "$DESC" ]; then
75+
ERRORS="$ERRORS\n- Missing required field: \`description\`"
76+
fi
77+
78+
# Validate integration_type against allowed values
79+
ALLOWED_TYPES="orchestrator windows-orchestrator iot-orchestrator ca-gateway anyca-plugin caplugin dns-plugin pam approval-handler orchestrator-registration metadata registration-handler alert-handler api-client terraform"
80+
if [ -n "$TYPE" ]; then
81+
if ! echo "$ALLOWED_TYPES" | grep -qw "$TYPE"; then
82+
ERRORS="$ERRORS\n- Invalid \`integration_type\`: \`$TYPE\`. Allowed: $ALLOWED_TYPES"
83+
fi
84+
fi
85+
86+
if [ -n "$ERRORS" ]; then
87+
printf "::error::Manifest validation failed:%b\n" "$ERRORS"
88+
exit 1
89+
fi
90+
91+
# --- Check repo visibility ---
92+
IS_PRIVATE=$(gh api "repos/$REPO_FULL" --jq '.private')
93+
94+
if [ "$IS_PRIVATE" = "true" ] && [ "$LINK_GITHUB" = "true" ]; then
95+
echo "::error::Repository '$REPO_FULL' is private but link_github=true. This would create a broken link on the public catalog. Set link_github=false in your manifest to create a 'Coming Soon' entry, or make the repo public first."
96+
exit 1
97+
fi
98+
99+
# Determine entry type for labeling
100+
if [ "$IS_PRIVATE" = "true" ] && [ "$LINK_GITHUB" = "false" ]; then
101+
echo "entry_type=coming-soon" >> "$GITHUB_OUTPUT"
102+
echo "Entry type: Coming Soon (private repo)" >> "$GITHUB_STEP_SUMMARY"
103+
elif [ -f "./catalog-temp/_integrations/$REPO_NAME.md" ]; then
104+
echo "entry_type=catalog-update" >> "$GITHUB_OUTPUT"
105+
echo "Entry type: Update (existing entry)" >> "$GITHUB_STEP_SUMMARY"
106+
else
107+
echo "entry_type=catalog-add" >> "$GITHUB_OUTPUT"
108+
echo "Entry type: Add (new entry)" >> "$GITHUB_STEP_SUMMARY"
109+
fi
110+
111+
echo "link_github=$LINK_GITHUB" >> "$GITHUB_OUTPUT"
112+
echo "Validation passed" >> "$GITHUB_STEP_SUMMARY"
113+
114+
# ----------------------------------------------------------------
115+
# STEP 2: Render Template
116+
# ----------------------------------------------------------------
117+
- uses: Keyfactor/jinja2-action@v1.2.0-multiple-data-files
118+
with:
119+
template: ./catalog-temp/_integration.md.tpl
120+
output_file: ${{ format('./catalog-temp/_integrations/{0}.md', github.event.repository.name) }}
121+
data_file: integration-manifest.json
122+
variables: |
123+
repository= ${{ format('https://github.com/{0}', github.repository) }}
124+
env:
125+
GITHUB_TOKEN: ${{ secrets.token }}
126+
127+
# ----------------------------------------------------------------
128+
# STEP 3: Create PR (instead of direct commit)
129+
# ----------------------------------------------------------------
130+
- name: Create or update PR
131+
env:
132+
GH_TOKEN: ${{ secrets.token }}
133+
REPO_NAME: ${{ github.event.repository.name }}
134+
ENTRY_TYPE: ${{ steps.validate.outputs.entry_type }}
135+
working-directory: './catalog-temp/'
136+
run: |
137+
BRANCH="catalog-update/$REPO_NAME"
138+
139+
git config user.name "Keyfactor"
140+
git config user.email "keyfactor@keyfactor.github.io"
141+
142+
# Check if branch already exists on remote
143+
if git ls-remote --heads origin "$BRANCH" | grep -q "$BRANCH"; then
144+
git fetch origin "$BRANCH"
145+
git checkout "$BRANCH"
146+
git reset --hard origin/main
147+
else
148+
git checkout -b "$BRANCH"
149+
fi
150+
151+
# Stage the rendered file
152+
git add "_integrations/$REPO_NAME.md" --force
153+
154+
# Check if there are actual changes
155+
if git diff --cached --quiet; then
156+
echo "No changes detected — catalog entry is already up to date."
157+
echo "### No changes" >> "$GITHUB_STEP_SUMMARY"
158+
echo "Catalog entry for \`$REPO_NAME\` is already current." >> "$GITHUB_STEP_SUMMARY"
159+
exit 0
160+
fi
161+
162+
# Determine commit message
163+
case "$ENTRY_TYPE" in
164+
catalog-add)
165+
COMMIT_MSG="Add catalog entry for $REPO_NAME"
166+
PR_TITLE="Add integration: $REPO_NAME"
167+
;;
168+
catalog-update)
169+
COMMIT_MSG="Update catalog entry for $REPO_NAME"
170+
PR_TITLE="Update integration: $REPO_NAME"
171+
;;
172+
coming-soon)
173+
COMMIT_MSG="Add Coming Soon catalog entry for $REPO_NAME"
174+
PR_TITLE="Coming Soon: $REPO_NAME"
175+
;;
176+
esac
177+
178+
git commit -m "$COMMIT_MSG"
179+
git push origin "$BRANCH" --force
180+
181+
# Ensure the three labels this workflow uses exist (idempotent — gh
182+
# label create exits 1 if the label is already present, which we
183+
# swallow). Without this, gh pr create --label hard-fails when the
184+
# label is missing in the catalog repo.
185+
gh label create catalog-add --color "2ECC71" --description "New integration entry added to the catalog" 2>/dev/null || true
186+
gh label create catalog-update --color "1A73E8" --description "Existing integration entry updated" 2>/dev/null || true
187+
gh label create coming-soon --color "F39C12" --description "Private repo published as a Coming Soon entry" 2>/dev/null || true
188+
189+
# Build PR body
190+
PR_BODY=$(cat <<EOF
191+
## Catalog Entry: \`$REPO_NAME\`
192+
193+
**Type:** $ENTRY_TYPE
194+
**Source:** [${REPO_NAME}](https://github.com/${{ github.repository }})
195+
**Triggered by:** Push to \`${{ github.ref_name }}\` by @${{ github.actor }}
196+
197+
### Entry Preview
198+
\`\`\`yaml
199+
$(cat "_integrations/$REPO_NAME.md")
200+
\`\`\`
201+
202+
---
203+
*This PR was automatically created by the catalog update workflow.*
204+
*Review the entry content above and merge when ready.*
205+
EOF
206+
)
207+
208+
# Check if a PR already exists for this branch
209+
EXISTING_PR=$(gh pr list --head "$BRANCH" --json number --jq '.[0].number' 2>/dev/null || true)
210+
211+
if [ -n "$EXISTING_PR" ] && [ "$EXISTING_PR" != "null" ]; then
212+
echo "PR #$EXISTING_PR already exists for branch $BRANCH — updated with force push."
213+
echo "### Updated existing PR #$EXISTING_PR" >> "$GITHUB_STEP_SUMMARY"
214+
else
215+
gh pr create \
216+
--title "$PR_TITLE" \
217+
--body "$PR_BODY" \
218+
--label "$ENTRY_TYPE" \
219+
--base main \
220+
--head "$BRANCH"
221+
222+
echo "### Created PR" >> "$GITHUB_STEP_SUMMARY"
223+
echo "PR created for \`$REPO_NAME\` with label \`$ENTRY_TYPE\`" >> "$GITHUB_STEP_SUMMARY"
224+
fi

0 commit comments

Comments
 (0)