Skip to content

feat(registry-webhook-sync): Introducing Terraform Registry webhook sync workflow 🚀 #400

Merged
hahirwar-cd merged 2 commits into
masterfrom
feat/registry-webhook-sync
Jun 30, 2026
Merged

feat(registry-webhook-sync): Introducing Terraform Registry webhook sync workflow 🚀 #400
hahirwar-cd merged 2 commits into
masterfrom
feat/registry-webhook-sync

Conversation

@clouddrove-ci

Copy link
Copy Markdown
Member

Problem

New tags for terraform-aws-* modules sometimes don't appear on the public Terraform Registry, forcing a manual Resync Module click in the registry UI. This automates the fix and makes it self-healing across orgs.

Root cause (investigated, with live evidence)

The registry installs a GitHub webhook per published repo. On every clouddrove module repo that hook is subscribed to push only:

  • git push origin <tag> (our release-tag.yml) fires a push event → registry syncs ✅
  • A tag cut via the GitHub Release UI/API fires create + release, often not push → the push-only hook never hears it → registry lags → manual resync ⚠️

There is no public registry API to force a resync (UI-only; the v2 can-resync action needs a fragile registry session token). So the durable fix is correcting the GitHub webhook, not calling the registry.

Evidence gathered (read-only, against clouddrove)

1. Org-wide webhook audit (dry-run, 177 terraform-* repos):

Result Count
Hook = ["push"] only → would fix to ["push","create","delete"] 120
No registry hook at all (unpublished / hook deleted) 57
Already correct 0

2. git-tag vs registry-version spot check (v-prefix normalized):

Repo git latest registry latest state
terraform-aws-eks 1.4.8 1.4.8 SYNCED
terraform-aws-vpc 2.0.5 2.0.5 SYNCED
terraform-aws-ec2 2.1.1 2.1.1 SYNCED
terraform-aws-iam 4.11.0 4.11.0 SYNCED
terraform-aws-kms 1.3.4 1.3.4 SYNCED
terraform-aws-s3 2.0.2 2.0.1 LAG
terraform-aws-lambda 1.3.3 1.3.2 LAG

So most modules sync fine (push-driven); the misses are intermittent and align with manually-cut releases (e.g. s3 v2.0.2 was a hand-cut GitHub Release). GitHub had purged old webhook delivery history, so the exact missed delivery couldn't be replayed — diagnosis rests on the audit + lag correlation + HashiCorp's documented requirement that the registry webhook fire on tag-creation events.

3. Reconcile logic validated against live repos in dry-run — drift detection correctly reports have=["push"] missing=["create","delete"] per repo and makes no changes in dry_run mode.

What this PR adds

  • .github/workflows/registry-webhook-sync.yml — reusable (workflow_call) + workflow_dispatch. Loops orgs → terraform-* repos → finds the registry hook → adds missing create/delete events. Has dry_run, writes a job-summary report, leaves repos with no registry hook untouched (reported only).
  • docs/registry-webhook-sync.md — problem writeup, inputs/secrets, scheduled cron caller example.
  • WORKFLOW_CATALOG.md — catalog entry.

Inputs: orgs (csv, required), repo_filter (default ^terraform-), required_events (default push,create,delete), dry_run. Secret: GH_ADMIN_TOKEN (PAT with admin:repo_hook + repo read).

Honest scope / caveats

  • Fixes future tags. Versions already missed need one manual resync each to backfill.
  • The webhook fix is probabilistic (adds a second trigger). For a guaranteed catch, a follow-up safety-net workflow can verify the version landed post-release and Slack-alert if not — not in this PR.
  • 57 repos have no registry hook — likely unpublished modules; only the registry creates that hook (on publish). Separate triage.

Validation done

  • YAML parses; has name: + jobs: (passes ci.yml structure checks).
  • Core reconcile loop executed live in dry_run against clouddrove — results above.

Follow-ups (not in this PR)

  • Run live on clouddrove (apply the 120) after review.
  • Add scheduled caller (cron) with all 5 org names in an ops repo.
  • Optional safety-net verify+notify workflow.
  • Triage the 57 no-hook repos.

Adds registry-webhook-sync.yml, a reusable + dispatchable workflow that
audits the Terraform Registry webhook on every terraform-* repo across one
or more orgs and ensures it fires on push,create,delete (not push only),
so module tags cut via the GitHub Release UI/API also reach the registry
without a manual "Resync Module".

- registry-webhook-sync.yml: audit/fix hooks, dry_run mode, job summary
- docs/registry-webhook-sync.md: problem writeup, inputs/secrets, cron caller
- WORKFLOW_CATALOG.md: catalog entry
@hahirwar-cd hahirwar-cd changed the title feat: add Terraform Registry webhook sync workflow feat(registry-webhook-sync): 🚀 Introducing Terraform Registry webhook sync workflow Jun 29, 2026
@hahirwar-cd hahirwar-cd changed the title feat(registry-webhook-sync): 🚀 Introducing Terraform Registry webhook sync workflow feat(registry-webhook-sync): Introducing Terraform Registry webhook sync workflow 🚀 Jun 29, 2026
@dverma-cd dverma-cd self-requested a review June 29, 2026 09:40
@hahirwar-cd hahirwar-cd merged commit f6ef7e5 into master Jun 30, 2026
15 checks passed
@hahirwar-cd hahirwar-cd deleted the feat/registry-webhook-sync branch June 30, 2026 12:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants