Skip to content

Commit 89bbeab

Browse files
locus313Copilot
andcommitted
refactor: replace get_all_pages with gh_api_paginate and use err() helper
- Replace 52-line hand-rolled get_all_pages() paginator in github-close-archived-repo-security-alerts with gh_api_paginate streaming via 'while read'; non-array responses (e.g. Advanced Security not enabled) silently filtered by select(.number) - Replace print_error + exit 1 two-liners with err() across 9 scripts (17 sites); two multi-message sites merged into single err() calls - Merge CI shellcheck and unit-test jobs into one job to eliminate duplicate checkout + apt-get update; update ci-skip.yml to match - Merge two apt-get steps in copilot-setup-steps.yml into one - Remove ~96 lines of verbatim-duplicate content from AGENTS.md (script anatomy, helpers table, adding-a-script steps); replace with cross-reference to .github/copilot-instructions.md Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent c501bd4 commit 89bbeab

13 files changed

Lines changed: 55 additions & 247 deletions

File tree

.github/workflows/ci-skip.yml

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,11 @@
11
name: CI
22

33
# Companion to ci.yml. When a pull request only touches documentation or other
4-
# path-ignored files, ci.yml is skipped entirely and its required "Shellcheck"
5-
# check never reports — which blocks the PR from merging. This workflow runs on
6-
# exactly those ignored paths and emits a passing check with the same job name,
7-
# so the required status is always satisfied without a manual bypass.
4+
# path-ignored files, ci.yml is skipped entirely and its required
5+
# "Shellcheck + Unit Tests" check never reports — which blocks the PR from
6+
# merging. This workflow runs on exactly those ignored paths and emits a passing
7+
# check with the same job name, so the required status is always satisfied
8+
# without a manual bypass.
89

910
on:
1011
pull_request:
@@ -14,8 +15,8 @@ on:
1415
- '.github/FUNDING.yml'
1516

1617
jobs:
17-
shellcheck:
18-
name: Shellcheck
18+
ci:
19+
name: Shellcheck + Unit Tests
1920
runs-on: ubuntu-latest
2021
permissions:
2122
contents: read

.github/workflows/ci.yml

Lines changed: 4 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -15,17 +15,17 @@ on:
1515
- '.github/FUNDING.yml'
1616

1717
jobs:
18-
shellcheck:
19-
name: Shellcheck
18+
ci:
19+
name: Shellcheck + Unit Tests
2020
runs-on: ubuntu-latest
2121
permissions:
2222
contents: read
2323
steps:
2424
- name: Checkout code
2525
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
2626

27-
- name: Install shellcheck
28-
run: sudo apt-get update -qq && sudo apt-get install -y shellcheck
27+
- name: Install shellcheck and bats
28+
run: sudo apt-get update -qq && sudo apt-get install -y shellcheck bats
2929

3030
- name: Run shellcheck on all scripts
3131
run: |
@@ -34,17 +34,5 @@ jobs:
3434
--exclude=SC2034,SC1091 \
3535
--shell=bash
3636
37-
test:
38-
name: Unit Tests
39-
runs-on: ubuntu-latest
40-
permissions:
41-
contents: read
42-
steps:
43-
- name: Checkout code
44-
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
45-
46-
- name: Install bats
47-
run: sudo apt-get update -qq && sudo apt-get install -y bats
48-
4937
- name: Run unit tests
5038
run: bats tests/

.github/workflows/copilot-setup-steps.yml

Lines changed: 2 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -10,11 +10,8 @@ jobs:
1010
- name: Checkout code
1111
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
1212

13-
- name: Install jq
14-
run: sudo apt-get update -qq && sudo apt-get install -y jq
15-
16-
- name: Install shellcheck
17-
run: sudo apt-get install -y shellcheck
13+
- name: Install jq, shellcheck, and bats
14+
run: sudo apt-get update -qq && sudo apt-get install -y jq shellcheck bats
1815

1916
- name: Install gitleaks
2017
run: |

AGENTS.md

Lines changed: 5 additions & 101 deletions
Original file line numberDiff line numberDiff line change
@@ -136,106 +136,10 @@ Use `_mock_curl_200` (defined in `test_script_validation.bats`) when a test must
136136

137137
## Key Patterns and Conventions
138138

139-
### Script anatomy
140-
141-
Every script begins with a `# ===` header (exactly 79 `=` chars), then immediately `set -euo pipefail`:
142-
143-
```bash
144-
#!/usr/bin/env bash
145-
# =============================================================================
146-
# github-<name>.sh
147-
#
148-
# <description>
149-
#
150-
# Usage:
151-
# export GITHUB_TOKEN=ghp_yourtoken
152-
# export ORG=my-org
153-
# ./github-<name>.sh
154-
#
155-
# Environment variables:
156-
# GITHUB_TOKEN Required. PAT with <scope> scope
157-
# ORG Required. GitHub organization name
158-
# API_URL_PREFIX Optional. GitHub API base URL (default: https://api.github.com)
159-
#
160-
# Requirements:
161-
# - curl
162-
# - jq
163-
# =============================================================================
164-
165-
set -euo pipefail
166-
167-
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
168-
# shellcheck source=../../lib/github-common.sh
169-
source "${SCRIPT_DIR}/../../lib/github-common.sh"
170-
```
171-
172-
### Sourcing the shared library
173-
174-
Always use `SCRIPT_DIR` to build the path to `lib/github-common.sh`. The library is two directory levels up from any script (`../../lib/github-common.sh`). Never hardcode absolute paths.
175-
176-
### Authentication
177-
178-
- Standard org/repo REST calls: `Authorization: token ${GITHUB_TOKEN}`
179-
- Enterprise endpoints and GraphQL: `Authorization: Bearer ${GITHUB_TOKEN}`
180-
- The `gh_api` helper in `lib/github-common.sh` always uses Bearer; pass `"bearer"` to `validate_github_token` for enterprise scripts
181-
182-
### Shared library helpers
183-
184-
| Function | Purpose |
185-
|----------|---------|
186-
| `print_status` / `print_success` / `print_warning` / `print_error` | Colored output |
187-
| `err <message>` | `print_error` + `exit 1` in one call |
188-
| `require_env_var <VAR>` | Exit with message if variable unset/empty |
189-
| `require_command <cmd>` | Exit if binary not in PATH |
190-
| `configure_gh_auth [scope_hint]` | Bridge GITHUB_TOKEN→GH_TOKEN or verify gh auth session |
191-
| `validate_github_token [bearer]` | Verify GITHUB_TOKEN via /user endpoint |
192-
| `validate_slug <value> <label>` | Reject values with non-alphanumeric/hyphen/underscore chars |
193-
| `gh_api <path> [--api-version V] [curl args...]` | Bearer-auth REST helper with 5-retry rate-limit handling; optional `--api-version` overrides the default `2022-11-28` header; returns literal `__404__` or `__422__` for those HTTP statuses — callers must check for these sentinels |
194-
| `gh_api_paginate <path> [filter] [version]` | Paginated REST helper, follows Link headers, streams items; returns silently with empty output on 404/422 |
195-
| `get_enterprise_orgs` | Three-tier enterprise org resolver (REST → GraphQL → /user/orgs) |
196-
| `get_repo_page_count <url>` | Returns total pages for a paginated endpoint |
197-
198-
### Error handling sequence
199-
200-
1. `require_env_var` all required variables
201-
2. `validate_github_token` (or `validate_token` for secondary tokens)
202-
3. Validate additional inputs with `validate_slug` where needed
203-
4. Proceed with main logic
204-
205-
> **Note on token auto-resolution:** Sourcing `lib/github-common.sh` automatically populates `GITHUB_TOKEN` from an active `gh` auth session if the variable is unset. This means `require_env_var GITHUB_TOKEN` may pass even when no explicit token was provided by the caller — the token was silently resolved from `gh auth token`. Script headers should document `GITHUB_TOKEN` as "Required (or provided by an active gh auth session)". Scripts that use the `gh` CLI instead of `curl` should call `configure_gh_auth` instead of step 2 above.
206-
207-
### Pagination (REST)
208-
209-
```bash
210-
for PAGE in $(seq "$(get_repo_page_count "${API_URL_PREFIX}/orgs/${ORG}/repos?per_page=100")"); do
211-
# process page
212-
done
213-
```
214-
215-
### Rate limiting
216-
217-
- Repo-level operations (permission grants, archival): `sleep 5` between each repo
218-
- Code search: configurable `SEARCH_SLEEP` (default 2s) and `CONTENT_SLEEP` (default 1s)
219-
- `gh_api` auto-retries on HTTP 403/429 with 60s sleep
220-
- `gh_api` returns the literal string `__404__` or `__422__` (exit 0) for those HTTP statuses — callers must check for these sentinels before passing output to `jq`
221-
222-
---
223-
224-
## Adding a New Script
225-
226-
1. **Create the directory:** `<domain>/github-<verb-noun>/`
227-
2. **Create the script:** `github-<verb-noun>.sh` (name must match the directory)
228-
3. **Copy the header template** from Script Anatomy above — fill in description, all env vars, requirements
229-
4. **Source the shared library** using `SCRIPT_DIR`
230-
5. **Validate all inputs** before any API calls
231-
6. **Create `action.yml`** in the same directory — expose every env var as an input (required inputs first, optional inputs with defaults); map CLI flags (`--dry-run`, `--type`, etc.) to boolean/string inputs and construct the `ARGS` array in the `run:` step. See existing `action.yml` files for the pattern.
232-
7. **Add tests to `tests/test_script_validation.bats`** — add a labelled section (`# ═══ github-<name> ═══`) with tests for: every required env var missing (exit 1), unknown CLI args (exit 1), `--help` exits 0, and any script-specific validation (enum guards, URL allowlists, positional args). See existing sections for the pattern.
233-
8. **Add to README.md** — follow the existing format: use case, env var table, usage example, output format; add a row to the Available Actions table in the "Using Scripts in GitHub Actions" section
234-
9. Place in the correct domain:
235-
- `org-admin/` — organization-level operations (repos, teams, members)
236-
- `enterprise/` — enterprise-level operations (licenses, org enumeration)
237-
- `reporting/` — read-only reports and audits
238-
- `personal/` — personal GitHub utilities (stars, profile)
139+
> The full reference for script anatomy, shared library helpers, authentication,
140+
> pagination, and adding new scripts lives in
141+
> [`.github/copilot-instructions.md`](.github/copilot-instructions.md).
142+
> The sections below cover only what is unique to this guide.
239143
240144
---
241145

@@ -244,7 +148,7 @@ done
244148
- **Pre-commit hook:** `.githooks/pre-commit` — runs gitleaks + shellcheck on staged `.sh` files
245149
- **Install:** `./install-hooks.sh` or `git config core.hooksPath .githooks`
246150
- **Bypass (emergency only):** `git commit --no-verify`
247-
- **CI:** shellcheck runs on all `.sh` files on every PR (`.github/workflows/ci.yml`); bats unit tests run in a dedicated `test` job (`bats tests/`)
151+
- **CI:** shellcheck + bats unit tests run together on every PR (`.github/workflows/ci.yml`)
248152
- **Releases:** automated by Release Please (`.github/workflows/release.yml`) — pushes to `main` trigger a release PR; merging it publishes the GitHub Release and tag
249153

250154
## Commit Messages — Conventional Commits (required)

enterprise/github-add-enterprise-team-read-permissions/github-add-enterprise-team-read-permissions.sh

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -138,8 +138,7 @@ print_status "Fetching organizations in enterprise: ${ENTERPRISE}..."
138138
mapfile -t ORG_LIST < <(get_enterprise_orgs)
139139

140140
if [ "${#ORG_LIST[@]}" -eq 0 ]; then
141-
print_error "No organizations found in enterprise '${ENTERPRISE}'. Check the enterprise slug and token permissions."
142-
exit 1
141+
err "No organizations found in enterprise '${ENTERPRISE}'. Check the enterprise slug and token permissions."
143142
fi
144143

145144
print_status "Found ${#ORG_LIST[@]} organization(s)"

enterprise/github-dockerfile-discovery/github-dockerfile-discovery.sh

Lines changed: 4 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -350,17 +350,15 @@ main() {
350350
fi
351351

352352
if [ "${#orgs[@]}" -eq 0 ]; then
353-
print_error "No organisations found. Check ENTERPRISE slug and token permissions."
354-
exit 1
353+
err "No organisations found. Check ENTERPRISE slug and token permissions."
355354
fi
356355

357356
# ── Apply ORG_FILTER (inclusion) if set ────────────────────────────────
358357
if [ -n "${ORG_FILTER}" ]; then
359358
local _rc=0
360359
echo "" | grep -qE "${ORG_FILTER}" 2>/dev/null || _rc=$?
361360
if [ "${_rc}" -eq 2 ]; then
362-
print_error "ORG_FILTER is not a valid ERE regex: ${ORG_FILTER}"
363-
exit 1
361+
err "ORG_FILTER is not a valid ERE regex: ${ORG_FILTER}"
364362
fi
365363
local filtered=()
366364
for org in "${orgs[@]}"; do
@@ -378,8 +376,7 @@ main() {
378376
local _rc=0
379377
echo "" | grep -qE "${ORG_EXCLUDE}" 2>/dev/null || _rc=$?
380378
if [ "${_rc}" -eq 2 ]; then
381-
print_error "ORG_EXCLUDE is not a valid ERE regex: ${ORG_EXCLUDE}"
382-
exit 1
379+
err "ORG_EXCLUDE is not a valid ERE regex: ${ORG_EXCLUDE}"
383380
fi
384381
local kept=()
385382
for org in "${orgs[@]}"; do
@@ -393,8 +390,7 @@ main() {
393390
fi
394391

395392
if [ "${#orgs[@]}" -eq 0 ]; then
396-
print_error "No organisations remain after applying ORG_FILTER/ORG_EXCLUDE filters."
397-
exit 1
393+
err "No organisations remain after applying ORG_FILTER/ORG_EXCLUDE filters."
398394
fi
399395
print_success "Found ${#orgs[@]} organisation(s): ${orgs[*]}"
400396

enterprise/github-get-public-repos/github-get-public-repos.sh

Lines changed: 3 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -78,8 +78,7 @@ resolve_orgs() {
7878
local _rc=0
7979
echo "" | grep -qE "${ORG_FILTER}" 2>/dev/null || _rc=$?
8080
if [ "${_rc}" -eq 2 ]; then
81-
print_error "ORG_FILTER is not a valid ERE regex: ${ORG_FILTER}"
82-
exit 1
81+
err "ORG_FILTER is not a valid ERE regex: ${ORG_FILTER}"
8382
fi
8483
raw_orgs=$(echo "${raw_orgs}" | grep -E "${ORG_FILTER}" || true)
8584
fi
@@ -89,8 +88,7 @@ resolve_orgs() {
8988
local _rc=0
9089
echo "" | grep -qE "${ORG_EXCLUDE}" 2>/dev/null || _rc=$?
9190
if [ "${_rc}" -eq 2 ]; then
92-
print_error "ORG_EXCLUDE is not a valid ERE regex: ${ORG_EXCLUDE}"
93-
exit 1
91+
err "ORG_EXCLUDE is not a valid ERE regex: ${ORG_EXCLUDE}"
9492
fi
9593
raw_orgs=$(echo "${raw_orgs}" | grep -Ev "${ORG_EXCLUDE}" || true)
9694
fi
@@ -186,8 +184,7 @@ main() {
186184
orgs=$(resolve_orgs)
187185

188186
if [ -z "${orgs}" ]; then
189-
print_error "No organisations found. Check your ENTERPRISE/ORGS/ORG_FILTER settings."
190-
exit 1
187+
err "No organisations found. Check your ENTERPRISE/ORGS/ORG_FILTER settings."
191188
fi
192189

193190
local org_count

enterprise/github-install-enterprise-app/github-install-enterprise-app.sh

Lines changed: 6 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -76,8 +76,7 @@ for arg in "$@"; do
7676
exit 0
7777
;;
7878
*)
79-
print_error "Unknown option: ${arg}"
80-
exit 1
79+
err "Unknown option: ${arg}"
8180
;;
8281
esac
8382
done
@@ -100,8 +99,7 @@ validate_slug "${INSTALLER_APP_INSTALL_ID}" "installer app installation ID"
10099
case "${REPO_SELECTION}" in
101100
all|selected) ;;
102101
*)
103-
print_error "REPO_SELECTION must be 'all' or 'selected' (got '${REPO_SELECTION}')"
104-
exit 1
102+
err "REPO_SELECTION must be 'all' or 'selected' (got '${REPO_SELECTION}')"
105103
;;
106104
esac
107105

@@ -111,12 +109,10 @@ INSTALLER_APP_PRIVATE_KEY="${INSTALLER_APP_PRIVATE_KEY/#\~/${HOME}}"
111109
AUTOMATION_APP_PRIVATE_KEY="${AUTOMATION_APP_PRIVATE_KEY/#\~/${HOME}}"
112110

113111
if [ ! -r "${INSTALLER_APP_PRIVATE_KEY}" ]; then
114-
print_error "Installer app private key not readable: ${INSTALLER_APP_PRIVATE_KEY}"
115-
exit 1
112+
err "Installer app private key not readable: ${INSTALLER_APP_PRIVATE_KEY}"
116113
fi
117114
if [ -n "${AUTOMATION_APP_PRIVATE_KEY}" ] && [ ! -r "${AUTOMATION_APP_PRIVATE_KEY}" ]; then
118-
print_error "Automation app private key not readable: ${AUTOMATION_APP_PRIVATE_KEY}"
119-
exit 1
115+
err "Automation app private key not readable: ${AUTOMATION_APP_PRIVATE_KEY}"
120116
fi
121117

122118
###
@@ -233,8 +229,7 @@ case "${INSTALL_HTTP_CODE}" in
233229
;;
234230
*)
235231
INSTALL_MESSAGE=$(echo "${INSTALL_BODY}" | jq -r '.message // empty' 2>/dev/null || true)
236-
print_error "Failed to install automation app (HTTP ${INSTALL_HTTP_CODE}): ${INSTALL_MESSAGE:-unknown error}"
237-
exit 1
232+
err "Failed to install automation app (HTTP ${INSTALL_HTTP_CODE}): ${INSTALL_MESSAGE:-unknown error}"
238233
;;
239234
esac
240235

@@ -256,6 +251,5 @@ if mint_installation_token \
256251
"${AUTOMATION_APP_INSTALL_ID}" > /dev/null; then
257252
print_success "Automation app authenticated successfully in '${ORG}'. Installation is live."
258253
else
259-
print_error "Could not authenticate the automation app after install."
260-
exit 1
254+
err "Could not authenticate the automation app after install."
261255
fi

org-admin/github-add-repo-collaborators-by-pattern/github-add-repo-collaborators-by-pattern.sh

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -59,8 +59,7 @@ validate() {
5959
case "${PERMISSION}" in
6060
pull|triage|push|maintain|admin) ;;
6161
*)
62-
print_error "Invalid PERMISSION '${PERMISSION}'. Must be one of: pull, triage, push, maintain, admin"
63-
exit 1
62+
err "Invalid PERMISSION '${PERMISSION}'. Must be one of: pull, triage, push, maintain, admin"
6463
;;
6564
esac
6665
require_command jq

org-admin/github-add-repo-permissions/github-add-repo-permissions.sh

Lines changed: 2 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -58,9 +58,7 @@ require_command jq
5858

5959
# Check if at least one permission level is set
6060
if [ -z "${REPO_ADMIN}" ] && [ -z "${REPO_MAINTAIN}" ] && [ -z "${REPO_PUSH}" ] && [ -z "${REPO_TRIAGE}" ] && [ -z "${REPO_PULL}" ]; then
61-
print_error "At least one permission level must be set."
62-
print_error "Available variables: REPO_ADMIN, REPO_MAINTAIN, REPO_PUSH, REPO_TRIAGE, REPO_PULL"
63-
exit 1
61+
err "At least one permission level must be set. Available: REPO_ADMIN, REPO_MAINTAIN, REPO_PUSH, REPO_TRIAGE, REPO_PULL"
6462
fi
6563

6664
validate_github_token
@@ -103,8 +101,7 @@ process_repos () {
103101

104102
if ! echo "${repos_json}" | jq -e 'type == "array"' > /dev/null 2>&1; then
105103
print_error "Unexpected API response for page ${PAGE}"
106-
print_error "$(echo "${repos_json}" | jq -r '.message // "unknown error"')"
107-
exit 1
104+
err "$(echo "${repos_json}" | jq -r '.message // "unknown error"')"
108105
fi
109106

110107
while IFS= read -r REPO; do

0 commit comments

Comments
 (0)