Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions .github/workflows/assets/lib.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,10 @@
# Shared shell library for add-submodules and start-translation workflows.
# Source env.sh before lib.sh so ORG, MODULE_ORG, BOT_NAME, BOT_EMAIL, BOOST_ORG, MASTER_BRANCH,
# LOCAL_BRANCH_PREFIX, TRANSLATION_BRANCH_PREFIX, WEBLATE_ENDPOINT_PATH, and TRANSLATIONS_REPO
# are set. Workflows also set GITHUB_TOKEN, LANG_CODES, and (for start-translation)
# WEBLATE_URL / WEBLATE_TOKEN in the step env before sourcing.
# are set. Workflows also set GITHUB_TOKEN (from secrets.SYNC_TOKEN — broad PAT),
# LANG_CODES, and (for start-translation) WEBLATE_URL / WEBLATE_TOKEN in the step
# env before sourcing. Some workflows set GH_TOKEN to the ephemeral Actions token
# (e.g. heartbeat.yml for gh CLI); that is not the same privilege level as GITHUB_TOKEN.
# Call validate_secrets (or validate_secrets weblate) after sourcing env.sh and lib.sh.
#
# Per-submodule batch return convention (see docs/ARCHITECTURE.md §6):
Expand Down Expand Up @@ -540,7 +542,7 @@ validate_secrets() {
local require_weblate=0
[[ "${1:-}" == "weblate" ]] && require_weblate=1

_require_nonempty GITHUB_TOKEN "SYNC_TOKEN secret is not set."
_require_nonempty GITHUB_TOKEN "SYNC_TOKEN secret is not set (mapped to GITHUB_TOKEN in workflow env)."
if [[ -n "${LANG_CODE:-}" ]]; then
validate_lang_codes "$LANG_CODE"
elif [[ -n "${LANG_CODES:-}" ]]; then
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/heartbeat.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ jobs:

- name: Check the last successful scheduled sync
env:
# Ephemeral Actions token for gh CLI — not the SYNC_TOKEN PAT in GITHUB_TOKEN elsewhere.
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
Expand Down
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,17 @@ are a separate namespace — see [README](README.md#releases) and
per-step verification and create-tag coverage.
- [endpoint-contract.md](docs/endpoint-contract.md) Outbound Weblate section:
request schema is now the source of truth for payload fields.
- `scripts/trigger-add-submodules.sh` now requires `--submodules`; the
`unordered, json` script default is removed so the operator script cannot
silently diverge from raw API auto-discovery.
- Bats coverage for `scripts/trigger-dispatch-common.sh`, `trigger-add-submodules.sh`,
and `trigger-start-translation.sh`.

### Fixed

- `build_dispatch_json` in `trigger-dispatch-common.sh`: pass key/value pairs
positionally via jq `--args` and `$ARGS.positional` instead of building an
intermediate array.
- Clone and finalize steps now fail fast: **`clone_repo`**, **`sync_translations_branch`**,
and **`finalize_translations_*`** propagate non-zero status instead of continuing with
incomplete state.
Expand Down
5 changes: 4 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,10 @@ These wrappers use only **`exit 0`** (success, including **`--help`**) and **`ex
Copy **`.env.example`** to **`.env`** and set **`GH_TOKEN`** (or **`GITHUB_TOKEN`**)
with permission to call **`POST /repos/{owner}/{repo}/dispatches`** on the target
repo. The workflows still use GitHub **secrets** and **variables** on the server as
documented below.
documented below. **Workflow jobs** map tokens differently: **`SYNC_TOKEN`** (broad
PAT) is set as **`GITHUB_TOKEN`**; the ephemeral Actions token used by **`gh`** (e.g.
in **`heartbeat.yml`**) is set as **`GH_TOKEN`**. Those names are interchangeable
only in the local trigger scripts, not across client and workflow contexts.

---

Expand Down
23 changes: 13 additions & 10 deletions docs/GETTING-STARTED.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,8 @@ linked README sections for scope, format, and which workflows consume each value

## 1. Local trigger setup (optional)

To fire dispatches from a clone of this repo instead of the GitHub API or UI:
To fire dispatches from a clone of this repo instead of the GitHub API
(`POST …/dispatches` or `gh api`):

```bash
cp .env.example .env # set GH_TOKEN (GITHUB_TOKEN is also accepted)
Expand All @@ -75,6 +76,10 @@ cp .env.example .env # set GH_TOKEN (GITHUB_TOKEN is also accepted)
- **`GH_TOKEN`** is **client-side only** — permission to call
`POST /repos/{owner}/{repo}/dispatches`. Workflows still use the GitHub
**secrets** from step 0 on the server.
- In **workflow jobs**, naming is inverted relative to the client scripts:
the broad **`SYNC_TOKEN`** PAT is exported as **`GITHUB_TOKEN`**, while the
ephemeral Actions token (e.g. for `gh` in **`heartbeat.yml`**) is exported as
**`GH_TOKEN`**. The two names are interchangeable only on the client side.
- Requires **curl** and **jq** or Python 3.

See [README § Scripts](../README.md#scripts-local-repository_dispatch) for script
Expand Down Expand Up @@ -117,14 +122,12 @@ scripts/trigger-add-submodules.sh \
```

- Omit **`--lang-codes`** to use repository variable **`LANG_CODES`**.
- Omit **`--submodules`** to use the script default **`DEFAULT_SUBMODULES`**
(`unordered, json`; see
[trigger-add-submodules.sh](../scripts/trigger-add-submodules.sh#L40)) — **not**
auto-discovery from **`boostorg/boost`**. The script substitutes that default
before building the payload
([`SUBMODULES` assignment](../scripts/trigger-add-submodules.sh#L99)), so `client_payload.submodules`
is never omitted. Full discovery runs only when the workflow receives a dispatch
**without** a `submodules` field (raw API / GitHub UI).
- **`--submodules` is required** when using this script (comma-separated library
names). To process all Boost libraries, omit `submodules` from a **raw**
`repository_dispatch` payload sent via the GitHub API (for example
`POST …/dispatches` or `gh api repos/{owner}/{repo}/dispatches`) — the workflow
then auto-discovers the full list from **`boostorg/boost`** `.gitmodules` via
**`resolve_add_submodules_names`**.

### Expected outcome

Expand Down Expand Up @@ -238,7 +241,7 @@ Runs automatically **daily at 00:00 UTC** (`0 0 * * *`) or on manual dispatch.
| Item | Value |
| -------- | --------------------------------------------------------------------- |
| Workflow | [`.github/workflows/sync-translation.yml`](../.github/workflows/sync-translation.yml) |
| Script | **None** — use the dispatches API, GitHub UI, or the curl example below |
| Script | **None** — use the dispatches API or the curl example below |
| Trigger | `repository_dispatch` with `event_type: sync-translation` (no `client_payload`) |

### Trigger (JSON)
Expand Down
2 changes: 1 addition & 1 deletion docs/endpoint-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ submodule pointer updates, and Boost release refs in `client_payload.version`.
| Workflow | `.github/workflows/add-submodules.yml` |
| Body shape | `{"event_type":"add-submodules","client_payload":{...}}` |
| `client_payload` | All optional: `version`, `submodules` (list-like string), `lang_codes` (comma-separated). See [README](../README.md). |
| Script | `scripts/trigger-add-submodules.sh` builds JSON with `jq` or Python; omits empty optional fields. Both trigger scripts source shared [`scripts/trigger-dispatch-common.sh`](../scripts/trigger-dispatch-common.sh). |
| Script | `scripts/trigger-add-submodules.sh` builds JSON with `jq` or Python; omits empty optional fields; **`--submodules` is required** on the script even though the API field is optional. Both trigger scripts source shared [`scripts/trigger-dispatch-common.sh`](../scripts/trigger-dispatch-common.sh). |

### `event_type: start-translation`

Expand Down
36 changes: 22 additions & 14 deletions scripts/trigger-add-submodules.sh
100644 → 100755
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
#
# Usage:
# scripts/trigger-add-submodules.sh [--repo OWNER/NAME] [--token PAT] \
# [--version REF] [--submodules 'a, b'] [--lang-codes zh_Hans,ja]
# --submodules 'a, b' [--version REF] [--lang-codes zh_Hans,ja]
#
# If --repo is omitted: GITHUB_REPOSITORY, then git origin, then DEFAULT_REPO below.

Expand All @@ -34,18 +34,13 @@ source "$_ASSETS_DIR/lib.sh"
source "$_REPO_ROOT/scripts/trigger-dispatch-common.sh"
unset _REPO_ROOT _ASSETS_DIR

# ---------------------------------------------------------------------------
# Typical run — edit these. CLI flags override (except --token uses env/PAT).
# ---------------------------------------------------------------------------
DEFAULT_SUBMODULES="unordered, json"

usage() {
cat <<'EOF'
Trigger add-submodules.yml via repository_dispatch (POST .../dispatches).

Usage:
scripts/trigger-add-submodules.sh [--repo OWNER/NAME] [--token PAT] \
[--version REF] [--submodules 'a, b'] [--lang-codes zh_Hans,ja]
--submodules 'a, b' [--version REF] [--lang-codes zh_Hans,ja]

Requires: curl; jq or Python 3 (python3 / python)
Auth: .env (GH_TOKEN), GH_TOKEN / GITHUB_TOKEN in env, or --token (needs repo scope on the target).
Expand All @@ -54,7 +49,7 @@ Options:
--repo OWNER/REPO Target repository (default: GITHUB_REPOSITORY, then origin, then DEFAULT_REPO)
--token PAT GitHub token
--version REF Boost ref; default DEFAULT_VERSION in script
--submodules LIST default DEFAULT_SUBMODULES in script
--submodules LIST comma-separated library names (required)
--lang-codes CSV optional; omit → workflow uses repo vars.LANG_CODES
EOF
}
Expand All @@ -65,18 +60,22 @@ VERSION=""
SUBMODULES=""
LANG_CODES=""

shift_pair_into() {
printf -v "$1" '%s' "${2:-}"
}

while [[ $# -gt 0 ]]; do
case "$1" in
--repo)
REPO="${2:-}"; shift 2 || exit 1 ;;
shift_pair_into REPO "$2"; shift 2 || exit 1 ;;
--token)
TOKEN="${2:-}"; shift 2 || exit 1 ;;
shift_pair_into TOKEN "$2"; shift 2 || exit 1 ;;
--version)
VERSION="${2:-}"; shift 2 || exit 1 ;;
shift_pair_into VERSION "$2"; shift 2 || exit 1 ;;
--submodules)
SUBMODULES="${2:-}"; shift 2 || exit 1 ;;
shift_pair_into SUBMODULES "$2"; shift 2 || exit 1 ;;
--lang-codes)
LANG_CODES="${2:-}"; shift 2 || exit 1 ;;
shift_pair_into LANG_CODES "$2"; shift 2 || exit 1 ;;
-h|--help)
usage; exit 0 ;;
*)
Expand All @@ -86,6 +85,16 @@ while [[ $# -gt 0 ]]; do
esac
done

# Trim leading/trailing whitespace; reject empty or whitespace-only values.
SUBMODULES="${SUBMODULES#"${SUBMODULES%%[![:space:]]*}"}"
SUBMODULES="${SUBMODULES%"${SUBMODULES##*[![:space:]]}"}"

if [[ -z "$SUBMODULES" ]]; then
echo "error: --submodules is required (comma-separated library names)" >&2
usage >&2
exit 1
fi
Comment thread
whisper67265 marked this conversation as resolved.

require_curl || exit 1

TOKEN="$(resolve_trigger_token "$TOKEN")" || exit 1
Expand All @@ -96,7 +105,6 @@ REPO="$(resolve_trigger_repo "$REPO")" || {
}

VERSION="${VERSION:-$DEFAULT_VERSION}"
SUBMODULES="${SUBMODULES:-$DEFAULT_SUBMODULES}"

validate_event_type "$EVENT_ADD_SUBMODULES"

Expand Down
40 changes: 20 additions & 20 deletions scripts/trigger-dispatch-common.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,11 @@ DEFAULT_REPO="cppalliance/boost-docs-translation"
DEFAULT_VERSION="boost-1.90.0"

infer_repo_from_git() {
local url root o r
root="$(git rev-parse --show-toplevel 2>/dev/null)" || return 1
url="$(git -C "$root" remote get-url origin 2>/dev/null)" || return 1
local url="${1:-}" root o r
if [[ -z "$url" ]]; then
root="$(git rev-parse --show-toplevel 2>/dev/null)" || return 1
url="$(git -C "$root" remote get-url origin 2>/dev/null)" || return 1
fi
if [[ "$url" =~ github\.com[:/]([^/]+)/([^[:space:]]+) ]]; then
o="${BASH_REMATCH[1]}"
r="${BASH_REMATCH[2]}"
Expand Down Expand Up @@ -44,6 +46,9 @@ resolve_trigger_repo() {
return 1
}

# Client-side dispatch auth: prefers GH_TOKEN, then GITHUB_TOKEN. Workflow jobs use a
# different mapping — SYNC_TOKEN PAT in GITHUB_TOKEN; ephemeral Actions token in GH_TOKEN
# (e.g. heartbeat.yml). The names are not interchangeable across those two contexts.
resolve_trigger_token() {
local explicit="${1:-}"
local token="${explicit:-${GH_TOKEN:-${GITHUB_TOKEN:-}}}"
Expand All @@ -67,26 +72,21 @@ build_dispatch_json() {
local event_type="$1"
shift
if command -v jq >/dev/null 2>&1; then
local json_pairs='[]' key val
while [[ $# -gt 0 ]]; do
key="$1"
val="$2"
shift 2
json_pairs="$(jq -n --argjson arr "$json_pairs" --arg k "$key" --arg v "$val" \
'$arr + [{key: $k, value: $v}]')"
done
jq -n --arg event_type "$event_type" --argjson pairs "$json_pairs" \
jq -n --arg event_type "$event_type" --args \
'{
event_type: $event_type,
client_payload: (
{}
| reduce pairs[] as $p (
.;
if ($p.value | length) > 0 then . + {($p.key): $p.value} else . end
)
reduce range(0; $ARGS.positional | length; 2) as $i (
{};
if ($ARGS.positional[$i + 1] | length) > 0
then . + {($ARGS.positional[$i]): $ARGS.positional[$i + 1]}
else .
end
)
)
}'
return 0
}' \
"$@"
return $?
fi
local py=""
command -v python3 >/dev/null 2>&1 && py="python3"
Expand All @@ -102,7 +102,7 @@ for i in range(0,len(pairs),2):
d[k]=v
print(json.dumps({"event_type":et,"client_payload":d}))' \
"$event_type" "$@"
return 0
return $?
fi
return 1
}
Expand Down
Loading