Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,16 @@ are a separate namespace — see [README](README.md#releases) and
per-step verification and create-tag coverage.
- [endpoint-contract.md](docs/endpoint-contract.md) Outbound Weblate section:
request schema is now the source of truth for payload fields.
- `scripts/trigger-add-submodules.sh` now requires `--submodules`; the
`unordered, json` script default is removed so the operator script cannot
silently diverge from raw API auto-discovery.
- Bats coverage for `scripts/trigger-dispatch-common.sh`, `trigger-add-submodules.sh`,
and `trigger-start-translation.sh`.

### Fixed

- `build_dispatch_json` in `trigger-dispatch-common.sh`: avoid jq 1.7 naming
collision with the built-in `pairs` function.
- Clone and finalize steps now fail fast: **`clone_repo`**, **`sync_translations_branch`**,
and **`finalize_translations_*`** propagate non-zero status instead of continuing with
incomplete state.
Expand Down
19 changes: 9 additions & 10 deletions docs/GETTING-STARTED.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,8 @@ linked README sections for scope, format, and which workflows consume each value

## 1. Local trigger setup (optional)

To fire dispatches from a clone of this repo instead of the GitHub API or UI:
To fire dispatches from a clone of this repo instead of the GitHub API
(`POST …/dispatches` or `gh api`):

```bash
cp .env.example .env # set GH_TOKEN (GITHUB_TOKEN is also accepted)
Expand Down Expand Up @@ -117,14 +118,12 @@ scripts/trigger-add-submodules.sh \
```

- Omit **`--lang-codes`** to use repository variable **`LANG_CODES`**.
- Omit **`--submodules`** to use the script default **`DEFAULT_SUBMODULES`**
(`unordered, json`; see
[trigger-add-submodules.sh](../scripts/trigger-add-submodules.sh#L40)) — **not**
auto-discovery from **`boostorg/boost`**. The script substitutes that default
before building the payload
([`SUBMODULES` assignment](../scripts/trigger-add-submodules.sh#L99)), so `client_payload.submodules`
is never omitted. Full discovery runs only when the workflow receives a dispatch
**without** a `submodules` field (raw API / GitHub UI).
- **`--submodules` is required** when using this script (comma-separated library
names). To process all Boost libraries, omit `submodules` from a **raw**
`repository_dispatch` payload sent via the GitHub API (for example
`POST …/dispatches` or `gh api repos/{owner}/{repo}/dispatches`) — the workflow
then auto-discovers the full list from **`boostorg/boost`** `.gitmodules` via
**`resolve_add_submodules_names`**.

### Expected outcome

Expand Down Expand Up @@ -238,7 +237,7 @@ Runs automatically **daily at 00:00 UTC** (`0 0 * * *`) or on manual dispatch.
| Item | Value |
| -------- | --------------------------------------------------------------------- |
| Workflow | [`.github/workflows/sync-translation.yml`](../.github/workflows/sync-translation.yml) |
| Script | **None** — use the dispatches API, GitHub UI, or the curl example below |
| Script | **None** — use the dispatches API or the curl example below |
| Trigger | `repository_dispatch` with `event_type: sync-translation` (no `client_payload`) |

### Trigger (JSON)
Expand Down
2 changes: 1 addition & 1 deletion docs/endpoint-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ submodule pointer updates, and Boost release refs in `client_payload.version`.
| Workflow | `.github/workflows/add-submodules.yml` |
| Body shape | `{"event_type":"add-submodules","client_payload":{...}}` |
| `client_payload` | All optional: `version`, `submodules` (list-like string), `lang_codes` (comma-separated). See [README](../README.md). |
| Script | `scripts/trigger-add-submodules.sh` builds JSON with `jq` or Python; omits empty optional fields. Both trigger scripts source shared [`scripts/trigger-dispatch-common.sh`](../scripts/trigger-dispatch-common.sh). |
| Script | `scripts/trigger-add-submodules.sh` builds JSON with `jq` or Python; omits empty optional fields; **`--submodules` is required** on the script even though the API field is optional. Both trigger scripts source shared [`scripts/trigger-dispatch-common.sh`](../scripts/trigger-dispatch-common.sh). |

### `event_type: start-translation`

Expand Down
36 changes: 22 additions & 14 deletions scripts/trigger-add-submodules.sh
100644 → 100755
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
#
# Usage:
# scripts/trigger-add-submodules.sh [--repo OWNER/NAME] [--token PAT] \
# [--version REF] [--submodules 'a, b'] [--lang-codes zh_Hans,ja]
# --submodules 'a, b' [--version REF] [--lang-codes zh_Hans,ja]
#
# If --repo is omitted: GITHUB_REPOSITORY, then git origin, then DEFAULT_REPO below.

Expand All @@ -34,18 +34,13 @@ source "$_ASSETS_DIR/lib.sh"
source "$_REPO_ROOT/scripts/trigger-dispatch-common.sh"
unset _REPO_ROOT _ASSETS_DIR

# ---------------------------------------------------------------------------
# Typical run — edit these. CLI flags override (except --token uses env/PAT).
# ---------------------------------------------------------------------------
DEFAULT_SUBMODULES="unordered, json"

usage() {
cat <<'EOF'
Trigger add-submodules.yml via repository_dispatch (POST .../dispatches).

Usage:
scripts/trigger-add-submodules.sh [--repo OWNER/NAME] [--token PAT] \
[--version REF] [--submodules 'a, b'] [--lang-codes zh_Hans,ja]
--submodules 'a, b' [--version REF] [--lang-codes zh_Hans,ja]

Requires: curl; jq or Python 3 (python3 / python)
Auth: .env (GH_TOKEN), GH_TOKEN / GITHUB_TOKEN in env, or --token (needs repo scope on the target).
Expand All @@ -54,7 +49,7 @@ Options:
--repo OWNER/REPO Target repository (default: GITHUB_REPOSITORY, then origin, then DEFAULT_REPO)
--token PAT GitHub token
--version REF Boost ref; default DEFAULT_VERSION in script
--submodules LIST default DEFAULT_SUBMODULES in script
--submodules LIST comma-separated library names (required)
--lang-codes CSV optional; omit → workflow uses repo vars.LANG_CODES
EOF
}
Expand All @@ -65,18 +60,22 @@ VERSION=""
SUBMODULES=""
LANG_CODES=""

shift_pair_into() {
printf -v "$1" '%s' "${2:-}"
}

while [[ $# -gt 0 ]]; do
case "$1" in
--repo)
REPO="${2:-}"; shift 2 || exit 1 ;;
shift_pair_into REPO "$2"; shift 2 || exit 1 ;;
--token)
TOKEN="${2:-}"; shift 2 || exit 1 ;;
shift_pair_into TOKEN "$2"; shift 2 || exit 1 ;;
--version)
VERSION="${2:-}"; shift 2 || exit 1 ;;
shift_pair_into VERSION "$2"; shift 2 || exit 1 ;;
--submodules)
SUBMODULES="${2:-}"; shift 2 || exit 1 ;;
shift_pair_into SUBMODULES "$2"; shift 2 || exit 1 ;;
--lang-codes)
LANG_CODES="${2:-}"; shift 2 || exit 1 ;;
shift_pair_into LANG_CODES "$2"; shift 2 || exit 1 ;;
-h|--help)
usage; exit 0 ;;
*)
Expand All @@ -86,6 +85,16 @@ while [[ $# -gt 0 ]]; do
esac
done

# Trim leading/trailing whitespace; reject empty or whitespace-only values.
SUBMODULES="${SUBMODULES#"${SUBMODULES%%[![:space:]]*}"}"
SUBMODULES="${SUBMODULES%"${SUBMODULES##*[![:space:]]}"}"

if [[ -z "$SUBMODULES" ]]; then
echo "error: --submodules is required (comma-separated library names)" >&2
usage >&2
exit 1
fi
Comment thread
whisper67265 marked this conversation as resolved.

require_curl || exit 1

TOKEN="$(resolve_trigger_token "$TOKEN")" || exit 1
Expand All @@ -96,7 +105,6 @@ REPO="$(resolve_trigger_repo "$REPO")" || {
}

VERSION="${VERSION:-$DEFAULT_VERSION}"
SUBMODULES="${SUBMODULES:-$DEFAULT_SUBMODULES}"

validate_event_type "$EVENT_ADD_SUBMODULES"

Expand Down
25 changes: 10 additions & 15 deletions scripts/trigger-dispatch-common.sh
Original file line number Diff line number Diff line change
Expand Up @@ -67,25 +67,20 @@ build_dispatch_json() {
local event_type="$1"
shift
if command -v jq >/dev/null 2>&1; then
local json_pairs='[]' key val
while [[ $# -gt 0 ]]; do
key="$1"
val="$2"
shift 2
json_pairs="$(jq -n --argjson arr "$json_pairs" --arg k "$key" --arg v "$val" \
'$arr + [{key: $k, value: $v}]')"
done
jq -n --arg event_type "$event_type" --argjson pairs "$json_pairs" \
jq -n --arg event_type "$event_type" --args \
'{
event_type: $event_type,
client_payload: (
{}
| reduce pairs[] as $p (
.;
if ($p.value | length) > 0 then . + {($p.key): $p.value} else . end
)
reduce range(0; $ARGS.positional | length; 2) as $i (
{};
if ($ARGS.positional[$i + 1] | length) > 0
then . + {($ARGS.positional[$i]): $ARGS.positional[$i + 1]}
else .
end
)
)
}'
}' \
"$@"
return 0
fi
local py=""
Expand Down
162 changes: 146 additions & 16 deletions tests/helpers/http_mock.bash
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,17 @@ MOCK_WEBLATE_REQUEST_LOG=""
MOCK_WEBLATE_SERVER_SCRIPT=""
_CURL_ORIG_PATH=""

common_setup() {
local root_dir
root_dir="$(cd "$BATS_TEST_DIRNAME/.." && pwd)"
# shellcheck disable=SC2034
export ROOT="$root_dir"
}

common_teardown() {
restore_dispatch_curl_stub
}

start_weblate_mock_server() {
local status="${MOCK_WEBLATE_STATUS:-202}"
local body="${MOCK_WEBLATE_BODY:-{\"status\":\"accepted\"}}"
Expand Down Expand Up @@ -64,7 +75,7 @@ PYEOF
python3 "$MOCK_WEBLATE_SERVER_SCRIPT" &
MOCK_WEBLATE_PID=$!

local i=0 port=""
local i=0 port="" sleep_sec=0.05
while [[ -z "$port" ]]; do
if [[ -s "$MOCK_WEBLATE_PORT_FILE" ]]; then
port="$(<"$MOCK_WEBLATE_PORT_FILE")"
Expand All @@ -84,7 +95,8 @@ PYEOF
echo "http_mock: mock server failed to start" >&2
return 1
fi
sleep 0.05
sleep "$sleep_sec"
sleep_sec="$(awk "BEGIN {v=$sleep_sec*2; print (v>0.5?0.5:v)}")"
done

MOCK_WEBLATE_PORT="$port"
Expand All @@ -109,18 +121,20 @@ stop_weblate_mock_server() {
unset MOCK_WEBLATE_STATUS MOCK_WEBLATE_BODY MOCK_WEBLATE_DELAY_SEC
}

install_curl_timeout_stub() {
install_curl_stub
}

install_curl_stub() {
_init_curl_wrapper_dir() {
_CURL_ORIG_PATH="$PATH"
local wrapper_dir
wrapper_dir="$(mktemp -d)"
CURL_WRAPPER_DIR="$(mktemp -d)"
REAL_CURL="$(command -v curl)"
export REAL_CURL
cat >"$wrapper_dir/curl" <<'EOF'
#!/usr/bin/env bash
export REAL_CURL CURL_WRAPPER_DIR
}

_activate_curl_wrapper() {
chmod +x "$CURL_WRAPPER_DIR/curl"
export PATH="$CURL_WRAPPER_DIR:$PATH"
}

_curl_stub_preamble() {
cat <<'STUB_EOF'
if [[ -n "${MOCK_CURL_EXIT:-}" ]]; then
echo "mock curl: simulated exit ${MOCK_CURL_EXIT}" >&2
exit "$MOCK_CURL_EXIT"
Expand All @@ -129,11 +143,27 @@ if [[ "${MOCK_CURL_TIMEOUT:-}" == "1" ]]; then
echo "mock curl: simulated timeout" >&2
exit 28
fi
STUB_EOF
}

_curl_stub_exec_line() {
cat <<'STUB_EOF'
exec "$REAL_CURL" "$@"
EOF
chmod +x "$wrapper_dir/curl"
CURL_WRAPPER_DIR="$wrapper_dir"
export PATH="$wrapper_dir:$PATH"
STUB_EOF
}

install_curl_timeout_stub() {
install_curl_stub
}

install_curl_stub() {
_init_curl_wrapper_dir
{
echo '#!/usr/bin/env bash'
_curl_stub_preamble
_curl_stub_exec_line
} >"$CURL_WRAPPER_DIR/curl"
_activate_curl_wrapper
}

restore_curl_stub() {
Expand All @@ -145,3 +175,103 @@ restore_curl_stub() {
fi
unset CURL_WRAPPER_DIR REAL_CURL _CURL_ORIG_PATH MOCK_CURL_TIMEOUT MOCK_CURL_EXIT
}

# Intercept GitHub repository_dispatch POST (no real network).
install_dispatch_curl_stub() {
_init_curl_wrapper_dir
MOCK_DISPATCH_REQUEST_LOG="${MOCK_DISPATCH_REQUEST_LOG:-$(mktemp)}"
export MOCK_DISPATCH_REQUEST_LOG
MOCK_DISPATCH_STATUS="${MOCK_DISPATCH_STATUS:-204}"
export MOCK_DISPATCH_STATUS
MOCK_DISPATCH_RESPONSE_BODY="${MOCK_DISPATCH_RESPONSE_BODY:-}"
export MOCK_DISPATCH_RESPONSE_BODY
{
echo '#!/usr/bin/env bash'
_curl_stub_preamble
cat <<'EOF'
args=("$@")
url=""
method=""
body_file=""
body_inline=""
out_file=""
write_out=""
i=0
while [[ $i -lt ${#args[@]} ]]; do
arg="${args[$i]}"
case "$arg" in
-X)
method="${args[$((i + 1))]}"
i=$((i + 2))
;;
-d)
next="${args[$((i + 1))]}"
if [[ -f "$next" ]]; then
body_file="$next"
else
body_inline="$next"
fi
i=$((i + 2))
;;
-o)
out_file="${args[$((i + 1))]}"
i=$((i + 2))
;;
-w)
write_out="${args[$((i + 1))]}"
i=$((i + 2))
;;
http://*|https://*)
url="$arg"
i=$((i + 1))
;;
*)
i=$((i + 1))
;;
esac
done
if [[ "$url" == *"api.github.com/repos/"*/dispatches && "$method" == "POST" ]]; then
{
echo "URL=$url"
echo "METHOD=$method"
if [[ -n "$body_file" ]]; then
echo "BODY_START"
cat "$body_file"
echo ""
echo "BODY_END"
elif [[ -n "$body_inline" ]]; then
echo "BODY_START"
printf '%s\n' "$body_inline"
echo "BODY_END"
fi
} >"$MOCK_DISPATCH_REQUEST_LOG"
if [[ -n "$out_file" ]]; then
if [[ -n "${MOCK_DISPATCH_RESPONSE_BODY:-}" ]]; then
printf '%s' "$MOCK_DISPATCH_RESPONSE_BODY" >"$out_file"
else
: >"$out_file"
fi
fi
if [[ -n "$write_out" ]]; then
printf '%s' "$MOCK_DISPATCH_STATUS"
fi
exit 0
fi
exec "$REAL_CURL" "$@"
EOF
} >"$CURL_WRAPPER_DIR/curl"
_activate_curl_wrapper
}

restore_dispatch_curl_stub() {
restore_curl_stub
if [[ -n "${MOCK_DISPATCH_REQUEST_LOG:-}" && -f "$MOCK_DISPATCH_REQUEST_LOG" ]]; then
rm -f "$MOCK_DISPATCH_REQUEST_LOG"
fi
unset MOCK_DISPATCH_REQUEST_LOG MOCK_DISPATCH_STATUS MOCK_DISPATCH_RESPONSE_BODY
}

extract_dispatch_request_body() {
local log="${1:?}"
awk '/^BODY_START$/{found=1; next} /^BODY_END$/{found=0} found' "$log"
}
Loading