compose-pr: retry gh pr create with backoff and guard resilience (#4203)

## Summary

Add bounded shell-level retry around the Step 5 `gh pr create` command
in `src/autoskillit/skills_extended/compose-pr/SKILL.md`. The retry runs
only after `gh auth status` has passed, preserves the existing `pr_url =
...` output contract, retries only transient API or network failures,
and keeps the PR body closing-issue guard effective for the new command
shape. The remediation pass keeps HTTP 429 and secondary rate-limit
responses retryable, keeps `compose_pr_body_guard.py` effective for the
actual Step 5 `PR_CREATE_BODY` command shape, and reports the final
executed attempt's stderr when transient retries are exhausted. Step 5
success is defined by capturing or recovering a PR URL, not by `gh pr
create` returning exit code 0 alone. The empty `pr_url =` success path
remains reserved for Step 4's `gh auth status` preflight when GitHub CLI
is unavailable or unauthenticated.

<details>
<summary>Individual Group Plans</summary>

### Group 1: Original retry plan

Add bounded shell-level retry around the Step 5 `gh pr create` command
in
[SKILL.md](/home/talon/projects/autoskillit-runs/impl-20260707-054510-064511/src/autoskillit/skills_extended/compose-pr/SKILL.md).
The retry should apply only after `gh auth status` has passed, preserve
the existing `pr_url = ...` output contract, retry only transient API or
network failures, and keep the PR body closing-issue guard effective for
the new command shape.

### Group 2: Remediation plan

Remediate the remaining `compose-pr` `gh pr create` retry gaps
identified by audit-impl. The implementation must keep HTTP 429 and
secondary rate-limit responses retryable, keep
`compose_pr_body_guard.py` effective for the actual Step 5
`PR_CREATE_BODY` command shape, and report the final executed attempt's
stderr when transient retries are exhausted.

Step 5 success must be defined by capturing or recovering a PR URL, not
by `gh pr create` returning exit code 0 alone. The empty `pr_url =`
success path remains reserved for Step 4's `gh auth status` preflight
when GitHub CLI is unavailable or unauthenticated.

</details>

Closes #3987

## Implementation Plan

Plan files:
-
`/home/talon/projects/autoskillit-runs/impl-20260707-054510-064511/.autoskillit/temp/make-plan/compose_pr_gh_pr_create_retry_plan_2026-07-07_060736.md`
-
`/home/talon/projects/autoskillit-runs/impl-20260707-054510-064511/.autoskillit/temp/make-plan/compose_pr_retry_remediation_plan_2026-07-07_071159.md`

🤖 Generated with [Claude Code](https://claude.com/claude-code) via
AutoSkillit
<!-- autoskillit:pipeline-signature
steps=prepare_pr,run_arch_lenses,compose_pr,annotate_pr_diff,review_pr
-->

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>

Author: Trecek

src/autoskillit/hooks/_command_classification.py

@@ -45,6 +45,24 @@
 
 _SHELL_OPS: frozenset[str] = frozenset({"&&", "||", ";", "!", "|", "("})
 
+# Shell control words that mark the start of a new command in compound
+# shell constructs (loops, conditionals, case statements). When a `gh` token
+# is preceded by one of these, treat it as the verb of a fresh command — even
+# though shlex does not treat them as operators. Keep this set narrow: only
+# words that legitimately precede a command in real shell scripts.
+_SHELL_CONTROL_WORDS: frozenset[str] = frozenset(
+    {
+        "do",
+        "done",
+        "then",
+        "else",
+        "elif",
+        "esac",
+        "fi",
+        "in",
+    }
+)
+
 _REDIRECT_TOKEN_RE = re.compile(r"^(\d*)>{1,2}(.+)$")
 _REDIRECT_OP_ONLY_RE = re.compile(r"^(\d*)>{1,2}$")
 _FD_REDIRECT_RE = re.compile(r"^\d*>{1,2}&")

src/autoskillit/hooks/guards/compose_pr_body_guard.py

@@ -20,7 +20,15 @@
 if _HOOKS_DIR not in sys.path:
     sys.path.insert(0, _HOOKS_DIR)
 
-from _command_classification import _SHELL_OPS  # type: ignore[import-not-found]  # noqa: E402
+from _command_classification import (  # type: ignore[import-not-found]  # noqa: E402
+    _SHELL_CONTROL_WORDS,
+    _SHELL_OPS,
+)
+
+# Tokens that mark the boundary of a fresh command — either a shell operator
+# (already in _SHELL_OPS) or a shell control word like `do`/`then` that
+# introduces a new command inside a compound construct (loops, conditionals).
+_GH_COMMAND_BOUNDARY: frozenset[str] = _SHELL_OPS | _SHELL_CONTROL_WORDS
 
 COMPOSE_PR_BODY_DENY_TRIGGER: str = "compose-pr body missing Closes reference"
 
@@ -31,17 +39,132 @@
 
 _METADATA_CLOSING_RE = re.compile(r"^-[ \t]*closing_issue:[ \t]*(\S+)", re.MULTILINE)
 
+_SIMPLE_ASSIGN_RE = re.compile(r"^([A-Za-z_][A-Za-z0-9_]*)=(.*)$")
+_VAR_REF_RE = re.compile(r"^\$\{?([A-Za-z_][A-Za-z0-9_]*)\}?$")
+_UNSAFE_ASSIGN_VALUE_RE = re.compile(r"[`()|&;]")
+_NESTED_VAR_RE = re.compile(r"\$\{?([A-Za-z_][A-Za-z0-9_]*)\}?")
+
+# Keywords that open a nesting level (loop/conditional/case bodies).
+_DEPTH_INCREASE: frozenset[str] = frozenset({"while", "for", "until", "if", "case"})
+# Keywords that close a nesting level.
+_DEPTH_DECREASE: frozenset[str] = frozenset({"done", "fi", "esac"})
+
+
+def _collect_depth0_assignments(tokens: list[str], before_index: int) -> dict[str, str]:
+    """Collect simple variable assignments at nesting depth 0 before *before_index*.
+
+    Only assignments with safe values (no command substitution, backticks, or
+    shell operators in the value) are included. Simple $VAR references in values
+    are left as-is for downstream resolution.
+    """
+    assignments: dict[str, str] = {}
+    depth = 0
+    for i, tok in enumerate(tokens):
+        if i >= before_index:
+            break
+        if tok in _DEPTH_INCREASE:
+            depth += 1
+        elif tok in _DEPTH_DECREASE:
+            if depth > 0:
+                depth -= 1
+        elif depth == 0:
+            m = _SIMPLE_ASSIGN_RE.match(tok)
+            if m:
+                name, value = m.group(1), m.group(2)
+                if not _UNSAFE_ASSIGN_VALUE_RE.search(value):
+                    assignments[name] = value
+    return assignments
+
+
+def _resolve_nested_vars(value: str, assignments: dict[str, str]) -> str | None:
+    """Expand simple $VAR references in *value* from *assignments*.
+
+    Returns the expanded string, or None if any variable cannot be resolved
+    (fail-open: caller should treat None as "path unknown").
+    """
+    result_parts: list[str] = []
+    i = 0
+    while i < len(value):
+        if value[i] == "$":
+            m = _NESTED_VAR_RE.match(value, i)
+            if not m:
+                return None
+            var_name = m.group(1)
+            if var_name not in assignments:
+                return None
+            nested_val = assignments[var_name]
+            if "$" in nested_val:
+                return None
+            result_parts.append(nested_val)
+            i = m.end()
+        else:
+            result_parts.append(value[i])
+            i += 1
+    return "".join(result_parts)
+
+
+def _resolve_variable_body_path(raw_token: str, tokens: list[str], gh_index: int) -> str | None:
+    """Resolve a $VAR or ${VAR} body-file token from depth-0 assignments before *gh_index*.
+
+    Returns the resolved path string, or None if resolution fails (fail-open).
+    """
+    m = _VAR_REF_RE.match(raw_token)
+    if not m:
+        return None
+    var_name = m.group(1)
+
+    assignments = _collect_depth0_assignments(tokens, gh_index)
+    if var_name not in assignments:
+        return None
+
+    raw_value = assignments[var_name]
+    if "$" not in raw_value:
+        return raw_value
+    return _resolve_nested_vars(raw_value, assignments)
+
+
+def _preprocess_newlines(cmd: str) -> str:
+    """Replace bare (unquoted) newlines with ' ; ' to preserve command boundaries.
+
+    In shell a bare newline terminates a command just like ';'. shlex.split()
+    collapses newlines to whitespace, so a later command's --body-file can
+    bleed into an earlier gh pr create that has none. Replacing unquoted
+    newlines before tokenisation inserts a proper ';' separator.
+    """
+    result: list[str] = []
+    in_single = False
+    in_double = False
+    i = 0
+    while i < len(cmd):
+        c = cmd[i]
+        if c == "\\" and not in_single and i + 1 < len(cmd):
+            result.append(c)
+            result.append(cmd[i + 1])
+            i += 2
+            continue
+        if c == "'" and not in_double:
+            in_single = not in_single
+        elif c == '"' and not in_single:
+            in_double = not in_double
+        elif c == "\n" and not in_single and not in_double:
+            result.append(" ; ")
+            i += 1
+            continue
+        result.append(c)
+        i += 1
+    return "".join(result)
+
 
 def _extract_body_file_path(cmd: str) -> str | None:
     try:
-        tokens = shlex.split(cmd)
+        tokens = shlex.split(_preprocess_newlines(cmd))
     except ValueError:
         return None
 
     for i, tok in enumerate(tokens):
         if tok != "gh":
             continue
-        if i != 0 and tokens[i - 1] not in _SHELL_OPS:
+        if i != 0 and tokens[i - 1] not in _GH_COMMAND_BOUNDARY:
             continue
         if i + 2 >= len(tokens) or tokens[i + 1] != "pr" or tokens[i + 2] != "create":
             continue
@@ -51,9 +174,15 @@ def _extract_body_file_path(cmd: str) -> str | None:
             if t in _SHELL_OPS:
                 break
             if t == "--body-file" and j + 1 < len(tokens) and tokens[j + 1] not in _SHELL_OPS:
-                return tokens[j + 1]
+                raw = tokens[j + 1]
+                if raw.startswith("$"):
+                    return _resolve_variable_body_path(raw, tokens, i)
+                return raw
             if t.startswith("--body-file="):
-                return t.split("=", 1)[1]
+                raw = t.split("=", 1)[1]
+                if raw.startswith("$"):
+                    return _resolve_variable_body_path(raw, tokens, i)
+                return raw
     return None
 
 

src/autoskillit/recipe/skill_contracts.yaml

@@ -563,12 +563,13 @@ skills:
     - name: pr_url
       type: string
     expected_output_patterns:
-    - "pr_url[ \\t]*=[ \\t]*https://github\\.com/.*/pull/\\d+"
+    - "pr_url[ \\t]*=[ \\t]*(?:https://github\\.com/.*/pull/\\d+)?"
     pattern_examples:
     - "pr_url = https://github.com/owner/repo/pull/42\n%%ORDER_UP%%"
+    - "pr_url = \n%%ORDER_UP%%"
     write_behavior: conditional
     write_expected_when:
-    - "pr_url[ \\t]*=[ \\t]*https://"
+    - "pr_url[ \\t]*=[ \\t]*(?:https://|$)"
     source_pin_fields:
     - field: task_title
       required_source: "prep file ## Title section"

src/autoskillit/skills_extended/compose-pr/SKILL.md

@@ -77,7 +77,8 @@ Parse positional arguments:
 
 Derive `feature_branch` and set shell variables:
 ```bash
-FEATURE_BRANCH=$(git -C $WORK_DIR rev-parse --abbrev-ref HEAD)
+WORK_DIR=$3
+FEATURE_BRANCH=$(git -C "$WORK_DIR" rev-parse --abbrev-ref HEAD)
 BASE_BRANCH=$4
 ```
 
@@ -230,15 +231,107 @@ If exit code is non-zero:
 
 ### Step 5: Create Pull Request
 
+Run `gh pr create` with bounded retry on transient failures. Capture PR URL from stdout.
+On success or successful response-loss recovery, emit `pr_url = <URL>`. On exhausted retries or
+terminal validation failures, print the final `gh pr create` stderr to stderr and return the
+final create status (the empty-`pr_url` path is reserved for Step 4's `gh auth status` preflight).
+
 ```bash
-gh pr create \
-  --base $BASE_BRANCH \
-  --head $FEATURE_BRANCH \
-  --title "$TASK_TITLE" \
-  --body-file {{AUTOSKILLIT_TEMP}}/compose-pr/pr_body_$ts.md
+PR_CREATE_BODY={{AUTOSKILLIT_TEMP}}/compose-pr/pr_body_$ts.md
+PR_CREATE_LOG_DIR={{AUTOSKILLIT_TEMP}}/compose-pr
+PR_CREATE_ATTEMPT=1
+PR_CREATE_MAX=3
+PR_CREATE_STATUS=0
+PR_CREATE_LAST_ATTEMPT=1
+PR_URL=""
+while [ "$PR_CREATE_ATTEMPT" -le "$PR_CREATE_MAX" ]; do
+  case "$PR_CREATE_ATTEMPT" in
+    2) sleep 1 ;;
+    3) sleep 2 ;;
+  esac
+  gh pr create \
+       --base "$BASE_BRANCH" \
+       --head "$FEATURE_BRANCH" \
+       --title "$TASK_TITLE" \
+       --body-file "$PR_CREATE_BODY" \
+       > "$PR_CREATE_LOG_DIR/pr_stdout_$ts.$PR_CREATE_ATTEMPT.txt" \
+       2> "$PR_CREATE_LOG_DIR/pr_stderr_$ts.$PR_CREATE_ATTEMPT.txt"
+  PR_CREATE_STATUS=$?
+  PR_CREATE_LAST_ATTEMPT=$PR_CREATE_ATTEMPT
+  if [ "$PR_CREATE_STATUS" -eq 0 ]; then
+    PR_URL=$(grep -E '^https://github\.com/' "$PR_CREATE_LOG_DIR/pr_stdout_$ts.$PR_CREATE_ATTEMPT.txt" | head -1)
+    if [ -n "$PR_URL" ]; then
+      break
+    fi
+  fi
+  PR_CREATE_ERR=$(cat "$PR_CREATE_LOG_DIR/pr_stderr_$ts.$PR_CREATE_ATTEMPT.txt")
+  case "$PR_CREATE_ERR" in
+    # Retryable — HTTP 5xx, HTTP 429, secondary rate limit, rate limit, timeout, connection reset/refused.
+    # Classified before broad HTTP 4xx terminal handling so retryable rate limits are not shadowed.
+    *HTTP\ 429*|*secondary\ rate\ limit*|*rate\ limit*|*HTTP\ 5*|*timeout*|*connection\ reset*|*connection\ refused*|*temporary\ DNS*|*internal\ server\ error*|*remote\ server\ error*)
+      : ;;
+    # Terminal — do not retry (broad HTTP 4xx / validation / required-field missing).
+    *HTTP\ 4*|*validation*|*required*|*not\ found*)
+      break ;;
+    # Ambiguous (empty / unknown stderr) — attempt response-loss recovery, then retry.
+    *)
+      : ;;
+  esac
+  RECOVERED=$(gh pr view "$FEATURE_BRANCH" --json url -q .url 2>/dev/null || true)
+  if [ -n "$RECOVERED" ]; then
+    PR_URL="$RECOVERED"
+    PR_CREATE_STATUS=0
+    break
+  fi
+  PR_CREATE_ATTEMPT=$((PR_CREATE_ATTEMPT + 1))
+done
+
+if [ -n "$PR_URL" ]; then
+  printf 'pr_url = %s\n' "$PR_URL"
+  exit 0
+fi
+
+if [ "$PR_CREATE_STATUS" -eq 0 ]; then
+  PR_CREATE_STATUS=1
+fi
+echo "gh pr create failed after ${PR_CREATE_LAST_ATTEMPT} attempt(s):" >&2
+LAST_STDERR="$PR_CREATE_LOG_DIR/pr_stderr_$ts.$PR_CREATE_LAST_ATTEMPT.txt"
+if [ -s "$LAST_STDERR" ]; then
+  cat "$LAST_STDERR" >&2
+else
+  echo "gh pr create exited 0 but returned no PR URL after ${PR_CREATE_LAST_ATTEMPT} attempt(s)." >&2
+fi
+exit "$PR_CREATE_STATUS"
 ```
 
-Capture PR URL from stdout.
+Bounded retry policy:
+- Maximum 3 total `gh pr create` attempts
+- Backoff sleeps of 1 second before attempt 2 and 2 seconds before attempt 3
+- Each attempt's stdout/stderr captured under `{{AUTOSKILLIT_TEMP}}/compose-pr/`
+  so final diagnostics can print useful detail without contaminating the PR URL capture
+- Retryable rate-limit patterns (`HTTP 429`, `secondary rate limit`, broad `rate limit`) are
+  classified before the broad terminal `HTTP 4xx` pattern so they are never shadowed
+- Retry only transient or ambiguous response-loss failures (HTTP 429, secondary rate limit,
+  broad rate limit, HTTP 5xx, timeout, connection reset/refused, temporary DNS resolution,
+  remote/internal server errors)
+- Do not retry terminal validation or usage failures (broad HTTP 4xx, validation errors,
+  missing required fields, not-found errors)
+
+Response-loss recovery: when a create attempt fails with a transient/ambiguous status (or empty
+stderr that suggests the create succeeded but the response was lost), run a best-effort
+`gh pr view "$FEATURE_BRANCH" --json url -q .url` before sleeping and retrying. If it returns
+a URL, treat the create as recovered and emit that URL. The recovery path runs only for
+transient/ambiguous failures — never for terminal validation failures.
+
+Last-attempt tracking: `PR_CREATE_LAST_ATTEMPT` is set to `$PR_CREATE_ATTEMPT` immediately
+after each `gh pr create` execution. Final failure reporting reads the stderr file for
+`PR_CREATE_LAST_ATTEMPT`, not the post-increment loop counter.
+
+Final status propagation: if the loop exits without a captured or recovered PR URL, the exit
+status is forced nonzero even if the last raw `gh pr create` exit code was 0 (zero-exit/no-URL
+is not a successful outcome). When the last attempt's stderr file is empty or absent (zero-exit
+path), a concise no-URL diagnostic is printed instead. The empty-`pr_url` success path is
+reserved exclusively for Step 4's `gh auth status` preflight failure.
 
 ## Output
 

tests/_test_filter.py

@@ -802,6 +802,7 @@ class ImportContext(enum.StrEnum):
             "skills/test_audit_impl_diff_discipline.py",
             "skills/test_skill_variable_threading.py",
             "skills/test_phoropter_structural.py",
+            "skills/test_compose_pr_retry.py",
             "smoke_utils",
         }
     ),