feat: make loop autonomous with persistent goals and robust codex binary resolution

Author: Dremig

README.md

@@ -1,12 +1,12 @@
 # codex-self-iter
 
-`codex-self-iter` is a plugin-style repository that runs a continuous autonomous loop for coding tasks:
+`codex-self-iter` is a plugin-style repository that runs a continuous autonomous goal loop:
 
 1. Read `TASK.md` + `plan.md`
-2. Generate one small next step
-3. Execute the step via agent command
-4. Review result and decide whether to continue
-5. Repeat until no meaningful next action exists or user stops it
+2. Execute one autonomous iteration in the repository
+3. Require the agent to emit a concrete `next_goal`
+4. Persist that goal and continue from it
+5. Repeat until completion/stop condition
 
 ## Repository Layout
 
@@ -62,6 +62,8 @@ Or use:
 - `.codex-self-iter/prompts/`: planner/executor/reviewer prompts
 - `.codex-self-iter/logs/iterations.jsonl`: loop records
 - `.codex-self-iter/status.json`: last iteration state
+- `.codex-self-iter/runtime.json`: persisted backoff counters and window
+- `.codex-self-iter/loop.lock`: single-instance guard file
 
 ## Configuration
 
@@ -72,6 +74,10 @@ Important fields:
 - `agent_command_template` must include `{prompt_file}`
 - `max_iterations = 0` means unbounded loop
 - `max_stagnation` prevents infinite cycling on identical `next_focus`
+- `no_change_gate` throttles no-op loops when no new git changes appear
+- `backoff_*` fields control exponential cooldown behavior
+- `lock_file_name` controls the single-instance lock path under state-dir
+- `completion_promise` marks completion when included in output
 
 ## Approval and Privilege Mode
 
@@ -91,6 +97,20 @@ In bypass mode, wrapper uses:
 - `codex --dangerously-bypass-approvals-and-sandbox exec ...`  
   This disables approvals and sandboxing entirely. Use only in externally sandboxed environments.
 
+## Binary Resolution (macOS launchd/system services)
+
+When running under `launchd` (or other service managers), interactive shell `PATH` is often not inherited.
+If `codex` is not found, set an explicit binary path:
+
+```bash
+export CODEX_BIN=/absolute/path/to/codex
+```
+
+The default wrapper checks in this order:
+1. `CODEX_BIN` (if executable)
+2. `PATH` via `command -v codex`
+3. common fallback paths (nvm/Homebrew/local bin)
+
 ## Development
 
 Run tests:

config.example.toml

@@ -23,3 +23,21 @@ reviewer_model_hint = "strict"
 
 # Number of history rows fed back to planner each round.
 history_tail = 8
+
+# Gate tight restart loops when no new git changes are produced.
+no_change_gate = true
+
+# Exponential backoff controls for transient errors.
+backoff_base_sec = 5
+backoff_max_sec = 300
+
+# Backoff controls for low-signal outcomes.
+no_change_backoff_sec = 60
+continue_false_backoff_sec = 300
+planner_stop_backoff_sec = 600
+
+# Single-instance lock under state-dir.
+lock_file_name = "loop.lock"
+
+# If this token appears in agent output, the loop treats task as complete.
+completion_promise = "DONE"

scripts/invoke-codex-agent.sh

@@ -1,5 +1,5 @@
 #!/usr/bin/env bash
-set -euo pipefail
+set -uo pipefail
 
 if [[ $# -lt 2 ]]; then
   echo "usage: $0 <prompt-file> <workspace>" >&2
@@ -8,12 +8,98 @@ fi
 
 prompt_file="$1"
 workspace="$2"
+state_dir="${CODEX_SELF_ITER_STATE_DIR:-$workspace/.codex-self-iter}"
+session_file="${CODEX_SELF_ITER_SESSION_FILE:-$state_dir/codex-session-id}"
 
-# Default mode: auto-confirm approval requests while keeping sandboxing.
-if [[ "${CODEX_SELF_ITER_AGENT_MODE:-}" == "bypass" ]]; then
-  # Extremely dangerous: no approval prompts and no sandbox.
-  cat "$prompt_file" | codex --dangerously-bypass-approvals-and-sandbox exec -C "$workspace" -
+mkdir -p "$state_dir"
+
+msg_file="$(mktemp)"
+err_file="$(mktemp)"
+trap 'rm -f "$msg_file" "$err_file"' EXIT
+
+session_id=""
+if [[ -f "$session_file" ]]; then
+  session_id="$(tr -d '[:space:]' < "$session_file" 2>/dev/null || true)"
+fi
+
+resolve_codex_bin() {
+  if [[ -n "${CODEX_BIN:-}" ]] && [[ -x "${CODEX_BIN:-}" ]]; then
+    echo "$CODEX_BIN"
+    return 0
+  fi
+  if command -v codex >/dev/null 2>&1; then
+    command -v codex
+    return 0
+  fi
+  local candidates=(
+    "$HOME/.nvm/versions/node/v24.10.0/bin/codex"
+    "$HOME/.nvm/versions/node/v*/bin/codex"
+    "/opt/homebrew/bin/codex"
+    "/usr/local/bin/codex"
+  )
+  local c
+  for c in "${candidates[@]}"; do
+    for m in $c; do
+      if [[ -x "$m" ]]; then
+        echo "$m"
+        return 0
+      fi
+    done
+  done
+  return 1
+}
+
+CODEX_CMD="$(resolve_codex_bin || true)"
+if [[ -z "$CODEX_CMD" ]]; then
+  cat >&2 <<'EOF'
+Error: codex binary not found.
+Tips:
+- Set CODEX_BIN to an absolute path (recommended for launchd/systemd)
+- Or ensure codex is available in PATH
+EOF
+  exit 127
+fi
+
+run_codex_new() {
+  if [[ "${CODEX_SELF_ITER_AGENT_MODE:-}" == "bypass" ]]; then
+    cat "$prompt_file" | "$CODEX_CMD" --dangerously-bypass-approvals-and-sandbox exec -C "$workspace" -o "$msg_file" - >/dev/null 2>"$err_file"
+  else
+    cat "$prompt_file" | "$CODEX_CMD" -a never exec -C "$workspace" -o "$msg_file" - >/dev/null 2>"$err_file"
+  fi
+  return $?
+}
+
+run_codex_resume() {
+  local sid="$1"
+  if [[ "${CODEX_SELF_ITER_AGENT_MODE:-}" == "bypass" ]]; then
+    cat "$prompt_file" | "$CODEX_CMD" --dangerously-bypass-approvals-and-sandbox exec resume "$sid" -o "$msg_file" - >/dev/null 2>"$err_file"
+  else
+    cat "$prompt_file" | "$CODEX_CMD" -a never exec resume "$sid" -o "$msg_file" - >/dev/null 2>"$err_file"
+  fi
+  return $?
+}
+
+rc=1
+if [[ -n "$session_id" ]]; then
+  run_codex_resume "$session_id"
+  rc=$?
+fi
+
+if [[ $rc -ne 0 ]]; then
+  run_codex_new
+  rc=$?
+fi
+
+if [[ $rc -eq 0 ]]; then
+  if [[ -z "$session_id" ]]; then
+    sid="$(rg -o -N 'session id:\s*([0-9a-fA-F-]{36})' "$err_file" | head -n1 | awk '{print $3}' || true)"
+    if [[ -n "$sid" ]]; then
+      echo "$sid" > "$session_file"
+    fi
+  fi
+  cat "$msg_file"
 else
-  # No manual confirmation; command failures are returned directly to the model.
-  cat "$prompt_file" | codex -a never exec -C "$workspace" -
+  cat "$err_file"
 fi
+
+exit $rc

src/codex_self_iter/config.py

@@ -20,6 +20,14 @@ class LoopConfig:
     planner_model_hint: str = "balanced"
     reviewer_model_hint: str = "strict"
     history_tail: int = 8
+    no_change_gate: bool = True
+    backoff_base_sec: int = 5
+    backoff_max_sec: int = 300
+    no_change_backoff_sec: int = 60
+    continue_false_backoff_sec: int = 300
+    planner_stop_backoff_sec: int = 600
+    lock_file_name: str = "loop.lock"
+    completion_promise: str = "DONE"
 
 
 def read_text(path: Path, default: str = "") -> str:
@@ -44,4 +52,12 @@ def load_config(config_path: Path | None, default_agent_cmd: str) -> LoopConfig:
         planner_model_hint=str(raw.get("planner_model_hint", "balanced")),
         reviewer_model_hint=str(raw.get("reviewer_model_hint", "strict")),
         history_tail=int(raw.get("history_tail", 8)),
+        no_change_gate=bool(raw.get("no_change_gate", True)),
+        backoff_base_sec=int(raw.get("backoff_base_sec", 5)),
+        backoff_max_sec=int(raw.get("backoff_max_sec", 300)),
+        no_change_backoff_sec=int(raw.get("no_change_backoff_sec", 60)),
+        continue_false_backoff_sec=int(raw.get("continue_false_backoff_sec", 300)),
+        planner_stop_backoff_sec=int(raw.get("planner_stop_backoff_sec", 600)),
+        lock_file_name=str(raw.get("lock_file_name", "loop.lock")),
+        completion_promise=str(raw.get("completion_promise", "DONE")),
     )

src/codex_self_iter/prompts.py

@@ -1,6 +1,47 @@
 from __future__ import annotations
 
 
+def autonomous_goal_prompt(
+    task_text: str,
+    plan_text: str,
+    current_goal: str,
+    history_jsonl_tail: str,
+    model_hint: str,
+    completion_promise: str,
+) -> str:
+    return f"""You are an autonomous coding agent in an endless explore-and-improve loop.
+Model style hint: {model_hint}
+
+Primary task:
+{task_text}
+
+Plan context:
+{plan_text}
+
+Current goal for this iteration:
+{current_goal}
+
+Recent iteration history:
+{history_jsonl_tail}
+
+You must do real work in the repository (code/tests/docs/commands), then decide the next goal.
+
+Output exactly one JSON object with keys:
+- summary: short string, what you did this round
+- completed: boolean, true only if overall task is truly complete
+- next_goal: short string, required when completed=false
+- reason: short string, why next_goal is the right continuation
+- evidence: short string, key checks or observations
+- stop: boolean (true only if no meaningful next action exists)
+
+Rules:
+1) Be self-directed: choose the most valuable next step based on actual repo state.
+2) Prefer measurable progress (tests, reproducibility, correctness).
+3) If not complete, always provide a concrete next_goal.
+4) If complete, include completion token "{completion_promise}" in summary.
+"""
+
+
 def planner_prompt(task_text: str, plan_text: str, history_jsonl_tail: str, model_hint: str) -> str:
     return f"""You are the planner in a self-iterating coding loop.
 Model style hint: {model_hint}