Merge changes from main: resolve barrel export conflicts and bring in new features

Brought in all changes from origin/main without creating a full merge history:
- New source files: api_types, attrs, insert_pop, pipe_apply, to_from_dict (core),
  categorical_ops, cut_qcut, format_ops, notna_isna, numeric_extended, string_ops,
  string_ops_extended, window_extended (stats), rolling_apply (window)
- Resolved barrel export conflicts in src/core/index.ts, src/stats/index.ts,
  src/index.ts, src/reshape/index.ts (PR's API naming preserved)
- Updated playground pages, tests, benchmarks, CI workflows from main

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Author: github-actions[bot]

.autoloop/programs/perf-comparison/program.md

@@ -0,0 +1,74 @@
+---
+schedule: every 6h
+---
+
+# Performance Comparison: tsb (TypeScript) vs pandas (Python)
+
+## Goal
+
+Systematically benchmark every tsb function against its pandas equivalent, one function per iteration. Each iteration picks a function that has not yet been benchmarked, writes a matching performance test for both tsb (TypeScript/Bun) and pandas (Python), runs both, and records the timing results. The benchmark results are displayed on the playground pages doc site.
+
+This is an open-ended program — it runs continuously, always adding the next benchmark comparison.
+
+### How each iteration works
+
+1. **Read existing benchmarks** — check `benchmarks/tsb/` and `benchmarks/pandas/` to see which functions are already benchmarked.
+2. **Pick ONE function** from `src/` that has no benchmark yet. Prioritize core operations (Series, DataFrame, GroupBy, etc.).
+3. **Write a TypeScript benchmark** in `benchmarks/tsb/bench_{function}.ts` that:
+   - Creates a realistic dataset (e.g. 100,000 rows)
+   - Runs the operation in a tight loop (warm-up + measured iterations)
+   - Outputs JSON: `{"function": "...", "mean_ms": ..., "iterations": ..., "total_ms": ...}`
+4. **Write a matching Python benchmark** in `benchmarks/pandas/bench_{function}.py` that:
+   - Creates the same dataset as the TypeScript version
+   - Runs the same operation with the same loop structure
+   - Outputs the same JSON format
+5. **Run both benchmarks** via `benchmarks/run_benchmarks.sh` and capture results.
+6. **Update `benchmarks/results.json`** with the new timing data.
+7. **Update `playground/benchmarks.html`** to display the new function's comparison metrics.
+
+### Key constraints
+
+- **Matching datasets** — both benchmarks must use identical data (same size, same values where possible).
+- **Fair comparison** — same number of warm-up and measured iterations for both.
+- **JSON output** — every benchmark script must output a single JSON line to stdout.
+- **No modifications to `src/`** — benchmark code is separate from library code.
+- **Python environment** — install pandas via pip if not present.
+
+## Target
+
+Only modify these files:
+- `benchmarks/**` — benchmark scripts and results
+- `playground/benchmarks.html` — performance comparison playground page
+- `playground/index.html` — add/update link to benchmarks page
+
+Do NOT modify:
+- `src/**` — library source code
+- `tests/**` — test files
+- `README.md` — read-only
+- `.autoloop/programs/**` — program definitions (except this file's code/ dir)
+- `.github/workflows/autoloop*` — autoloop workflow files
+
+## Evaluation
+
+```bash
+# Set up Python environment if needed
+if ! command -v python3 &>/dev/null; then
+  echo "Python3 not found, skipping"
+fi
+pip3 install pandas --quiet 2>/dev/null || true
+
+# Count the number of benchmark pairs (functions with both TS and Python benchmarks)
+ts_benchmarks=$(ls benchmarks/tsb/bench_*.ts 2>/dev/null | wc -l | tr -d ' ')
+py_benchmarks=$(ls benchmarks/pandas/bench_*.py 2>/dev/null | wc -l | tr -d ' ')
+
+# The metric is the minimum of the two (both must exist for a complete benchmark)
+if [ "$ts_benchmarks" -lt "$py_benchmarks" ]; then
+  count=$ts_benchmarks
+else
+  count=$py_benchmarks
+fi
+
+echo "{\"benchmarked_functions\": ${count:-0}}"
+```
+
+The metric is `benchmarked_functions`. **Higher is better.**

.github/workflows/autoloop.lock.yml

@@ -77,6 +77,7 @@ tools:
   repo-memory:
     branch-name: memory/autoloop
     file-glob: ["*.md"]
+    max-file-size: 30720
 
 imports:
   - shared/reporting.md
@@ -430,11 +431,50 @@ steps:
           if selected in issue_programs:
               selected_issue = issue_programs[selected]["issue_number"]
 
+      # Look up existing PR for the selected program's canonical branch
+      existing_pr = None
+      head_branch = None
+      if selected:
+          head_branch = f"autoloop/{selected}"
+          owner = repo.split("/")[0] if "/" in repo else ""
+          if owner:
+              try:
+                  pr_api_url = (
+                      f"https://api.github.com/repos/{repo}/pulls"
+                      f"?state=open&head={owner}:{head_branch}&per_page=5"
+                  )
+                  pr_req = urllib.request.Request(pr_api_url, headers={
+                      "Authorization": f"token {github_token}",
+                      "Accept": "application/vnd.github.v3+json",
+                  })
+                  with urllib.request.urlopen(pr_req, timeout=30) as pr_resp:
+                      open_prs = json.loads(pr_resp.read().decode())
+                  if open_prs:
+                      existing_pr = open_prs[0]["number"]
+                      print(f"  Found existing PR #{existing_pr} for branch {head_branch}")
+                  else:
+                      print(f"  No existing PR found for branch {head_branch}")
+              except Exception as e:
+                  print(f"  Warning: could not check for existing PRs: {e}")
+          else:
+              print(f"  Warning: could not parse owner from GITHUB_REPOSITORY='{repo}'")
+
+          # Also check the state file for a recorded PR number as fallback
+          if existing_pr is None:
+              state = read_program_state(selected)
+              pr_field = state.get("pr") or ""
+              pr_match = re.match(r'^#?(\d+)$', pr_field.strip())
+              if pr_match:
+                  existing_pr = int(pr_match.group(1))
+                  print(f"  Found PR #{existing_pr} from state file for {selected}")
+
       result = {
           "selected": selected,
           "selected_file": selected_file,
           "selected_issue": selected_issue,
           "selected_target_metric": selected_target_metric,
+          "existing_pr": existing_pr,
+          "head_branch": head_branch,
           "issue_programs": {name: info["issue_number"] for name, info in issue_programs.items()},
           "deferred": deferred,
           "skipped": skipped,
@@ -448,6 +488,10 @@ steps:
 
       print("=== Autoloop Program Check ===")
       print(f"Selected program:      {selected or '(none)'} ({selected_file or 'n/a'})")
+      if existing_pr:
+          print(f"Existing PR:           #{existing_pr} (branch: {head_branch})")
+      else:
+          print(f"Existing PR:           (none — will create on first accepted iteration)")
       print(f"Deferred (next run):   {deferred or '(none)'}")
       print(f"Programs skipped:      {[s['name'] for s in skipped] or '(none)'}")
       print(f"Programs unconfigured: {unconfigured or '(none)'}")
@@ -537,6 +581,8 @@ The pre-step has already determined which program to run. Read `/tmp/gh-aw/autol
 - **`selected_file`**: The full path to the program's markdown file (either `.autoloop/programs/<name>/program.md`, `.autoloop/programs/<name>.md`, or `/tmp/gh-aw/issue-programs/<name>.md` for issue-based programs).
 - **`selected_issue`**: The GitHub issue number if the selected program came from an issue, or `null` if it came from a file.
 - **`selected_target_metric`**: The `target-metric` value from the program's frontmatter (a number), or `null` if the program is open-ended. Used to check the [halting condition](#halting-condition) after each accepted iteration.
+- **`existing_pr`**: The PR number (e.g., `42`) of an already-open PR for this program's branch, or `null` if no open PR exists. **If this is not null, you MUST use `push-to-pull-request-branch` to push to this PR — do NOT call `create-pull-request`.**
+- **`head_branch`**: The canonical branch name for this program (e.g., `autoloop/coverage`). Always use this exact branch name — never append suffixes.
 - **`issue_programs`**: A mapping of program name → issue number for all discovered issue-based programs.
 - **`deferred`**: Other programs that were due but will be handled in future runs.
 - **`unconfigured`**: Programs that still have the sentinel or placeholder content.
@@ -549,6 +595,7 @@ If `selected` is not null:
 3. Read the current state of all target files.
 4. Read the state file `{selected}.md` from the repo-memory folder for all state: the ⚙️ Machine State table (scheduling fields) plus the research sections (priorities, lessons, foreclosed avenues, iteration history).
 5. If `selected_issue` is not null, this is an issue-based program — also read the issue comments for any human steering input.
+6. **Check `existing_pr`**: if it is not null, a PR already exists — use `push-to-pull-request-branch` to push commits to it. Only call `create-pull-request` when `existing_pr` is null.
 
 ## Multiple Programs
 
@@ -693,7 +740,7 @@ Each run executes **one iteration for the single selected program**:
 
    If the state file does not yet exist, create it in the repo-memory folder using the template defined in the [Repo Memory](#repo-memory) section.
 
-3. Note the `PR` field from the Machine State table. If it contains a PR number (e.g., `#42`), that is the **existing draft PR** for this program — you must update it, not create a new one.
+3. Note the `existing_pr` field from `/tmp/gh-aw/autoloop.json`. If it is not null, that is the **existing draft PR** for this program — you must push to it using `push-to-pull-request-branch`, not create a new one. Also check the `PR` field from the Machine State table as a fallback.
 
 ### Step 2: Analyze and Propose
 
@@ -742,15 +789,15 @@ Each run executes **one iteration for the single selected program**:
    - Commit message body (after a blank line): `Run: {run_url}` referencing the GitHub Actions run URL.
 2. Push the commit to the long-running branch `autoloop/{program-name}`.
 3. **Find the existing PR or create one** — follow these steps in order:
-   a. Check the `PR` field in the state file's **⚙️ Machine State** table. If it contains a PR number (e.g., `#42`), that is the existing draft PR.
-   b. If the state file has no PR number, search GitHub for open PRs with head branch `autoloop/{program-name}`. Use the GitHub API: `GET /repos/{owner}/{repo}/pulls?state=open&head={owner}:autoloop/{program-name}`.
-   c. **If an existing PR is found** (from either step a or b): use `push-to-pull-request-branch` to push additional commits to the existing PR. Update the PR body with the latest metric and a summary of the most recent accepted iteration. Add a comment to the PR summarizing the iteration: what changed, old metric, new metric, improvement delta, and a link to the actions run. **Do NOT call `create-pull-request`.**
-   d. **If NO PR exists** for `autoloop/{program-name}`: create one using `create-pull-request`:
+   a. **First, check `existing_pr` from `/tmp/gh-aw/autoloop.json`.** The pre-step has already looked up the open PR for this program. If `existing_pr` is not null, that is the existing draft PR — skip to step (c).
+   b. If `existing_pr` is null, also check the `PR` field in the state file's **⚙️ Machine State** table as a fallback. If it contains a PR number (e.g., `#42`), verify it is still open via the GitHub API.
+   c. **If an existing PR is found** (from step a or b): use `push-to-pull-request-branch` to push additional commits to the existing PR. Update the PR body with the latest metric and a summary of the most recent accepted iteration. Add a comment to the PR summarizing the iteration: what changed, old metric, new metric, improvement delta, and a link to the actions run. **Do NOT call `create-pull-request`.**
+   d. **If NO PR exists** for `autoloop/{program-name}` (both `existing_pr` is null AND the state file has no PR): create one using `create-pull-request`:
       - Branch: `autoloop/{program-name}` (the branch you already created in Step 3 — do NOT let the framework auto-generate a branch name)
       - Title: `[Autoloop: {program-name}]`
       - Body includes: a summary of the program goal, link to the steering issue, the current best metric, and AI disclosure: `🤖 *This PR is maintained by Autoloop. Each accepted iteration adds a commit to this branch.*`
 
-   > ⚠️ **Never create a new PR if one already exists for `autoloop/{program-name}`.** Each program must have exactly one draft PR at any time. If you are unsure whether a PR exists, check the GitHub API before calling `create-pull-request`.
+   > ⚠️ **Never create a new PR if one already exists for `autoloop/{program-name}`.** Each program must have exactly one draft PR at any time. The pre-step provides `existing_pr` in autoloop.json — always check it first. Only call `create-pull-request` when `existing_pr` is null AND the state file has no PR number.
 4. Ensure the steering issue exists (see [Steering Issue](#steering-issue) below). Add a comment to the steering issue linking to the commit and actions run.
 5. Add an entry to the experiment log issue.
 6. Update the state file `{program-name}.md` in the repo-memory folder:
@@ -789,6 +836,13 @@ Maintain a single open issue **per program** titled `[Autoloop: {program-name}]
 ```markdown
 🤖 *Autoloop — an iterative optimization agent for this repository.*
 
+| | |
+|---|---|
+| **Branch** | [`autoloop/{program-name}`](https://github.com/{owner}/{repo}/tree/autoloop/{program-name}) |
+| **Pull Request** | #{pr_number} |
+| **Steering Issue** | #{steering_issue_number} |
+| **State File** | [`{program-name}.md`](https://github.com/{owner}/{repo}/blob/memory/autoloop/{program-name}.md) |
+
 ## Program
 
 **Goal**: {one-line summary from program.md}
@@ -816,6 +870,7 @@ Maintain a single open issue **per program** titled `[Autoloop: {program-name}]
 - Iterations in **reverse chronological order** (newest first).
 - Each iteration heading links to its GitHub Actions run.
 - Use `${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}` for the current run URL.
+- The **links table at the top** must always show the current branch, PR, steering issue, and state file. Update the PR number when a new PR is created. When creating a continuation issue for a new month, copy the links table from the previous issue.
 - Close the previous month's issue and create a new one at month boundaries.
 - Maximum 50 iterations per issue; create a continuation issue if exceeded.
 
@@ -1122,6 +1177,7 @@ After each iteration, prepend an entry to the **📊 Iteration History** section
 - **Always** update the state file after each iteration, regardless of outcome.
 - **Update the Machine State table first** — the scheduling pre-step depends on it.
 - **Prepend** iteration history entries (newest first).
+- **Keep the state file compact.** The state file must stay under 30 KB. When prepending a new iteration entry, collapse older iteration entries (beyond the most recent 10) into compressed summary lines (e.g. `### Iters 50–100 — ✅ (metrics 20→55): brief summary`). Also prune Lessons Learned to only the most recent and relevant entries.
 - **Accumulate** Lessons Learned — add new insights, don't overwrite existing ones.
 - **Add to Foreclosed Avenues** only when an approach is conclusively ruled out (not just rejected once).
 - **Respect Current Priorities** — if a maintainer has written priorities, follow them in your next proposal.
@@ -1146,9 +1202,10 @@ After each iteration, prepend an entry to the **📊 Iteration History** section
 > ❌ **Do NOT create a new branch with a suffix for each iteration.**
 > Correct: `autoloop/coverage`
 > Wrong: `autoloop/coverage-abc123`, `autoloop/coverage-iter42`, `autoloop/coverage-deadbeef1234`
+> Use the `head_branch` field from `autoloop.json` — it is always the canonical name.
 
 > ❌ **Do NOT create a new PR if one already exists for `autoloop/{program-name}`.**
-> Always check the state file's `PR` field and the GitHub API before calling `create-pull-request`. If a PR exists, use `push-to-pull-request-branch` instead.
+> The pre-step provides `existing_pr` in `autoloop.json`. If it is not null, **always** use `push-to-pull-request-branch` — never call `create-pull-request`. Only create a PR when `existing_pr` is null AND the state file has no PR number.
 
 > ❌ **Do NOT let the gh-aw framework auto-generate a branch name when creating a PR.**
 > Always specify the branch explicitly as `autoloop/{program-name}` when calling `create-pull-request`.

.github/workflows/autoloop.md

@@ -59,3 +59,20 @@ jobs:
         with:
           name: dist
           path: dist/
+
+  validate-python-examples:
+    name: Validate Python Examples
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Setup Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Install Python dependencies
+        run: pip install pandas numpy
+
+      - name: Validate Python playground examples
+        run: python scripts/validate-python-examples.py playground/