Merge pull request #26 from githubnext/autoloop-progress/go-migration-iter24-7b126167538465d9

[autoloop-progress] docs: update Go migration progress page to iteration 24

Author: mrjf

docs/src/content/docs/progress/autoloop-go-migration.mdx

@@ -18,22 +18,34 @@ The `Autoloop Go Migration Progress Site` agentic workflow refreshes this page a
 | Autoloop branch | [`autoloop/python-to-go-migration`](https://github.com/githubnext/apm/tree/autoloop/python-to-go-migration) |
 | Tracking issue | [#3 Python-to-Go Migration](https://github.com/githubnext/apm/issues/3) |
 | Accumulating PR | [#17 Autoloop: python-to-go-migration](https://github.com/githubnext/apm/pull/17) |
-| Last accepted iteration | Iteration 4, 2026-05-12 16:30 UTC |
-| Best metric | 1.15% migrated |
-| Migrated lines | 827 of 71,696 original Python lines |
-| Migrated modules | 11 |
+| Last accepted iteration | Iteration 24, 2026-05-13 14:22 UTC |
+| Best metric | 9.89% migrated (`python_lines_migrated_pct`) |
+| Migrated lines (best) | ~7,092 of 71,696 original Python lines |
+| Migrated modules (status.json) | 24 (iter-13 baseline; see note below) |
+
+:::caution[Branch reset caveat]
+The migration branch periodically fast-forward-resets when main merges. Each Autoloop iteration rebuilds lost modules before adding new ones. `benchmarks/migration-status.json` reflects the iter-13 stable baseline (24 modules, 4,245 lines). Iteration 24 committed an additional 30 Go packages on top of that baseline; those modules are not yet captured in the JSON. The machine-state `best_metric` of 9.89 corresponds to ~7,092 migrated lines and is the authoritative progress signal.
+:::
 
 ## Migration progress
 
 | Iteration | Run | Change | Metric |
 |---:|---|---|---:|
-| 1 | [25717987972](https://github.com/githubnext/apm/actions/runs/25717987972) | Initialized the Go module and migrated `constants.py`, `version.py`, `utils/short_sha.py`, `utils/paths.py`, and `utils/normalization.py`. | 0.40% |
-| 2 | [25736801433](https://github.com/githubnext/apm/actions/runs/25736801433) | Migrated `utils/yaml_io.py`, `utils/atomic_io.py`, and `utils/git_env.py`. | 0.68% |
-| 3 | [25744614816](https://github.com/githubnext/apm/actions/runs/25744614816) | Migrated `utils/guards.py` as `internal/utils/guards`. | 0.85% |
-| 4 | [25747630390](https://github.com/githubnext/apm/actions/runs/25747630390) | Migrated `utils/subprocess_env.py` and `utils/helpers.py`. | 1.15% |
+| 1 | [25717987972](https://github.com/githubnext/apm/actions/runs/25717987972) | Initialized Go module; migrated `constants.py`, `version.py`, `utils/short_sha.py`, `utils/paths.py`, `utils/normalization.py`. | 0.40% |
+| 2 | [25736801433](https://github.com/githubnext/apm/actions/runs/25736801433) | Migrated `utils/yaml_io.py`, `utils/atomic_io.py`, `utils/git_env.py`. | 0.68% |
+| 3 | [25744614816](https://github.com/githubnext/apm/actions/runs/25744614816) | Migrated `utils/guards.py`. | 0.85% |
+| 4 | [25747630390](https://github.com/githubnext/apm/actions/runs/25747630390) | Migrated `utils/subprocess_env.py`, `utils/helpers.py`. | 1.15% |
+| 5-12 | — | Migrated `utils/content_hash.py`, `utils/exclude.py`, `utils/path_security.py`, `utils/version_checker.py`, `utils/file_ops.py`, `utils/console.py`, `utils/diagnostics.py`, `utils/install_tui.py`, `utils/github_host.py`, `utils/reflink.py`; branch resets caused repeated rebuilds. | 0.0%–5.41% |
+| 13 | [25771166584](https://github.com/githubnext/apm/actions/runs/25771166584) | Migrated 13 modules including `install/errors.py`, `install/cache_pin.py`, `install/context.py`; 4,245 total lines — stable baseline now recorded in `benchmarks/migration-status.json`. | 5.92% |
+| 14-21 | — | Repeatedly rebuilt modules lost to branch resets; added `compilation/*`, `models/*`, `policy/*`, `marketplace/*`, `cache/*`, `integration/*`, `workflow/*`, `primitives/*`; metrics oscillated with each reset. | 6.39%–8.66% |
+| 22 | [25798457534](https://github.com/githubnext/apm/actions/runs/25798457534) | Migrated 26 modules: `compilation/buildid+outputwriter+constitution`, `models/results+dependency/types`, `policy/schema+matcher+inheritance`, `install/request+summary+mcpargs`, `runtime/base`, `marketplace/validator+errors+tagpattern`, `cache/urlnormalize+paths+integrity`, `integration/utils+coverage`, `workflow/discovery+parser`, `core/nulllogger`, `deps/gitremoteops+aggregator`, `primitives/models`. | 9.37% |
+| 23 | [25801585559](https://github.com/githubnext/apm/actions/runs/25801585559) | Rebuilt 28 modules after another branch reset to iter-13 state. | 6.99% |
+| 24 | [25805061357](https://github.com/githubnext/apm/actions/runs/25805061357) | Rebuilt 27 modules and added 3 new ones (`marketplace/shadowdetector`, `core/dockerargs`, `deps/installedpackage`); +2,847 lines over iter-13 baseline. | **9.89%** |
 
 ## Migrated modules
 
+The table below lists the 24 modules recorded in `benchmarks/migration-status.json` on the migration branch (iter-13 stable baseline). Iteration 24 also includes 30 additional Go packages on the branch that are not yet captured in the JSON.
+
 | Python module | Go package | Python lines |
 |---|---|---:|
 | `src/apm_cli/constants.py` | `internal/constants` | 55 |
@@ -47,38 +59,61 @@ The `Autoloop Go Migration Progress Site` agentic workflow refreshes this page a
 | `src/apm_cli/utils/guards.py` | `internal/utils/guards` | 123 |
 | `src/apm_cli/utils/subprocess_env.py` | `internal/utils/subprocenv` | 84 |
 | `src/apm_cli/utils/helpers.py` | `internal/utils/helpers` | 131 |
+| `src/apm_cli/utils/content_hash.py` | `internal/utils/contenthash` | 108 |
+| `src/apm_cli/utils/exclude.py` | `internal/utils/exclude` | 169 |
+| `src/apm_cli/utils/path_security.py` | `internal/utils/pathsecurity` | 130 |
+| `src/apm_cli/utils/version_checker.py` | `internal/utils/versionchecker` | 193 |
+| `src/apm_cli/utils/file_ops.py` | `internal/utils/fileops` | 326 |
+| `src/apm_cli/utils/console.py` | `internal/utils/console` | 224 |
+| `src/apm_cli/utils/diagnostics.py` | `internal/utils/diagnostics` | 486 |
+| `src/apm_cli/utils/install_tui.py` | `internal/utils/installtui` | 365 |
+| `src/apm_cli/utils/github_host.py` | `internal/utils/githubhost` | 624 |
+| `src/apm_cli/utils/reflink.py` | `internal/utils/reflink` | 281 |
+| `src/apm_cli/install/errors.py` | `internal/install/errors` | 113 |
+| `src/apm_cli/install/cache_pin.py` | `internal/install/cachepin` | 233 |
+| `src/apm_cli/install/context.py` | `internal/install/installctx` | 166 |
 
-## Benchmark snapshot
+## Benchmark signals
 
-Autoloop's current migration metric is `python_lines_migrated_pct`: `(migrated_python_lines / original_python_lines) * 100`. The latest recorded status file on the migration branch reports 827 migrated lines out of 71,696 total, or 1.15%.
+### Migration metric
 
-### Manifest operations benchmark
+Autoloop tracks `python_lines_migrated_pct = (migrated_python_lines / original_python_lines) * 100`. The best recorded value is **9.89%** (iteration 24, ~7,092 of 71,696 lines). `go build ./...` and `go test ./...` pass on all accepted iterations. Module-specific Python-vs-Go timing data is not yet present in `benchmarks/migration-status.json`.
 
-The existing `scripts/benchmark_manifest_ops.py` benchmark measures Python manifest operation optimizations that remain relevant while Go parity work advances.
+### Manifest operations benchmark (`scripts/benchmark_manifest_ops.py`)
+
+Run locally on 2026-05-13 with `python3 scripts/benchmark_manifest_ops.py`. Results are nondeterministic micro-benchmarks; order-of-magnitude values are stable across runs.
 
 | Scale | `check_collision` speedup | `sync_remove_files` speedup | `cleanup_empty_parents` speedup | Scoped uninstall speedup |
 |---|---:|---:|---:|---:|
-| Current: 10 packages, 50 paths | 21.6x | 1.9x | 1.0x | 1.7x |
-| Growing: 50 packages, 250 paths | 85.4x | 1.7x | 0.6x | 9.7x |
-| Large monorepo: 100 packages, 2,000 paths | 519.8x | 1.7x | 0.5x | 20.5x |
+| Current: 10 pkgs, 50 paths | 24.9x | 1.8x | 0.9x | 2.0x |
+| Growing: 50 pkgs, 250 paths | 117.6x | 1.7x | 0.6x | 9.6x |
+| Large monorepo: 100 pkgs, 2,000 paths | 923.3x | 1.7x | 0.5x | 24.2x |
+
+`cleanup_empty_parents` shows a small regression at scale (0.5x–0.9x) because the batch bottom-up algorithm has higher constant overhead than the legacy per-file walk-up at low deleted-file counts. This is expected and acceptable given the gains on the other three operations.
 
-### Go validation signals
+### Go build/test validation
 
-The Autoloop memory records that the Go module builds cleanly and `go test ./...` passes on accepted iterations. Module-specific Python-vs-Go timing data is not yet present in `benchmarks/migration-status.json`; the progress workflow should publish those timings here once Autoloop starts recording them.
+| Signal | Status | Notes |
+|---|---|---|
+| `go build ./...` | Pass (all accepted iters) | Confirmed in iter-13, iter-22, iter-23, iter-24 notes |
+| `go test ./...` | Pass (all accepted iters) | Confirmed in iter-13, iter-22, iter-23, iter-24 notes |
+| External deps | Unavailable in sandbox | `gopkg.in/yaml.v3` blocked; stdlib-only YAML scanner used for `workflow/parser` and `deps/aggregator` |
 
 ## Next up
 
-The Autoloop memory identifies these likely next targets:
+From the Autoloop memory `Future Directions`:
 
-- `utils/content_hash.py`: SHA-256 tree hashing.
-- `utils/exclude.py`: gitignore-style file exclusion.
-- `utils/console.py`: CLI output helpers.
-- Eventually wire Go packages into the Python CLI via subprocess or replace the entry point.
+- `install/pipeline.py` (741 lines), `install/sources.py` (734 lines), `install/services.py` (734 lines) -- larger install modules, clear patterns.
+- `policy/discovery.py` (1,365 lines) -- highest-impact policy module.
+- `deps/github_downloader.py` (1,686 lines) -- requires HTTP client work.
+- `install/phases/finalize.py` (92), `install/template.py` (140), `install/service.py` (146) -- smaller fill-in modules.
+- Wire Go packages into the Python CLI via subprocess or subprocess-replacement.
+- Address recurring branch-reset issue: consider a stable upstream merge strategy so `benchmarks/migration-status.json` stays current.
 
 ## Operating notes
 
-- Prefer leaf modules first; they have fewer internal APM dependencies and lower integration risk.
-- Keep Go implementations stdlib-only when sandbox networking blocks external module fetches.
-- Continue recording every accepted iteration in `benchmarks/migration-status.json` so this page can report real migration and benchmark data instead of estimates.
+- Prefer leaf modules first; fewer internal APM dependencies and lower integration risk.
+- Keep Go implementations stdlib-only when sandbox networking blocks external module fetches (e.g., use a line-scanner for YAML frontmatter instead of `gopkg.in/yaml.v3`).
+- Record every accepted iteration in `benchmarks/migration-status.json` so this page can report complete module data.
 
-Last seeded: 2026-05-12 17:11 UTC.
+Last updated: 2026-05-13 15:14 UTC.