Add shared .felt knowledge base + CLAUDE.md

.felt/dependabot-pr-triage/dependabot-pr-triage.md

@@ -0,0 +1,86 @@
+---
+name: Triage open dependabot uv.lock PRs
+status: closed
+tags:
+    - shapepipe
+    - constitution
+    - pr
+    - dependabot
+created-at: 2026-05-27T11:57:31.076323766+02:00
+closed-at: 2026-05-28T15:27:23.026Z
+outcome: All 6 PRs merged in-conversation (727 idna, 722 mistune, 724 gitpython, 720 jupyterlab + pyproject floor, 721 jupyter-server, 726 urllib3). 19 of 20 Dependabot alerts cleared. Last one (sqlitedict) dismissed as tolerable_risk; see [[shapepipe/sqlitedict-pickle-smell]]. Shuttle daemon didn't pick up the dispatch — root cause unknown, worth a separate look.
+shuttle:
+    enabled: true
+    kind: oneshot
+    project_dir: .
+    agent: pi-gpt-5.4
+tempered: true
+---
+
+## Why this exists
+
+`uv.lock` landed in `develop` on 2026-05-05 via PR #719. Before that there was no lockfile and Dependabot had nothing to pin against. After that landing, Dependabot's security-updates channel (which fires automatically without a `dependabot.yml` config) started opening one PR per GHSA-touched package in the lockfile. Six have accumulated; none merged. CI is green on the ones spot-checked. Maintainers have write access and merge their own PRs (e.g. #718), so these can be merged directly.
+
+This fiber's job is the one-time backlog clearance. The longer-term policy (a `.github/dependabot.yml` that groups bumps, schedules them, scopes ecosystems) is **out of scope** — that's being decided in parent conversation.
+
+## Desired State
+
+All six PRs (#720, #721, #722, #724, #726, #727) closed — each either squash-merged into `develop`, or closed with a clear reason posted as a PR comment.
+
+`gh pr list --repo CosmoStat/shapepipe --author "app/dependabot" --state open` returns empty when this is done.
+
+Each merged PR's commit appears on `origin/develop` as a single squash commit with a sensible message.
+
+## Scope (what to leave alone)
+
+- **Do not write `.github/dependabot.yml`.** Policy is being decided elsewhere; writing it from here would race with that conversation.
+- **Do not modify `pyproject.toml`.** If a bump would require widening or narrowing a direct-dep ceiling, leave a comment on the PR explaining the gap and move on. Don't push edits.
+- **Do not touch non-dependabot PRs or issues.** There's other backlog (issues #709, #711, #712 from April; older issues from Martin/Lucie/Axel) — that's a separate sweep.
+- **Do not edit the lockfile manually.** If a Dependabot PR has gone stale (lockfile conflict against develop), trigger a rebase via `@dependabot rebase` and move on; don't resolve conflicts by hand.
+- **Don't merge across pyproject ceilings.** If `pyproject.toml` has a direct constraint like `package<X` and the bump would cross it, that's not a clean merge — comment and skip.
+
+## The six PRs
+
+| # | Package | Bump | Notes |
+|---|---------|------|-------|
+| 720 | jupyterlab | 4.5.6 → 4.5.7 | Direct dep (`jupyterlab>=4.3` in pyproject's `jupyter` extra), patch bump |
+| 721 | jupyter-server | 2.17.0 → 2.18.0 | Transitive |
+| 722 | mistune | 3.2.0 → 3.2.1 | Transitive, patch bump |
+| 724 | gitpython | 3.1.47 → 3.1.50 | Transitive, replaced earlier #723 (3.1.49) |
+| 726 | urllib3 | 2.6.3 → 2.7.0 | Transitive |
+| 727 | idna | 3.13 → 3.15 | Transitive, CVE-2026-45409 (medium severity DoS) |
+
+All sampled PRs are lockfile-only (3+/3- in `uv.lock`). Spot-checked CI was green on #727 and #720; assume the others are too but verify.
+
+## Per-PR workflow
+
+For each PR, in this order (lowest-risk-first: idna's CVE is the cleanest justification, lockfile-only patches before minors):
+
+```
+727 → 722 → 724 → 720 → 721 → 726
+```
+
+1. **Inspect.** `gh pr view <N> --repo CosmoStat/shapepipe --json files,additions,deletions,mergeable,mergeStateStatus`. Confirm `files` is only `uv.lock`, additions+deletions are small (~3-30).
+2. **Check CI.** `gh pr checks <N> --repo CosmoStat/shapepipe`. Required: `build-and-push-image` passes. If failing, read logs (`gh run view <run-id> --log-failed`) to understand whether it's a real regression or flaky.
+3. **Check freshness.** If `mergeStateStatus` is `BEHIND` or shows conflicts, comment `@dependabot rebase` and move to the next PR; loop back later.
+4. **Verify no pyproject ceiling violation.** `grep -E "<package_name>" pyproject.toml` — if there's a direct upper-bound that the bump crosses, stop and comment.
+5. **Squash-merge.** `gh pr merge <N> --repo CosmoStat/shapepipe --squash`. Default commit title is fine (`Bump <pkg> from X to Y (#N)`). The repo doesn't auto-delete merged branches (`delete_branch_on_merge: false`) — leave that as-is, Dependabot handles its own branches.
+6. **Brief pause.** After each merge, the other dependabot branches go stale on `uv.lock`. Dependabot auto-rebases on `develop` updates, but it takes a moment. Move to the next PR; if `mergeStateStatus` shows behind, `@dependabot rebase` and circle back.
+
+## Evidence (how to know you're done)
+
+- `gh pr list --repo CosmoStat/shapepipe --author "app/dependabot" --state open` returns `no open pull requests`.
+- `git log origin/develop --oneline -10` shows the squash-merged bump commits.
+- Any PR closed without merge has a comment explaining why (e.g. "deferring, pyproject ceiling at `<X`").
+
+## Open Questions
+
+None for the worker — kick everything that needs human judgment back to the PR as a comment and skip that PR. The parent conversation will handle anything left after the loop.
+
+## Handoff (when exiting)
+
+Update this fiber's `outcome:` to reflect what landed (e.g. "5 merged, 1 deferred for pyproject ceiling on `<pkg>`"), set `status: closed`, leave `tempered` unset (the maintainer will accept once verified). Append a `felt history` event with the merge SHAs and any deferred-PR rationales.
+
+## Skills
+
+Activate `felt` (you'll touch the fiber). The `shuttle` skill is already loaded by virtue of being the dispatched worker.

.felt/docker-multistage/docker-multistage.md

@@ -0,0 +1,29 @@
+---
+name: 'Docker multi-stage: runtime + dev targets'
+tags:
+    - shapepipe
+    - docker
+    - infra
+    - docs
+created-at: 2026-04-28T14:09:33.463318009+02:00
+outcome: 'Multi-stage Dockerfile (runtime + dev) committed (24c90d11) addressing Martin''s missing-pytest, RO-fs, and missing-vi feedback. Three-layer docs (pyproject / uv.lock / Dockerfile) drafted in docs/source/container.md (uncommitted, awaiting review). Mutation model: pre-bake all extras in dev + chmod go+rwX /app; runtime intentionally immutable; live uv sync --frozen not supported.'
+insights:
+    design-split:
+        claim: 'Splitting runtime vs dev resolves Martin report without compromising image hygiene: runtime stays minimal+immutable for canfar batch jobs; dev pre-bakes all extras (so uv sync --extra X is unnecessary) AND chmod go+rwX /app (so uv add / uv pip install still work in writable apptainer sandboxes). Live uv sync --frozen is intentionally not supported — the operation it enables (toggle extras) is dissolved by pre-baking.'
+    sandbox-bindmount:
+        claim: 'A common dev workflow uses apptainer --writable sandbox containers for months at a time, with a host-side clone of the repo bind-mounted in and pip install -e /mnt/clone done inside. Implication: the in-image COPY . /app + pip install -e . is scratch for this workflow (canfar batch uses it; sandbox shadows it). chmod -R go+rwX /app at build time only needs to support ad-hoc uv add / uv pip install, not full uv sync --frozen.'
+    three-layers:
+        claim: 'pyproject.toml = abstract minimums (SSOT for what shapepipe needs); uv.lock = concrete pins generated by uv from pyproject (commit it; never hand-edit; uv sync --frozen fails if drifted from pyproject); Dockerfile = system-level deps + invocations (no Python deps duplicated). Asymmetry: Python deps go pyproject+lockfile (reproducible/auditable); system deps go Dockerfile (Debian versioning). Documented in docs/source/container.md.'
+decisions:
+    mutation:
+        label: Live mutation model
+        rationale: 'The sandbox dev workflow is months-long apptainer with host repo bind-mounted (pip install -e /mnt/clone shadows the in-image install). Mutation perms are needed for ad-hoc ''uv add foo'' / ''uv pip install foo'' during exploration, NOT for toggling extras. Pre-baking all extras into dev dissolves the toggle-extras case (Martin''s actual error); chmod go+rwX /app keeps the venv writable for ad-hoc additions. Long-term promotion path: copy mutated pyproject+uv.lock back to host, commit, rebuild. Live ''uv sync --frozen'' is not supported because runtime is intentionally immutable.'
+        default: pre-bake-all
+        options:
+            fully-immutable:
+                label: Both targets immutable; require rebuild for any change
+            live-sync:
+                label: Support live uv sync --extra X by making venv mutable in runtime too
+            pre-bake-all:
+                label: Pre-bake all extras into dev; runtime stays immutable; sandbox mutation via uv add / uv pip install on writable venv
+---

.felt/docker-uv-revert/docker-uv-revert.md

@@ -0,0 +1,107 @@
+---
+name: 'Docker: revert skaha→python base, switch to uv lockfile'
+status: active
+tags:
+    - shapepipe
+    - docker
+    - infra
+created-at: 2026-04-27T11:26:45.677512058+02:00
+outcome: 'PR #719 (chore: switch Dockerfile to slim Python + uv lockfile) opened and CI-green on first try (3m31s); ready for Martin''s review. Drops conda double-install, makes pyproject SSOT + uv.lock the pinned manifest, switches WeightWatcher from sed-patched source build to Debian''s pre-patched 1.12+dfsg-3 package, adds binary smoke tests to deploy-image.yml.'
+decisions:
+    base:
+        label: Base image
+        rationale: Conda double-install was the actual problem; cleanest resolution is to drop conda entirely. Martin's canfar concern is satisfied as long as the slim image works on canfar.
+        default: python-slim
+        options:
+            python-slim:
+                label: python:3.12-slim with deps via uv
+            skaha-astroml:
+                label: 'skaha-astroml: brings conda Python; pip+conda double-install everything; image bloats 5-10 GB'
+                excluded: true
+                excluded_reason: Conda was the source of the bloat and double-install
+            skaha-no-conda:
+                label: skaha base + uv (avoid conda's python)
+                excluded: true
+                excluded_reason: Compromise with unclear benefit; revisit if canfar deployment specifically needs skaha
+    ci:
+        label: CI binary smoke test
+        rationale: The original failure mode was a binary that built fine but couldn't run on canfar. Adding sextractor/weightwatcher invocation to deploy-image.yml will catch that class of regression at PR time, not deployment time.
+        default: binary-smoke
+        options:
+            binary-smoke:
+                label: Build + invoke sextractor and weightwatcher inside built image
+            build-only:
+                label: Build-only (current); misses runtime regressions like the weightwatcher/sextractor failure on canfar
+                excluded: true
+    deps:
+        label: Dependency management
+        rationale: We want control over when upstream changes propagate; lockfile gives that. pyproject becomes the SSOT for what shapepipe needs; uv.lock is the pinned manifest that's fully reproducible.
+        default: uv-lock
+        options:
+            hand-pinned:
+                label: Hand-pinned in Dockerfile (status quo); pyproject and Dockerfile both list deps and drift
+                excluded: true
+                excluded_reason: Source of repeated bugs; no upstream-pinning discipline
+            poetry:
+                label: Poetry + poetry.lock
+                excluded: true
+                excluded_reason: Same shape as uv but ~10x slower; uv is the modern default
+            uv-lock:
+                label: uv + pyproject + uv.lock; uv sync --frozen in Dockerfile
+    modernize:
+        label: Modernize package versions
+        rationale: 'We determined which versions MUST stay pinned: only ngmix (Axel''s stable_version branch — replacement is tracked separately). Everything else can move to current latest because uv resolved cleanly and CI smoke test still passes (3m42s). If a real pipeline run on canfar surfaces a numpy-2 / pandas-3 break, the fix is a targeted constraint + uv lock, not a wholesale revert.'
+        default: stay-current
+        options:
+            stay-conservative:
+                label: Keep pre-v2 minimums (numpy 1.26, astropy 6.1, pandas 2.2); only bump when forced
+                excluded: true
+                excluded_reason: Drift between pyproject signal and lockfile reality; loses the chance to surface numpy-2/pandas-3 incompatibilities at PR time when CI is fast
+            stay-current:
+                label: Bump pyproject minimums to current major versions (numpy 2, astropy 7, pandas 3, galsim 2.8, mpi4py 4.1, etc.); pin ngmix to Axel's stable_version branch
+insights:
+    ci-fast:
+        claim: 'First CI run on PR #719 went green in 3m31s. uv installed 238 packages in 322ms — everything resolved to prebuilt wheels, no source compilation of galsim/mpi4py/python-pysap/etc. Massive speedup vs. previous build.'
+    failure-mode:
+        claim: The triggering failure was a binary (weightwatcher / sextractor) that built fine but couldn't run on canfar. Build-only CI doesn't catch that class; binary-version CI does.
+    patches-explained:
+        claim: WeightWatcher 1.12 (2014) source uses pre-GCC-10 'common symbol' globals (prefstruct prefs; etc. in headers). GCC 10 (Apr 2020) flipped default to -fno-common, breaking the build. Old Dockerfile fixed it inline with sed converting headers to extern + adding single .c definition. Debian's weightwatcher 1.12+dfsg-3 package applies an equivalent patch upstream of the build, which is why apt install just works.
+    skaha-uv-incompatible:
+        claim: skaha-astroml's value comes from conda's pre-built scientific stack; layering uv on top defeats that purpose, so "skaha + uv (no conda)" is incoherent rather than just hard.
+    weightwatcher-apt:
+        claim: Debian bookworm packages WeightWatcher 1.12 directly — no need to build from source with the patched headers the v2 Dockerfile uses.
+---
+
+## Where the work landed
+
+[CosmoStat/shapepipe#719](https://github.com/CosmoStat/shapepipe/pull/719) — draft PR.
+
+Commit: `9af10257 chore: switch Dockerfile to slim Python + uv lockfile`. Four files:
+
+- `Dockerfile` — `python:3.12-slim-bookworm` base, apt for astromatic binaries (psfex, source-extractor, weightwatcher), uv for Python deps.
+- `pyproject.toml` — bumped minimums to current versions; new `jupyter` extra holds ipython/jupyterlab/snakemake (previously inlined in Dockerfile).
+- `uv.lock` — 286 packages resolved by `uv lock`.
+- `.github/workflows/deploy-image.yml` — split `Test` into `Test — binaries` and `Test — shapepipe entry point`.
+
+## Lockfile workflow notes
+
+```bash
+uv lock                # regenerate after editing dependencies
+uv add 'pkg>=1.2'      # adds to pyproject AND uv.lock
+uv lock --upgrade      # bump pinned versions deliberately
+uv sync --frozen       # in Dockerfile; fails if pyproject/uv.lock drift
+```
+
+The `--frozen` flag is the discipline mechanism: a stale lockfile cannot ship.
+
+## Followups
+
+- Watch CI on #719. The slim-base apt list is conjectural — galsim/mpi4py/python-pysap pull a lot of system deps and we may need to add more (`libatlas-base-dev`, `libblas-dev`, etc).
+- If CI needs anything beyond what's in the apt block, that's the surface that benefits from a [[shapepipe/prs-in-flight]] note for next time.
+- After this lands, [[shapepipe/prs-in-flight]] PRs #708 and #714 may need a small rebase.
+- Optional: separate `Dockerfile.canfar` building on skaha if there's a concrete deployment reason. Currently conjectural — Martin floated it but we agreed slim should work on canfar.
+
+## Connections
+
+- [[shapepipe]] — root
+- [[shapepipe/prs-in-flight]] — touches the testing-scaffold xfail set and the develop-bugs PR

.felt/fabian-coord-bug/fabian-coord-bug.md

@@ -0,0 +1,10 @@
+---
+name: Fabian's coord-propagation bug + image-sim code on github
+tags:
+    - shapepipe
+    - bug
+    - collaboration
+    - future
+created-at: 2026-04-27T11:26:52.878118978+02:00
+outcome: 'Fabian: 1-line fix in shapepipe needs porting; first need him to put image-sim code/configs on github so it''s testable. Beg if necessary.'
+---

.felt/ngmix-update/ngmix-update.md

@@ -0,0 +1,9 @@
+---
+name: ngmix library upgrade + Lucy wrapper sync
+tags:
+    - shapepipe
+    - ngmix
+    - future
+created-at: 2026-04-27T11:26:51.026191639+02:00
+outcome: 'Future: replace Axel''s stable_version fork with upstream ngmix; reconcile with Lucy''s cleaned-up wrapper from her visit'
+---