Skip to content

Remove obsolete in-ShapePipe rho-stats / PSF-diagnostic plotting#736

Closed
cailmdaley wants to merge 1 commit into
developfrom
cleanup/remove-rho-stats-psf-plots
Closed

Remove obsolete in-ShapePipe rho-stats / PSF-diagnostic plotting#736
cailmdaley wants to merge 1 commit into
developfrom
cleanup/remove-rho-stats-psf-plots

Conversation

@cailmdaley

Copy link
Copy Markdown
Contributor

What

Removes the dead in-ShapePipe PSF-systematics plotting path (mean-shape focal-plane plots + rho statistics).

Martin confirmed on #657 (2026-04-22): "the rho stats within shapepipe are obsolete, and have been superseded by sp_validation / cosmo_val, which uses shear_psf_leakage." Per the stack division — ShapePipe produces catalogues, sp_validation validates — this path is deleted rather than migrated or preserved.

Changes

  • Deleted modules/mccd_plots_runner.py and mccd_package/mccd_plot_utilities.py. Worth noting: despite the docstring's mention of "rho statistics", the code only ever computed mean shapes and ellipticity histograms — there is no treecorr/stile correlation-function code in it. The module is auto-discovered by filename glob, so deleting the file fully deregisters it.
  • Deleted example/cfis/config_Pl_mccd.ini and config_valjoint_Pl_mccd.ini — these existed solely to run mccd_plots_runner. Rather than leave them as husks with an empty MODULE = line, they're removed whole.
  • Edited the six remaining configs that referenced the runner (config_exp_mccd, config_MsPl_psfex, config_MsPl_mccd, config_MsPl_stars, cfis_simu/config_tile_Sx_exp_mccd, cfis/defunct/config_tile_Sx_exp_mccd): dropped mccd_plots_runner from the MODULE list and removed the orphaned [MCCD_PLOTS_RUNNER] section.
  • Edited mccd_package/__init__.py (removed the MCCD_PLOTS_RUNNER docstring block and the mccd_plot_utilities __all__ entry) and the doc references in pipeline_tutorial.md / post_processing.md, pointing PSF diagnostics at sp_validation/cosmo_val.

Two deliberate scope decisions (please sanity-check)

  1. stile was already gone. It is not referenced anywhere in src/, example/, docs/, pyproject.toml, or uv.lock — it was never a declared dependency and isn't imported. Nothing to remove; flagging so the absence isn't read as an oversight.

  2. random_cat is deliberately kept. The cleanup brief raised an open question — is random_cat rho-stats-only? On inspection it is a general random-catalogue generator (random points within the survey mask, healpix output; config_Rc.ini) for clustering / LSS null tests — Landy-Szalay-style randoms, not PSF rho statistics. It's authored by Martin and is independent of the removed plotting path, so bundling its removal would have overreached what was flagged as obsolete. If you do want it gone too, happy to do it in a follow-up.

Verification

In-image pytest suite green against this branch — 250 passed (run via Apptainer sandbox from ghcr.io/cosmostat/shapepipe:develop on candide). Structural tier (config-parse, runner-metadata, imports) covers the dangling-config / dangling-import cases.

grep -rIn "mccd_plots_runner\|MCCD_PLOTS_RUNNER\|mccd_plot_utilities\|stile\|rho_stats" src/ example/ docs/  →  clean

🤖 Generated with Claude Code

— Claude on behalf of Cail

The in-ShapePipe PSF-systematics plotting path (mean-shape focal-plane plots
and rho statistics) is dead code. Martin confirmed on #657 (2026-04-22) that
"the rho stats within shapepipe are obsolete, and have been superseded by
sp_validation / cosmo_val, which uses shear_psf_leakage." Per the stack
division, ShapePipe produces catalogues and sp_validation validates them, so
this path is deleted rather than migrated.

Removed:
- modules/mccd_plots_runner.py and mccd_package/mccd_plot_utilities.py
  (mean-shape plotting; the "rho statistics" were only ever a docstring
  promise — the code computed mean shapes and histograms, no treecorr/stile).
- The MCCD_PLOTS_RUNNER docstring section and the mccd_plot_utilities entry
  in mccd_package/__init__.py __all__.
- example/cfis/config_Pl_mccd.ini and config_valjoint_Pl_mccd.ini, which
  existed solely to run mccd_plots_runner (deleted whole rather than gutted
  to an empty MODULE list).
- mccd_plots_runner from the MODULE list and the orphaned [MCCD_PLOTS_RUNNER]
  section in the six remaining configs that referenced it.
- Doc references in pipeline_tutorial.md and post_processing.md, updated to
  point PSF diagnostics at sp_validation/cosmo_val.

Note: stile is not referenced anywhere in src/example/docs/pyproject/uv.lock —
it was already vestigial, so there was nothing to remove there.

Note: random_cat (random_cat_runner + random_cat_package) is deliberately
kept. It is a general random-catalogue generator for clustering/LSS null
tests, not part of the PSF rho-stats path, so it falls outside what Martin
flagged as obsolete.

In-image pytest suite green (250 passed).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cailmdaley added a commit that referenced this pull request May 30, 2026
…t question

Worker delivered both cleanups as open PRs against develop (not merged):
#736 (rho-stats/PSF-plot removal) and #737 (candide container scripts).
Resolves the open question — random_cat is a general LSS random-catalogue
generator, not rho-stats, so it was kept. Records that stile was already
vestigial and the MPI-verification gap.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@cailmdaley

Copy link
Copy Markdown
Contributor Author

Closing this — on review it over-reaches what was actually authorized, and the part that was authorized already shipped.

The rho-stats removal Martin signed off on (#657: "option 1, delete") landed in #715 (merged 2026-04-23), which surgically stripped the rho-stats code out of mccd_plot_utilities.py / mccd_plots_runner.py and dropped stile/treecorr. As of develop, those modules carry no rho/stile/treecorr code — they're purely plot_meanshapes() / mean_shapes_plot() (focal-plane ellipticities, sizes, residuals, 1D histograms).

So everything this PR removes is the meanshapes/ellipticity plotting, not rho-stats. And on #715 Martin was explicit about that path:

Let's keep meanshapes, this is very useful and can be run on merged star and PSF catalogues as well. Not sure whether this is what @fabianhervaspeters is using for the 2D plots in the catalogue paper.

Deleting it here would directly contradict that, and potentially break a figure path for the catalogue paper. Net, there's nothing left to salvage in this PR — closing rather than trimming.

Apologies for the noise, @martinkilbinger — the branch was cut against a stale assumption that mccd_plots_runner was still the rho-stats path.

— Claude on behalf of Cail

@cailmdaley cailmdaley closed this May 31, 2026
cailmdaley added a commit that referenced this pull request May 31, 2026
Done-state re-verified fresh: PR #736 (rho-stats removal) and #737 (candide
scripts → container) both OPEN against develop, no reviews, not merged. Noted
PR #636 (rho-stats feature PR) needs Martin to reconcile against #736's removal.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cailmdaley added a commit that referenced this pull request May 31, 2026
…open & green

Cold survey found zero remaining worker work. PR #736 (rho-stats removal)
and #737 (candide scripts via apptainer) are both open, mergeable, and
CI-green. The only remaining step is Martin's review, which is outside
worker scope. Includes mechanical felt reindex churn.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cailmdaley added a commit that referenced this pull request May 31, 2026
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant