Remove conda: CI and docs build from the container (#733)

* ci: run pytest inside the dev image; retire the conda test suite

The Docker image (slim Python + apt + uv-frozen wheels) is already the
deployment environment; testing a separate conda env in ci-release.yml
tested something nothing ships — and that conda path had silently rotted.

Move the suite into the artifact: deploy-image.yml now runs `pytest`
inside the dev image alongside the existing example-pipeline and binary
smokes. Split build/test from publish — pull_request builds and tests
without pushing (works for fork PRs too); only pushes to develop/main/
master publish to ghcr. Delete ci-release.yml.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* ci: build API docs in the dev image; drop conda from doc deploy

cd.yml built docs via conda + install_shapepipe + `conda install pandoc`.
Build them inside the dev image instead (its `doc` extra carries sphinx +
myst-parser + the theme). pandoc was vestigial — docs use myst_parser and
autodoc, no nbsphinx/notebooks — so it's dropped, along with the old
importlib-metadata workaround.

Delete doc-tests.yml: a workflow_dispatch-only duplicate of cd.yml's
build with no deploy.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: remove conda machinery (install_shapepipe, environment*.yml)

Nothing builds from conda anymore: the container uses apt + uv-frozen
wheels, CI tests inside that image, and docs build there too. The
install_shapepipe script (conda env creation + from-source C builds) and
the production/dev environment files served only the retired conda path —
including the unsolvable production pin (astropy==5.2 vs python=3.12),
which is now moot rather than something to fix.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs: document container/uv install; drop conda instructions

- installation.md: remove the deprecated conda section and the
  install_shapepipe-based MPI/uninstall sections; keep container
  (recommended) + library-only (pip / `.[dev]`) paths. Note the image
  bundles the astromatic binaries and MPI.
- pipeline_tutorial.md: point the quick-start install at installation.md
  (container), not a conda env + a broken anchor.
- CONTRIBUTING.md: cite pyproject.toml / uv.lock for version compat
  instead of environment.yml.

Cluster job scripts under example/pbs/ and scripts/sh/ still reference
personal conda envs; those are operational, cluster-specific, and out of
scope here — tracked as follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: cailmdaley

.github/workflows/cd.yml

@@ -1,41 +1,49 @@
 name: Deploy API Docs
 
+# Build the Sphinx API docs inside the dev image (same slim-Python + uv
+# environment everything else uses — no conda) and publish to gh-pages.
+# autodoc imports shapepipe, so it needs the full runtime; the dev image's
+# `doc` extra provides sphinx + myst-parser + the theme on top of that.
 on:
   push:
     branches:
-     - master
-     - main
+      - master
+      - main
+  workflow_dispatch:
 
 jobs:
-
   api:
     name: Deploy API Documentation
     runs-on: ubuntu-latest
 
     steps:
-      - uses: actions/checkout@ee0669bd1cc54295c223e0bb666b733df41de1c5 # v2.7.0
+      - name: Checkout repository
+        uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
 
-      - name: Set up conda
-        uses: conda-incubator/setup-miniconda@9f54435e0e72c53962ee863144e47a4b094bfd35 # v2.3.0
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
         with:
-          auto-update-conda: true
-          python-version: 3.9
-          auto-activate-base: true
-
-      - name: Install dependencies
-        shell: bash -l {0}
-        run: |
-          ./install_shapepipe --develop --no-exe --no-mpi
-          conda activate shapepipe
-          python -m pip install --upgrade importlib-metadata
-          conda install -c conda-forge pandoc
+          driver-opts: network=host
 
+      - name: Build dev image
+        uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6.19.2
+        with:
+          context: .
+          target: dev
+          load: true
+          tags: shapepipe-dev:docs
+          cache-from: type=gha
+          cache-to: type=gha,mode=max
+
+      # Generate API stubs and build the HTML inside the image. docs/ is
+      # bind-mounted so sphinx-apidoc's generated rst and the _build output
+      # land back on the runner for the deploy step.
       - name: Build API documentation
-        shell: bash -l {0}
         run: |
-          conda activate shapepipe
-          sphinx-apidoc -t docs/_templates -feTMo docs/source shapepipe shapepipe/modules/*_runner.py
-          sphinx-build -E docs/source docs/_build
+          docker run --rm -v "${GITHUB_WORKSPACE}/docs:/app/docs" shapepipe-dev:docs bash -c "
+            sphinx-apidoc -t docs/_templates -feTMo docs/source shapepipe shapepipe/modules/*_runner.py &&
+            sphinx-build -E docs/source docs/_build
+          "
 
       - name: Deploy API documentation
         uses: peaceiris/actions-gh-pages@cf301857435c1f95963b21d58f452a7617770d60 # v3.5.9

.github/workflows/ci-release.yml

@@ -1,10 +1,31 @@
-name: Create and publish a Docker image
-on: [push, workflow_dispatch]
+name: Docker image — build, test, publish
+
+# Single source of truth for ShapePipe's environment is the Dockerfile
+# (slim Python + apt system deps + uv-frozen wheels). This workflow builds
+# that image, runs the test suite *inside it* — so CI tests exactly what
+# ships — and publishes to ghcr only on pushes to the integration branches.
+#
+#   pull_request → build + test, no publish (also works for fork PRs)
+#   push         → build + test + publish (:develop, :latest, …)
+on:
+  push:
+    branches:
+      - develop
+      - main
+      - master
+  pull_request:
+    branches:
+      - develop
+      - main
+      - master
+  workflow_dispatch:
+
 env:
   REGISTRY: ghcr.io
   IMAGE_NAME: ${{ github.repository }}
+
 jobs:
-  build-and-push-image:
+  build-test-publish:
     runs-on: ubuntu-latest
     permissions:
       contents: read
@@ -13,13 +34,6 @@ jobs:
       - name: Checkout repository
         uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
 
-      - name: Log in to the Container registry
-        uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3.7.0
-        with:
-          registry: ${{ env.REGISTRY }}
-          username: ${{ github.actor }}
-          password: ${{ secrets.GITHUB_TOKEN }}
-
       - name: Set up Docker Buildx
         uses: docker/setup-buildx-action@8d2750c68a42422c14e847fe6c8ac0403b4cbd6f # v3.12.0
         with:
@@ -41,6 +55,10 @@ jobs:
           flavor: |
             suffix=-runtime,onlatest=true
 
+      # ----------------------------------------------------------------
+      # Build + test (every event)
+      # ----------------------------------------------------------------
+
       # Build runtime first (smaller, used to smoke-test pipeline binaries)
       - name: Build runtime (load)
         uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6.19.2
@@ -84,15 +102,35 @@ jobs:
           cache-to: type=gha,mode=max
 
       # Verify the dev-only additions are present and runnable.
-      - name: Test dev — interactive tools and test extras
+      - name: Test dev — interactive tools
         run: |
           IMAGE=$(echo "${{ steps.meta-dev.outputs.tags }}" | head -n1)
           docker run --rm "$IMAGE" vim --version | head -n1
           docker run --rm "$IMAGE" rg --version | head -n1
-          docker run --rm "$IMAGE" pytest --version
 
-      # Push both targets
+      # The actual test suite, run inside the shipped image — replacing the
+      # retired conda-based suite. pytest exercises the same wheels, binaries,
+      # and Python (3.12) that production runs on, not a parallel environment.
+      # pyproject's addopts add `--cov=shapepipe`; COVERAGE_FILE is set to /tmp
+      # in the image so it works on read-only filesystems too.
+      - name: Test dev — pytest suite
+        run: |
+          IMAGE=$(echo "${{ steps.meta-dev.outputs.tags }}" | head -n1)
+          docker run --rm "$IMAGE" pytest
+
+      # ----------------------------------------------------------------
+      # Publish (push events only — never on pull_request, incl. forks)
+      # ----------------------------------------------------------------
+      - name: Log in to the Container registry
+        if: github.event_name == 'push'
+        uses: docker/login-action@c94ce9fb468520275223c153574b00df6fe4bcc9 # v3.7.0
+        with:
+          registry: ${{ env.REGISTRY }}
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
       - name: Push runtime
+        if: github.event_name == 'push'
         uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6.19.2
         with:
           context: .
@@ -103,6 +141,7 @@ jobs:
           cache-from: type=gha
 
       - name: Push dev
+        if: github.event_name == 'push'
         uses: docker/build-push-action@10e90e3645eae34f1e60eeb005ba3a3d33f178e8 # v6.19.2
         with:
           context: .

.github/workflows/deploy-image.yml

@@ -159,7 +159,7 @@ In some cases it may be necessary to modify the unit tests, but this should be c
 
 All contributions should adhere to the following style guides currently implemented in ShapePipe.
 
-1. All code should be compatible with the Python package versions listed in the [Conda environment](https://github.com/CosmoStat/shapepipe/blob/develop/environment.yml).
+1. All code should be compatible with the Python package versions declared in [`pyproject.toml`](https://github.com/CosmoStat/shapepipe/blob/develop/pyproject.toml) (pinned exactly in `uv.lock`).
 
 1. All code should adhere to [PEP8](https://www.python.org/dev/peps/pep-0008/) standards.