Skip to content

Commit f6b985d

Browse files
committed
Add turnkey container build/push + model-fetch scripts for NIS submission
Since the automated session cannot build/push the GPU container or reach quay.io/pytorch (no Docker daemon, blocked hosts) nor open the cross-repo PR (GitHub scope), provide everything pre-filled so the maintainer can run it in one sitting on a GPU/Docker host: - containers/nis/build_and_push.sh: build + push the container, with the cu128 LibTorch URL from cpp/README.md / build_main_hummer.sh baked in. - containers/nis/fetch_models.sh: gdown the TorchScript model weights from the G-drive folder documented in cpp/README.md (runtime models + Zenodo test data). - docs/nfcore_submission.md: concrete prerequisite commands using both scripts. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TRbz5C1u1pxcvB5DGYVZMo
1 parent 8b57082 commit f6b985d

3 files changed

Lines changed: 90 additions & 10 deletions

File tree

containers/nis/build_and_push.sh

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
#!/usr/bin/env bash
2+
#
3+
# Build and push the CellPheno NIS GPU container referenced by
4+
# modules/nf-core/cellpheno/nis (container "quay.io/nf-core/cellpheno-nis:1.0.0").
5+
#
6+
# Run this on a machine with Docker + a CUDA GPU + push access to the target
7+
# registry. It is the step the automated session cannot do (no Docker daemon,
8+
# blocked registry hosts). Once the image is pushed, the module's only remaining
9+
# lint failure (container-not-reachable) clears.
10+
#
11+
# Usage:
12+
# bash containers/nis/build_and_push.sh [REGISTRY_IMAGE] [VERSION]
13+
# Examples:
14+
# bash containers/nis/build_and_push.sh quay.io/nf-core/cellpheno-nis 1.0.0
15+
# bash containers/nis/build_and_push.sh ghcr.io/chrisa142857/cellpheno-nis 1.0.0
16+
set -euo pipefail
17+
18+
IMAGE="${1:-quay.io/nf-core/cellpheno-nis}"
19+
VERSION="${2:-1.0.0}"
20+
REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
21+
22+
# LibTorch CUDA build. The repo's cpp/README.md downloads LibTorch from
23+
# download.pytorch.org; cpp/build_main_hummer.sh targets CUDA 12.8 (cu128).
24+
# The Dockerfile defaults to the matching cu128 LibTorch; override via build-arg
25+
# if you need a different CUDA/LibTorch version.
26+
LIBTORCH_URL="${LIBTORCH_URL:-https://download.pytorch.org/libtorch/cu128/libtorch-cxx11-abi-shared-with-deps-2.7.0%2Bcu128.zip}"
27+
28+
echo ">> Building ${IMAGE}:${VERSION} (LibTorch: ${LIBTORCH_URL})"
29+
docker build \
30+
-f "${REPO_ROOT}/containers/nis/Dockerfile" \
31+
--build-arg "LIBTORCH_URL=${LIBTORCH_URL}" \
32+
--build-arg "NIS_VERSION=${VERSION}" \
33+
-t "${IMAGE}:${VERSION}" \
34+
"${REPO_ROOT}"
35+
36+
echo ">> Pushing ${IMAGE}:${VERSION}"
37+
echo " (run 'docker login quay.io' first if pushing to quay.io/nf-core)"
38+
docker push "${IMAGE}:${VERSION}"
39+
40+
echo ">> Done. If you used a registry other than quay.io/nf-core/cellpheno-nis,"
41+
echo " update the container line in modules/nf-core/cellpheno/nis/main.nf to match."

containers/nis/fetch_models.sh

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
#!/usr/bin/env bash
2+
#
3+
# Fetch the TorchScript model weights the CellPheno NIS binary loads at runtime.
4+
# These are NOT baked into the container; they are passed to the module as the
5+
# `models` input (and are needed to prepare real test data for nf-core).
6+
#
7+
# Source: cpp/README.md -> G-drive model folder.
8+
#
9+
# Usage:
10+
# pip install gdown
11+
# bash containers/nis/fetch_models.sh [DEST_DIR]
12+
set -euo pipefail
13+
14+
DEST="${1:-downloads/resource}"
15+
# G-drive folder with the NIS .pt models (see cpp/README.md).
16+
GDRIVE_MODELS_FOLDER="https://drive.google.com/drive/folders/12YGRtoW4DHftVyhaGoZMl-xdc02Mj9SB"
17+
18+
mkdir -p "${DEST}"
19+
echo ">> Downloading NIS models into ${DEST}"
20+
gdown --folder "${GDRIVE_MODELS_FOLDER}" -O "${DEST}"
21+
22+
echo ">> Expected files (loaded by cpp/main.cpp):"
23+
echo " nis_unet_cpu.pt grad_2Dto3D_<device>.pt gnn_message_passing_<device>.pt"
24+
echo " gnn_classifier_<device>.pt flow_3DtoSeed.pt"
25+
echo ">> Note: only cuda:0 and cuda:1 device variants are provided."

docs/nfcore_submission.md

Lines changed: 24 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -9,23 +9,37 @@ Reference: <https://nf-co.re/docs/contributing/contribute-components>
99

1010
## One-time prerequisites (resolve the two external items)
1111

12-
1. **Publish the container.** Build and push the image referenced by the module.
13-
For the upstream PR the cleanest registry is the nf-core org namespace
14-
(passes the lint registry-prefix check without an allowlist entry):
12+
1. **Publish the container.** Build and push the image on a machine with Docker +
13+
a CUDA GPU + push access. A turnkey script is provided
14+
([`containers/nis/build_and_push.sh`](../containers/nis/build_and_push.sh)),
15+
pre-filled with the cu128 LibTorch source (`cpp/README.md` /
16+
`cpp/build_main_hummer.sh`):
1517

1618
```bash
17-
docker build -f containers/nis/Dockerfile -t quay.io/nf-core/cellpheno-nis:1.0.0 .
18-
docker push quay.io/nf-core/cellpheno-nis:1.0.0
19-
# then set `container "quay.io/nf-core/cellpheno-nis:1.0.0"` in modules/nf-core/cellpheno/nis/main.nf
19+
docker login quay.io # nf-core push access
20+
bash containers/nis/build_and_push.sh quay.io/nf-core/cellpheno-nis 1.0.0
2021
```
2122

22-
(Building needs a CUDA host; the image is large.)
23+
(The image is large — CUDA base + LibTorch. If you don't have nf-core quay
24+
access yet, push to `ghcr.io/chrisa142857/cellpheno-nis` and update the
25+
`container` line, or ask the nf-core team to host it.)
2326

2427
2. **Add minimal test data** to a branch of
2528
[nf-core/test-datasets](https://github.com/nf-core/test-datasets) (the
26-
`numorph/3dunet` precedent): a tiny lightsheet tile directory and the
27-
TorchScript `.pt` models, then reference them via
28-
`params.modules_testdata_base_path` and add a real, `gpu`-tagged test.
29+
`numorph/3dunet` precedent). The fixtures and their layout are in
30+
[`tests/testdata/cellpheno_nis`](../tests/testdata/cellpheno_nis). The real
31+
TorchScript model weights come from the G-drive folder in `cpp/README.md`:
32+
33+
```bash
34+
pip install gdown
35+
bash containers/nis/fetch_models.sh downloads/resource # G-drive NIS models
36+
```
37+
38+
Put the tiny tile under
39+
`data/imaging/segmentation/cellpheno_nis/tile/` on test-datasets; host the
40+
large `.pt` weights on Zenodo (numorph precedent) and reference them in the
41+
`gpu`-tagged test. Then run it on a GPU runner to add its snapshot entry:
42+
`nf-test test modules/nf-core/cellpheno/nis/... --profile docker,gpu --update-snapshot`.
2943

3044
## Open the PR
3145

0 commit comments

Comments
 (0)