Skip to content

Commit 0e39283

Browse files
claudeclaude
committed
Add local-GPU-server runbook + setup script to finish the nf-core submission
So the work can be completed where Docker/GPU/network/credentials exist: - scripts/setup_local_gpu.sh: install nf-core/nextflow/nf-test/gdown, check Docker + NVIDIA runtime, prompt for quay.io/ghcr.io/GitHub auth. - docs/local_gpu_run.md: step-by-step runbook (build/push container -> verify lint goes green -> optional test data -> fork + PR), including the ghcr.io fallback registry. Designed to be driven by Claude Code running on the server. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TRbz5C1u1pxcvB5DGYVZMo
1 parent f6b985d commit 0e39283

2 files changed

Lines changed: 139 additions & 0 deletions

File tree

‎docs/local_gpu_run.md‎

Lines changed: 98 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,98 @@
1+
# Finishing the CellPheno NIS submission on a local GPU server
2+
3+
This is the runbook for the three steps the automated cloud session could not do
4+
(no Docker daemon, blocked registry hosts, GitHub scope). Run it on your GPU
5+
server, where Docker + GPU + network + your credentials are available. You can
6+
either run the commands yourself, or launch **Claude Code** in this repo and say:
7+
*"complete the CellPheno NIS nf-core submission per docs/local_gpu_run.md"* — the
8+
agent will execute these steps and verify each one.
9+
10+
## 0. One-time setup
11+
12+
```bash
13+
git clone https://github.com/Chrisa142857/Lightsheet_microscopy_image_3D_nuclei_instance_segmentation
14+
cd Lightsheet_microscopy_image_3D_nuclei_instance_segmentation
15+
git checkout claude/nis-nfcore-conversion-afd7qu
16+
17+
bash scripts/setup_local_gpu.sh # installs nf-core, nextflow, nf-test, gdown; checks Docker+GPU
18+
docker login ghcr.io # your GitHub container registry
19+
docker login quay.io # only if you have nf-core org push access
20+
gh auth login # for the fork + PR
21+
```
22+
23+
## 1. Build + push the GPU container
24+
25+
```bash
26+
# Preferred: nf-core org namespace (matches numorph/3dunet). Needs nf-core quay access.
27+
bash containers/nis/build_and_push.sh quay.io/nf-core/cellpheno-nis 1.0.0
28+
29+
# Fallback you fully control (works without nf-core membership):
30+
bash containers/nis/build_and_push.sh ghcr.io/chrisa142857/cellpheno-nis 1.0.0
31+
```
32+
33+
If you used the **ghcr fallback**, point the module at it so lint can reach the
34+
image, and make the package public (`ghcr.io` → package settings → Public):
35+
36+
```bash
37+
sed -i 's#quay.io/nf-core/cellpheno-nis#ghcr.io/chrisa142857/cellpheno-nis#' \
38+
modules/nf-core/cellpheno/nis/main.nf
39+
```
40+
41+
## 2. Verify the module is fully green
42+
43+
Lint must run inside a clone of nf-core/modules (it compares against that repo).
44+
45+
```bash
46+
git clone --depth 1 https://github.com/nf-core/modules.git /tmp/nfcore-modules
47+
cp -r modules/nf-core/cellpheno /tmp/nfcore-modules/modules/nf-core/
48+
cd /tmp/nfcore-modules
49+
nf-core modules lint cellpheno/nis # container check now PASSES (image is reachable)
50+
cd -
51+
```
52+
53+
Expected: the previous `container_links` failure is gone (image now reachable).
54+
The remaining `process_gpu` / container-version messages are non-fatal warnings
55+
that every GPU module carries.
56+
57+
## 3. (Optional, for the real GPU test) prepare test data
58+
59+
The committed test has a `-stub` test (already snapshotted) plus a `gpu`-tagged
60+
real test. The real test needs data; the stub is enough to open the PR. To enable
61+
the real test later:
62+
63+
```bash
64+
bash containers/nis/fetch_models.sh downloads/resource # G-drive NIS .pt models (cpp/README.md)
65+
# Tiny tile fixture is in tests/testdata/cellpheno_nis/ — replace with a real small tile.
66+
# Then: PR the tile to nf-core/test-datasets, host .pt weights on Zenodo, and run
67+
nf-test test modules/nf-core/cellpheno/nis/tests/main.nf.test --profile docker,gpu --update-snapshot
68+
```
69+
70+
## 4. Fork nf-core/modules and open the PR
71+
72+
```bash
73+
gh repo fork nf-core/modules --clone --remote
74+
cd modules
75+
git checkout -b cellpheno-nis
76+
cp -r ../Lightsheet_microscopy_image_3D_nuclei_instance_segmentation/modules/nf-core/cellpheno \
77+
modules/nf-core/
78+
nf-core modules lint cellpheno/nis
79+
pre-commit run --all-files
80+
git add modules/nf-core/cellpheno
81+
git commit -m "new module: cellpheno/nis"
82+
git push -u origin cellpheno-nis
83+
gh pr create --repo nf-core/modules --base master \
84+
--title "new module: cellpheno/nis" \
85+
--body-file ../Lightsheet_microscopy_image_3D_nuclei_instance_segmentation/PR_BODY.md
86+
```
87+
88+
The PR body text is in [`docs/nfcore_submission.md`](nfcore_submission.md) — copy
89+
it into `PR_BODY.md` first (or paste it into the PR on GitHub).
90+
91+
## Notes
92+
93+
- **Registry choice:** `quay.io/nf-core/cellpheno-nis` is the ideal home but needs
94+
nf-core org access; maintainers commonly mirror a contributor image there during
95+
review. Using `ghcr.io/chrisa142857/cellpheno-nis` (public) is fine to get a green
96+
lint and open the PR — note it in the PR and the team can re-host.
97+
- **Conda CI:** the module intentionally errors under `-profile conda/mamba`
98+
(GPU binary, no conda) — this is expected and matches `parabricks/*`.

‎scripts/setup_local_gpu.sh‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
#!/usr/bin/env bash
2+
#
3+
# Set up a local GPU server to finish the CellPheno NIS nf-core contribution
4+
# (build/push the container, lint, fork/PR). Installs the toolchain and checks
5+
# prerequisites. Credentials (quay.io / ghcr.io / GitHub) are configured
6+
# interactively at the end — they are never stored by this script.
7+
#
8+
# Usage: bash scripts/setup_local_gpu.sh
9+
set -euo pipefail
10+
11+
echo "==> Checking Docker + NVIDIA GPU runtime"
12+
docker info >/dev/null 2>&1 || { echo "ERROR: Docker daemon not usable"; exit 1; }
13+
if ! docker run --rm --gpus all nvidia/cuda:12.8.0-base-ubuntu22.04 nvidia-smi >/dev/null 2>&1; then
14+
echo "WARN: 'docker run --gpus all' failed — install the NVIDIA Container Toolkit:"
15+
echo " https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html"
16+
fi
17+
18+
echo "==> Installing nf-core tools, gdown (pip)"
19+
python3 -m pip install --quiet --upgrade nf-core gdown
20+
21+
echo "==> Installing Nextflow (if missing)"
22+
command -v nextflow >/dev/null 2>&1 || { curl -s https://get.nextflow.io | bash && sudo mv nextflow /usr/local/bin/; }
23+
24+
echo "==> Installing nf-test (if missing)"
25+
command -v nf-test >/dev/null 2>&1 || { curl -fsSL https://code.askimed.com/install/nf-test | bash && sudo mv nf-test /usr/local/bin/ 2>/dev/null || true; }
26+
27+
echo "==> Checking GitHub CLI"
28+
command -v gh >/dev/null 2>&1 || echo "WARN: install gh (https://cli.github.com) for the fork/PR step"
29+
30+
cat <<'NEXT'
31+
32+
==> Toolchain ready. Now authenticate (these prompt for / read your stored creds):
33+
34+
docker login quay.io # if you have nf-core org push access
35+
docker login ghcr.io # your GitHub container registry (always works)
36+
gh auth login # for the fork + PR
37+
38+
Then run the contribution steps (see docs/local_gpu_run.md), or launch Claude
39+
Code in this repo and tell it: "complete the CellPheno NIS nf-core submission
40+
per docs/local_gpu_run.md".
41+
NEXT

0 commit comments

Comments
 (0)