AmSC Genesis (#127)

- New AmSC IRI client with facility, status, account, and compute APIs
- Facility adapters for ALCF, NERSC, and OLCF; choose compute resource by name; facility shown in Genesis status table
- New AmSC transfer client 
- Live status monitoring and shared task-progress reporting for processing and fluorescence pipelines
- New token-management CLIs: `ptychodus-iri-tokens` and `ptychodus-transfer-tokens`
- Per-facility helper scripts under `src/ptychodus/scripts/genesis/{alcf,nersc,olcf}/`
- Update to pty-chi 1.4.0; add Bilinear Hessian reconstructor backend and expose new probe-support methods and magnitude/phase optimization components
- Update PtychoPINN-Torch integration: in-memory prediction, multiple-probe-mode export, sync to upstream API
- Per-reconstructor model autosave / autoload, wired through the Genesis workflow
- Support APS 31-ID LYNX area-detector file format
- `ptychodus-bdp`: load bad-pixel masks from file
- Clean up diffraction-pattern / probe-position association so pairing is driven strictly by diffraction pattern index, with duplicate-index averaging and interpolation for missing positions; out-of-range patterns are dropped rather than extrapolated
- Paganin object builder (single-material phase retrieval from the central diffraction pattern)
- Hermite probe builder
- Estimate diffraction-pattern crop center and probe-size metrics from raw data
- New `ReconstructionAmbiguities` estimator (global complex scale, linear phase ramp, sub-pixel shift) so quality metrics compare reconstructions on equal footing
- FRC (Fourier ring correlation) preprocessing, metrics, and GUI panel
- Visualize reconstruction residuals (per-pattern modeled-vs-measured diffraction)
- Revise affine-transform estimator for probe-position correction (RANSAC, Hodges–Lehmann centering)
- Propagator view: improved layout and on-the-fly probe-size metrics
- XMCD dialog fixes
- Allow and gracefully render NaN values in visualizations
- Group navigation subviews in the left-panel toolbar
- Diffraction loading progress monitor
- Fluorescence enhancer progress monitor (shared `TaskMonitor`) with `enhance_fluorescence_local` workflow API
- Auto-size item-view columns
- Split the single `Dockerfile` into per-accelerator variants: `Dockerfile.cpu`, `Dockerfile.cuda`, `Dockerfile.rocm`, `Dockerfile.xpu`, each with build-arg knobs (CUDA / ROCm / PyTorch versions, XPU base tag)
- Add a `podman` wrapper script (`src/ptychodus/scripts/podman/ptychodus`) that builds, runs, and mounts the container with the right GPU / display / volume flags
- New `ptychodus-system-check` CLI for smoke-checking the installed environment
- Improved documentation and test coverage

Author: stevehenke

.devcontainer/devcontainer.json

@@ -0,0 +1,16 @@
+{
+  "name": "ptychodus (CPU)",
+  "build": {
+    "context": "..",
+    "dockerfile": "../Dockerfile.cpu"
+  },
+  "postCreateCommand": "pip install -e .",
+  "customizations": {
+    "vscode": {
+      "extensions": [
+        "ms-python.python",
+        "charliermarsh.ruff"
+      ]
+    }
+  }
+}

.dockerignore

@@ -0,0 +1,13 @@
+.git
+.github
+dist/
+build/
+docs/build/
+memoized_data/
+*.h5
+*.npy
+*.ini
+NOTES
+TODO
+__pycache__/
+*.pyc

.github/workflows/python-package.yml

@@ -1,33 +1,89 @@
-# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
-# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions
+# Install ptychodus, run the unit tests, and check lint/types across the
+# supported Python versions. See pyproject.toml for the source-of-truth on
+# Python version support and tool configuration.
 
-name: Python package
+name: CI
 
 on:
   push:
-    branches: [ "master" ]
+    branches: [ "main" ]
   pull_request:
-    branches: [ "master" ]
+    branches: [ "main" ]
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: ${{ github.event_name == 'pull_request' }}
 
 jobs:
-  build:
+  smoke:
+    name: Install & CLI smoke (py${{ matrix.python-version }})
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.11", "3.12", "3.13"]
+    steps:
+      - uses: actions/checkout@v4
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+          cache: pip
+      - name: Install ptychodus (minimal)
+        run: pip install .
+      - name: ptychodus --version
+        run: ptychodus --version
 
+  test:
+    name: Unit tests (py${{ matrix.python-version }})
     runs-on: ubuntu-latest
     strategy:
       fail-fast: false
       matrix:
-        python-version: ["3.10", "3.11", "3.12"]
+        python-version: ["3.11", "3.12", "3.13"]
+    steps:
+      - uses: actions/checkout@v4
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+          cache: pip
+      - name: Install ptychodus + pytest
+        run: pip install . pytest
+      - name: Run pytest
+        run: pytest tests/ -v
+
+  lint:
+    name: Ruff (lint + format)
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Set up Python 3.11
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+          cache: pip
+      - name: Install ruff
+        run: pip install ruff
+      - name: ruff check
+        run: ruff check .
+      - name: ruff format --check
+        run: ruff format --check .
 
+  typecheck:
+    name: Mypy
+    runs-on: ubuntu-latest
     steps:
-    - uses: actions/checkout@v4
-    - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v5
-      with:
-        python-version: ${{ matrix.python-version }}
-    - name: Install with minimal dependencies
-      run: |
-        python -m pip install --upgrade pip
-        python -m pip install .
-    - name: Test CLI with headless minimal install
-      run: |
-        ptychodus --version
+      - uses: actions/checkout@v4
+      - name: Set up Python 3.11
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.11"
+          cache: pip
+      - name: Install ptychodus + mypy
+        run: pip install . mypy pyqt5-stubs types-psutil types-pyyaml types-requests
+      - name: mypy
+        run: mypy src/ptychodus

.readthedocs.yaml

@@ -14,6 +14,7 @@ sphinx:
 # See https://docs.readthedocs.io/en/stable/guides/reproducible-builds.html
 python:
    install:
-   - requirements: docs/requirements.txt
    - method: pip
      path: .
+     extra_requirements:
+       - docs

CLAUDE.md

@@ -0,0 +1,114 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project
+
+Ptychodus is a ptychography data-analysis application that ingests instrument data, prepares it for processing, and dispatches it through several reconstruction libraries (PtyChi, PtychoPINN, PtychoPINN-Torch). It runs interactively as a PyQt5 GUI, headless via batch CLI, and as a streaming processor inside beamline pipelines (pvapy area-detector). Python ≥3.11.
+
+## Common Commands
+
+Project uses `uv` (preferred) and a developer install with extras.
+
+```sh
+# Dev install (preferred). Other available extras: docs, ptychopinn
+uv sync --extra globus --extra gui --extra ptychi
+
+# Launch GUI
+uv run ptychodus
+
+# Headless batch
+uv run ptychodus -b reconstruct -i <input_dir> -o <output_dir>
+uv run ptychodus -b train      -i <input_dir> -o <output_dir>
+# Batch mode reads <input_dir>/settings.ini, <input_dir>/diffraction.h5, <input_dir>/product-in.h5
+# (see StandardFileLayout in src/ptychodus/api/io.py)
+
+# Beamline data-prep CLI
+uv run ptychodus-bdp --product-name <name> --diffraction-input <h5> \
+                     --probe-position-input <csv> --output-directory <dir> \
+                     --settings <ini>
+
+# Other entry points (see [project.scripts] in pyproject.toml)
+uv run convert-to-ptychodus
+uv run ptychodus-system-check
+uv run ptychodus-iri-tokens          # Genesis/IRI auth
+uv run ptychodus-transfer-tokens     # AmSC data-transfer auth
+uv run ptychodus-ptychopinn-tf-test  # PtychoPINN TensorFlow smoke check
+```
+
+Tests, lint, types:
+
+```sh
+uv run pytest                                  # full suite (tests/)
+uv run pytest tests/test_io.py                 # one file
+uv run pytest tests/test_io.py::test_name      # one test
+uv run ruff check .                            # lint  (rules: F, N, NPY)
+uv run ruff format .                           # format (single quotes, line-length 100)
+uv run mypy src/ptychodus                      # type check (py 3.11)
+```
+
+CI (`.github/workflows/python-package.yml`) runs four jobs on push/PR to `main`: a `pip install` + `ptychodus --version` smoke test (Py 3.11/3.12/3.13), `pytest tests/`, `ruff check` + `ruff format --check`, and `mypy src/ptychodus`. Run the local equivalents before pushing — CI will block the PR otherwise.
+
+Container & docs:
+
+```sh
+podman build -f Dockerfile.cuda -t ptychodus:cuda13.0 .   # also: Dockerfile.cpu / .rocm / .xpu
+docker build  -f Dockerfile.cuda -t ptychodus:cuda13.0 .
+make -C docs html        # Sphinx docs into docs/build/
+```
+
+## High-Level Architecture
+
+### Three-layer separation: api / model / view+controller
+
+- **`src/ptychodus/api/`** — pure-Python domain layer with **no** GUI or model dependencies. Defines core data structures (`Product`, `ProbeSequence`, `Object`, `ProbePositionSequence`, `DiffractionDataset`, `AssembledDiffractionData`), abstract interfaces (`DiffractionFileReader/Writer`, `ProductFileReader/Writer`, `Reconstructor`, `TrainableReconstructor`, `WorkflowAPI`), and infrastructure (`Observable`/`Observer`, `Parameter`/`ParameterGroup`, `SettingsRegistry`, `PluginRegistry`/`PluginChooser`). All other layers depend on this; this layer depends on nothing else in ptychodus.
+- **`src/ptychodus/model/`** — application logic. Each subpackage (`diffraction/`, `product/`, `processing/`, `reconstructor/`, `analysis/`, `fluorescence/`, `globus/`, `genesis/`, `automation/`, `agent/`, `visualization/`, `ptychi/`, `ptychopinn/`, `ptychopinn_torch/`) exposes a `*Core` class that owns its settings, repositories, and APIs. `model/core.py::ModelCore` is the composition root: it constructs every `*Core` in dependency order and wires them together. Used both by the GUI and by `__main__.py` batch mode.
+- **`src/ptychodus/view/`** (PyQt5 widgets, no logic) and **`src/ptychodus/controller/`** (mediates between widgets and model). Mirror the model package layout. `view/core.py::ViewCore` and `controller/core.py::ControllerCore` are the composition roots; the navigation toolbar order in `ViewCore` is the source of truth for the left/right stacked-panel indexes.
+
+The GUI is optional: `__main__.py` falls back to headless mode if PyQt5 is missing. `ptychodus_stream_processor.py` (the `PtychodusAdImageProcessor`) is the third entry mode and is only imported when `pvapy` is available.
+
+### Index-based pattern/position association
+
+Diffraction patterns and probe positions are paired by **integer scan index**, never by array order. This is what makes streaming and mismatched-rate ingest robust: patterns and positions can arrive from different files or different PV channels, with dropped or extra samples on either side, and the matcher still pairs them correctly.
+
+- Producers: `DiffractionArray.get_indexes()` on the pattern side (`api/diffraction.py`); `ProbePosition.index` on the position side (`api/probe_positions.py`).
+- Matcher: `AssembledDiffractionData.prepare_reconstruct_input` in `api/reconstructor.py` treats pattern indexes as authoritative — duplicate position indexes are averaged into anchors, pattern indexes inside the anchor range with no exact position are linearly interpolated, and pattern indexes outside the anchor range are dropped (no extrapolation). The `Product` is rebuilt from the resulting per-pattern positions.
+- Round-trip: HDF5 and NPZ product writers persist position indexes via `ProductFileKeys.PROBE_POSITION_INDEXES`.
+
+### Plugin system
+
+File-format support is dynamic. `api/plugins.py::PluginRegistry.load_plugins()` walks `ptychodus.plugins.*` with `pkgutil.iter_modules` and calls each module's `register_plugins(registry)` function. Module-load failures are logged and skipped — this is how optional-dependency plugins (e.g., LCLS, NSLS-II) silently disable themselves.
+
+To add a new file format: create a module under `src/ptychodus/plugins/`, implement the relevant `*FileReader`/`*FileWriter` from `ptychodus.api`, and define `register_plugins(registry)` calling the appropriate `registry.<category>.register_plugin(...)`. For product readers, prefer `register_product_file_reader_with_adapters` so the reader is also reachable as a probe/probe-position/object reader.
+
+### Settings & observers
+
+Settings flow through `api/parametric.py::Parameter[T]` and `ParameterGroup`. Each `*Core` calls `settings_registry.create_group(name)` and creates typed parameters on it. `SettingsRegistry` serializes the full tree as INI. Components react to settings changes through the `Observer`/`Observable` pattern in `api/observer.py`; `PluginChooser.synchronize_with_parameter` is the typical bridge for "settings string → currently selected plugin." When adding cross-component reactivity, hook observers rather than poll.
+
+### Reconstructor libraries
+
+Each reconstructor backend (`model/ptychi/`, `model/ptychopinn/`, `model/ptychopinn_torch/`) exposes a `*ReconstructorLibrary` class. They are constructed in `ModelCore` and passed as a list to `ProcessingCore`. Backends that aren't installed should fail cleanly at import inside their own `__init__.py` — keep the surface a stable `*ReconstructorLibrary` regardless. Reconstructors implement `Reconstructor`/`TrainableReconstructor` from `api/reconstructor.py` and yield `ReconstructOutput` per iteration so the GUI can stream progress.
+
+### Standard HDF5 layout
+
+`api/io.py::StandardFileLayout` is the canonical contract for batch mode and remote workflows: `diffraction.h5`, `product-in.h5`, `product-out.h5`, `fluorescence-in.h5`, `fluorescence-out.h5`, `settings.ini`. `load_diffraction_data` / `save_diffraction_data` and `load_product` / `save_product` round-trip these files; their key names live in `DiffractionFileKeys` / `ProductFileKeys` enums — change those carefully, they are an external interface.
+
+### Remote compute
+
+Two providers, both gated by optional dependencies and constructed by `ModelCore` even when disabled:
+
+- `model/globus/` — Globus Compute submission, used by the original APS workflow.
+- `model/genesis/` — IRI/AmSC HPC submission with facility adapters in `model/genesis/facility_adapters.py` and per-facility scripts under `src/ptychodus/scripts/genesis/{alcf,nersc,olcf}/`.
+
+`WorkflowAPI` (see `api/workflow.py` and `model/workflow.py::ConcreteWorkflowAPI`) is the unified façade that GUI, batch mode, and remote workflows all drive.
+
+## Conventions
+
+- Ruff is configured for **single-quoted** strings, 100-char lines, py311 target. The selected lint rules (F, N, NPY) flag pyflakes errors, PEP-8 naming, and NumPy-specific issues. NumPy/Qt-style names (e.g., `probe_energy_eV`, `set_value_from_string`) are accepted via `# noqa: N802/N806/N815` — preserve the unit suffixes on physical quantities; reviewers expect them.
+- Type hints are mandatory; `pyproject.toml` lists modules whose missing stubs are intentionally ignored. Keep new code typed and avoid widening that ignore list.
+- `model/core.py::ModelCore.is_developer_mode_enabled` is `True` whenever the effective log level ≤ DEBUG (`--log-level 10`). Some controllers gate features (Agent panel, probe-position analysis) behind it.
+
+## Repository Notes
+
+- The git CI workflow targets `main`; local development branches such as `amsc` are active — confirm the intended target before opening PRs.
+- Sample `.h5`/`.npy` data and the `dist/` build output in the working tree are typically **untracked** local artifacts — do not stage them in commits unless explicitly asked.

Dockerfile

@@ -0,0 +1,21 @@
+FROM python:3.13-slim-trixie
+
+WORKDIR /app
+COPY . /src
+
+# Pre-install CPU-only torch from the PyTorch CPU index so the later
+# pip install of ptychi sees torch as already satisfied and does NOT
+# pull a CUDA-flavored torch wheel from PyPI.
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends git libqt5gui5 && \
+    rm -rf /var/lib/apt/lists/* && \
+    python3 -m pip install --root-user-action=ignore --no-cache-dir --upgrade pip && \
+    python3 -m pip install --root-user-action=ignore --no-cache-dir \
+        --index-url https://download.pytorch.org/whl/cpu \
+        torch && \
+    python3 -m pip install --root-user-action=ignore --no-cache-dir \
+        /src[globus,gui] \
+        git+https://github.com/AdvancedPhotonSource/pty-chi.git && \
+    rm -rf /src
+
+CMD ["python3", "-m", "ptychodus"]

Dockerfile.cpu

@@ -0,0 +1,18 @@
+ARG CUDA_VERSION=13.0
+ARG PYTORCH_VERSION=2.12.0
+ARG CUDNN_VERSION=9
+FROM pytorch/pytorch:${PYTORCH_VERSION}-cuda${CUDA_VERSION}-cudnn${CUDNN_VERSION}-devel
+
+WORKDIR /app
+COPY . /src
+
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends git libqt5gui5 && \
+    rm -rf /var/lib/apt/lists/* && \
+    python3 -m pip install --root-user-action=ignore --no-cache-dir --upgrade pip && \
+    python3 -m pip install --root-user-action=ignore --no-cache-dir \
+        /src[globus,gui] \
+        git+https://github.com/AdvancedPhotonSource/pty-chi.git && \
+    rm -rf /src
+
+CMD ["python3", "-m", "ptychodus"]

Dockerfile.cuda

@@ -0,0 +1,21 @@
+ARG ROCM_VERSION=7.2.4
+ARG UBUNTU_VERSION=24.04
+ARG PYTHON_VERSION=3.12
+ARG PYTORCH_VERSION=2.10.0
+FROM rocm/pytorch:rocm${ROCM_VERSION}_ubuntu${UBUNTU_VERSION}_py${PYTHON_VERSION}_pytorch_release_${PYTORCH_VERSION}
+
+WORKDIR /app
+COPY . /src
+
+# The base image already includes a ROCm-built torch; pip sees the torch
+# requirement satisfied and leaves it alone.
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends git libqt5gui5 && \
+    rm -rf /var/lib/apt/lists/* && \
+    python3 -m pip install --root-user-action=ignore --no-cache-dir --upgrade pip && \
+    python3 -m pip install --root-user-action=ignore --no-cache-dir \
+        /src[globus,gui] \
+        git+https://github.com/AdvancedPhotonSource/pty-chi.git && \
+    rm -rf /src
+
+CMD ["python3", "-m", "ptychodus"]

Dockerfile.rocm

@@ -0,0 +1,19 @@
+ARG BASE_TAG=latest
+FROM intel/intel-optimized-pytorch:${BASE_TAG}
+
+WORKDIR /app
+COPY . /src
+
+# The base image already includes Intel-optimized torch (with IPEX / XPU
+# support for ALCF Aurora); pip sees the torch requirement satisfied and
+# leaves it alone.
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends git libqt5gui5 && \
+    rm -rf /var/lib/apt/lists/* && \
+    python3 -m pip install --root-user-action=ignore --no-cache-dir --upgrade pip && \
+    python3 -m pip install --root-user-action=ignore --no-cache-dir \
+        /src[globus,gui] \
+        git+https://github.com/AdvancedPhotonSource/pty-chi.git && \
+    rm -rf /src
+
+CMD ["python3", "-m", "ptychodus"]