Skip to content

Commit 446f45f

Browse files
feat: bump to v0.2.0 with refreshed docs and AI instructions
Summary of changes since v0.1.0 (now reflected in the docs / AI instruction files): - Pluggable compute backends (mdtraj / numba / cupy / torch / jax) for compute_distances, compute_rmsd_matrix, and compute_dccm. DCCM defaults to "numpy" (BLAS GEMM); distance and RMSD default to "mdtraj" for PBC correctness. - sklearn-style callable clustering classes: Gromos, Hierarchical, DBSCAN, HDBSCAN, KMeans, MiniBatchKMeans, RegularSpace. - compute_delta_rmsf with MSF-space averaging and SEM propagation. - compute_native_contacts (vectorised) and refreshed plot helpers (plot_delta_rmsf, plot_feature_clustering, plot_native_contacts). - Stable float dtype system (_dtype.py, DtypeArg, float32 default). Docs / AI instruction refresh: - docs/index.md: new "Highlights" section, backends + clustering. - docs/guide/analysis.md: correct compute_delta_rmsf example, new DCCM backend table and dispatch notes. - README.md: backends row covers DCCM, clustering row lists all seven classes, new "Cluster conformations" quick-start snippet. - CLAUDE.md / AGENTS.md: DCCM default exception, refreshed _backends/ tree, clustering class table, np.floating Protocol return type, GPU cache cleanup expanded. - .cursor/rules/*.mdc: project + python-analysis + shell-scripts brought in line with the current dtype system and backend conventions; shell shebang corrected to env bash. Verified with: pre-commit (ruff/mypy/mdformat/shellcheck) clean, pytest -m "not gpu and not slow and not benchmark" (583 passed), mkdocs build --clean --strict (no warnings).
1 parent 3bc6195 commit 446f45f

9 files changed

Lines changed: 218 additions & 53 deletions

File tree

.cursor/rules/project.mdc

Lines changed: 15 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -9,10 +9,15 @@ alwaysApply: true
99

1010
```
1111
src/mdpp/
12+
├── _types.py # shared type aliases (StrPath, PathLike, DtypeArg)
13+
├── _dtype.py # float dtype config (get/set_default_dtype, resolve_dtype)
1214
├── constants.py # physical constants (GAS_CONSTANT_KJ_MOL_K, DEFAULT_TEMPERATURE_K)
1315
├── core/ # trajectory I/O (load_trajectory, align), file parsers (read_xvg, read_edr)
1416
├── analysis/ # compute_* functions returning frozen dataclass results
15-
├── plots/ # plot_* functions returning matplotlib Axes
17+
│ └── _backends/ # private subpackage: mdtraj/numba/cupy/torch/jax kernels for
18+
│ # distances, RMSD matrix, DCCM; BackendRegistry[F] + Literals
19+
├── chem/ # RDKit-based cheminformatics (descriptors, fingerprints, similarity, filters)
20+
├── plots/ # plot_* / draw_* / view_* functions returning matplotlib Axes / 3D widgets
1621
└── prep/ # system preparation (protein fixing, ligand topology, trajectory manipulation)
1722
```
1823

@@ -40,15 +45,19 @@ Every `compute_*` function follows this contract:
4045

4146
## Compute Backend Default Rule
4247

43-
Every public analysis function that accepts a `backend=` argument MUST default to `"mdtraj"`. Other backends (`numba`, `torch`, `jax`, `cupy`) exist for performance (50-200x CPU speedup or GPU acceleration) and MUST be opted into explicitly by the caller.
48+
Every public analysis function that accepts a `backend=` argument MUST default to `"mdtraj"` **when mdtraj provides a native kernel for that computation**. Other backends (`numba`, `torch`, `jax`, `cupy`) exist for performance (50-200x CPU speedup or GPU acceleration) and MUST be opted into explicitly by the caller. The only current exception is `compute_dccm`, which defaults to `"numpy"` because mdtraj has no covariance kernel -- `numpy`'s BLAS GEMM is multi-threaded and ships with the base install.
4449

4550
Why:
4651

4752
- Only `mdtraj` supports periodic boundary conditions -- defaulting to anything else silently drops PBC.
4853
- Consistent defaults across all analysis functions avoid surprising behavior when users switch between them.
4954
- The GPU backends (`[gpu]` extra) must never be required for the common path.
5055

51-
**Never change a public function's default backend away from `"mdtraj"`.** Add performance notes in the docstring pointing users to `backend="numba"` or a GPU backend when they need speed.
56+
**Never silently change a public function's default backend away from its current value** (`"mdtraj"`, or `"numpy"` for DCCM). Add performance notes in the docstring pointing users to `backend="numba"` or a GPU backend when they need speed.
57+
58+
## Float Dtype Rule
59+
60+
mdpp uses **float32 by default**, matching mdtraj's coordinate precision. Never force float64 for "numerical stability". Every `compute_*` function takes `dtype: DtypeArg = None` as the final keyword argument, calls `resolved = resolve_dtype(dtype)` at the top, and casts outputs with `astype(resolved, copy=False)` (zero-copy when the backend's native dtype already matches). Import `DtypeArg` from `mdpp._types`; never inline the union type.
5261

5362
## Backend Registry Typing Rule
5463

@@ -62,11 +71,13 @@ class RMSDMatrixBackendFn(Protocol):
6271
self,
6372
traj: md.Trajectory,
6473
atom_indices: NDArray[np.int_],
65-
) -> NDArray[np.float64]: ...
74+
) -> NDArray[np.floating]: ...
6675

6776
rmsd_matrix_backends: BackendRegistry[RMSDMatrixBackendFn] = BackendRegistry(default="mdtraj")
6877
```
6978

79+
Protocols return `NDArray[np.floating]` (not `NDArray[np.float64]`) so each backend can return its native dtype -- float32 for mdtraj / torch / jax / cupy, float64 for numba. The public `compute_*` wrapper casts with `astype(resolved, copy=False)` so when the backend's native dtype already matches the resolved dtype, no second buffer is allocated. Never add an unconditional `.astype(np.float64)` at a backend boundary -- on an N^2 matrix at N=120k that is 115 GB of pointless copying.
80+
7081
**Never** declare a bare `BackendRegistry` without a type parameter -- `registry.get(backend)` would return an unbound `F`, and callers like `compute_fn = rmsd_matrix_backends.get(backend)` would lose type information. The Protocol lives next to the backends it describes (not in the shared `_registry.py`) so the registry module stays decoupled.
7182

7283
**Uniform signature rule**: every backend registered in the same registry MUST accept the exact Protocol call signature. If one backend needs an extra keyword argument (e.g. `periodic` on mdtraj), every other backend MUST also accept that keyword, silently ignoring it if unused (`# noqa: ARG001` + docstring note). This preserves type inference at call sites and keeps dispatchers branch-free.

.cursor/rules/python-analysis.mdc

Lines changed: 33 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
description: Patterns for analysis modules — dataclass results, compute functions
2+
description: Patterns for analysis modules — dataclass results, compute functions, dtype handling
33
globs: src/mdpp/analysis/*.py
44
alwaysApply: false
55
---
@@ -12,40 +12,67 @@ alwaysApply: false
1212
@dataclass(frozen=True, slots=True)
1313
class ExampleResult:
1414
"""One-line summary."""
15-
time_ps: NDArray[np.float64]
16-
values_nm: NDArray[np.float64]
15+
time_ps: NDArray[np.floating]
16+
values_nm: NDArray[np.floating]
1717
atom_indices: NDArray[np.int_]
1818

1919
@property
20-
def time_ns(self) -> NDArray[np.float64]:
20+
def time_ns(self) -> NDArray[np.floating]:
21+
"""Return frame times in nanoseconds."""
2122
return self.time_ps / 1000.0
2223

2324
@property
24-
def values_angstrom(self) -> NDArray[np.float64]:
25+
def values_angstrom(self) -> NDArray[np.floating]:
26+
"""Return values in Angstrom."""
2527
return self.values_nm * 10.0
2628
```
2729

30+
Use `NDArray[np.floating]` (not `np.float64`) in result fields so the dataclass reflects the actual stored dtype, which defaults to float32.
31+
2832
## Compute Functions
2933

3034
```python
35+
from mdpp._dtype import resolve_dtype
36+
from mdpp._types import DtypeArg
37+
from mdpp.core.trajectory import select_atom_indices, trajectory_time_ps
38+
3139
def compute_example(
3240
traj: md.Trajectory,
3341
*,
3442
atom_selection: str = "backbone",
3543
timestep_ps: float | None = None,
44+
dtype: DtypeArg = None,
3645
) -> ExampleResult:
3746
"""Compute X from a trajectory.
3847

3948
Args:
4049
traj: Input trajectory.
4150
atom_selection: MDTraj selection string.
4251
timestep_ps: Optional timestep override in ps.
52+
dtype: Output float dtype. If ``None``, uses the package default
53+
(see :func:`mdpp.set_default_dtype`).
4354

4455
Returns:
4556
ExampleResult with per-frame values.
4657
"""
58+
resolved = resolve_dtype(dtype)
59+
atom_indices = select_atom_indices(traj.topology, atom_selection)
60+
# ... compute values_nm here ...
61+
return ExampleResult(
62+
time_ps=trajectory_time_ps(traj, timestep_ps=timestep_ps, dtype=resolved),
63+
values_nm=np.asarray(values_nm, dtype=resolved),
64+
atom_indices=atom_indices,
65+
)
4766
```
4867

68+
- First positional arg is `traj: md.Trajectory` (or a feature matrix).
69+
- All remaining args are keyword-only.
70+
- `dtype: DtypeArg = None` is the **last** keyword argument.
71+
- Call `resolve_dtype(dtype)` once at the top, then thread `resolved` through every cast.
4972
- Import trajectory helpers from `mdpp.core.trajectory`, never from `mdpp.analysis.trajectory`.
5073
- Validate inputs early (raise `ValueError` with descriptive messages).
51-
- Cast outputs to explicit dtypes: `np.asarray(..., dtype=np.float64)`.
74+
- Cast with `np.asarray(value, dtype=resolved)` or `.astype(resolved, copy=False)` -- both are no-copy when the source dtype already matches.
75+
76+
## Multi-Backend Compute Functions
77+
78+
If the function dispatches through a `BackendRegistry`, add `backend: <Kind>Backend = "mdtraj"` (or `"numpy"` if mdtraj has no kernel) and resolve via `compute_fn = <kind>_backends.get(backend)`. Every backend must accept the same call signature; extra kwargs on one backend (e.g. `periodic` on mdtraj) must be silently accepted by all others.

.cursor/rules/shell-scripts.mdc

Lines changed: 26 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -6,14 +6,32 @@ alwaysApply: false
66

77
# Shell Script Conventions
88

9-
- All scripts start with `#!/bin/bash` and `set -euo pipefail`.
9+
- Shebang: **always `#!/usr/bin/env bash`** (never `#!/bin/bash`).
10+
- Executable bit: every `.sh` and `.sbatch` file must be `chmod +x`.
11+
- First lines after the shebang: `set -euo pipefail`.
1012
- Indent with 4 spaces (enforced by shfmt via pre-commit).
1113
- Pass shellcheck at error severity.
14+
- ASCII only -- no Unicode arrows, ligatures, emoji, or non-ASCII punctuation.
1215
- Organize under `scripts/<engine>/<category>/`:
13-
- `scripts/gromacs/mdrun/` — production run scripts and MDP files
14-
- `scripts/gromacs/analysis/` — gmx analysis commands
15-
- `scripts/amber/` — AMBER/AmberTools workflows
16-
- `scripts/openfe/` — OpenFE RBFE workflows (quickrun.sh, quickrun.sbatch, runtime/check_status.sh)
17-
- SLURM batch scripts (`.sbatch`) live alongside `.sh` scripts in the same category directory.
18-
- OpenFE scripts require >= 1.10.0 for `--resume` checkpoint support.
19-
- `quickrun.sbatch` starts CUDA MPS for Sherlock's `Exclusive_Process` GPU mode — do not remove the MPS setup block.
16+
- `scripts/gromacs/mdrun/` -- production run scripts
17+
- `scripts/gromacs/mdps/<ff>/` -- force-field-specific MDP templates
18+
- `scripts/gromacs/analysis/` -- gmx analysis commands
19+
- `scripts/gromacs/compilation/` -- GROMACS build helpers
20+
- `scripts/gromacs/runtime/` -- status / restart / extend / export
21+
- `scripts/openfe/quickrun/` -- RBFE SLURM submission
22+
- `scripts/openfe/runtime/` -- status + monitor scripts
23+
- SLURM batch scripts (`.sbatch`) live alongside their `.sh` counterparts in the same category directory.
24+
- OpenFE scripts require **OpenFE >= 1.10.0** for `--resume` checkpoint support.
25+
- `quickrun.sbatch` starts CUDA MPS for Sherlock's `Exclusive_Process` GPU mode -- do not remove the MPS setup block.
26+
27+
## Argument Parsing
28+
29+
Scripts that accept flags/options must:
30+
31+
- Use manual `while [[ $# -gt 0 ]]; do case "$1" in ...` loops (not `getopts`) to support both short and long flags.
32+
- Define a `usage()` function that documents every argument.
33+
- Always support `-h` / `--help`.
34+
- Provide short and long forms for every flag (e.g. `-j` / `--jobs`, `-n` / `--dry-run`).
35+
- Validate required arguments and print clear error messages on invalid input.
36+
37+
Scripts that only accept simple positional arguments (e.g. a single `$1`) do not need this treatment.

AGENTS.md

Lines changed: 30 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ Source is under `src/mdpp/` using the src-layout convention:
7070
| `core/` | Trajectory I/O, file parsers | `load_trajectory`, `load_trajectories`, `read_xvg`, `read_edr` |
7171
| `constants.py` | Physical constants | `GAS_CONSTANT_KJ_MOL_K`, `DEFAULT_TEMPERATURE_K` |
7272
| `analysis/` | Compute functions | `compute_*(traj, *, ...) -> FrozenDataclass` |
73-
| `analysis/_backends/` | Private backend subpackage | `BackendRegistry[F]`, `require_torch/jax/cupy`, `DistanceBackend`/`RMSDBackend` Literals |
73+
| `analysis/_backends/` | Private backend subpackage | `BackendRegistry[F]`, `require_torch/jax/cupy`, `DistanceBackend`/`RMSDBackend`/`DCCMBackend` Literals, `clean_torch_cache`/`clean_cupy_cache` decorators |
7474
| `chem/` | Small-molecule cheminformatics | `MolSupplier`, `calc_descs`, `gen_fp`, `calc_sim`, `is_pains` |
7575
| `plots/` | Visualization (2D, 3D, molecules) | `plot_*(result, *, ax=None) -> Axes`, `draw_mol`, `view_mol_3d` |
7676
| `prep/` | System preparation | `fix_pdb`, `strip_solvent`, `run_propka`, ligand tools |
@@ -119,6 +119,29 @@ Tests live in `tests/analysis/`, `tests/plots/`, and `tests/chem/`, mirroring th
119119
- Shell scripts (not packaged): `scripts/<engine>/<category>/<script>.sh`
120120
- SLURM scripts: `scripts/<engine>/<category>/<script>.sbatch`
121121

122+
## Clustering API
123+
124+
`mdpp.analysis.clustering` exposes seven sklearn-style callable classes:
125+
126+
| Class | Input | Backend / notes |
127+
|---|---|---|
128+
| `Gromos` | RMSD matrix | Numba JIT (greedy largest-first, Daura 1999) |
129+
| `Hierarchical` | RMSD matrix | scipy linkage + fcluster |
130+
| `DBSCAN` | RMSD matrix | Numba JIT (default) or sklearn `metric="precomputed"` |
131+
| `HDBSCAN` | RMSD matrix | sklearn `metric="precomputed"` |
132+
| `KMeans` | Feature matrix | scikit-learn |
133+
| `MiniBatchKMeans` | Feature matrix | scikit-learn |
134+
| `RegularSpace` | Feature matrix | deeptime |
135+
136+
Each class is `@dataclass(frozen=True, slots=True)` with parameters at construction and a `__call__(data) -> ClusteringResult | FeatureClusteringResult` invocation.
137+
138+
```python
139+
result = Gromos(cutoff_nm=0.15)(rmsd_matrix.rmsd_matrix_nm)
140+
result = KMeans(n_clusters=10)(pca.projections)
141+
```
142+
143+
Do **not** add the old function-form wrappers (`compute_gromos_clusters`, etc.) -- they were removed and there is no backward-compat shim.
144+
122145
## Adding a New Analysis
123146

124147
1. Create/extend a file in `src/mdpp/analysis/`.
@@ -130,15 +153,15 @@ Tests live in `tests/analysis/`, `tests/plots/`, and `tests/chem/`, mirroring th
130153

131154
## Compute Backend Conventions
132155

133-
**Default backend rule**: every public compute function that accepts a `backend=` argument MUST default to `"mdtraj"`. Other backends (`numba`, `torch`, `jax`, `cupy`) are performance options that callers must opt into explicitly.
156+
**Default backend rule**: every public compute function that accepts a `backend=` argument MUST default to `"mdtraj"` **when mdtraj provides a native kernel for that computation**. Other backends (`numba`, `torch`, `jax`, `cupy`) are performance options that callers must opt into explicitly. The only current exception is `compute_dccm`, which defaults to `"numpy"` because mdtraj has no native covariance kernel -- `numpy`'s BLAS GEMM is multi-threaded and works without any optional dependency.
134157

135158
Reasons:
136159

137160
- Only `mdtraj` supports periodic boundary conditions -- defaulting to anything else would silently drop PBC for users who don't read the backend parameter.
138-
- All analysis functions sharing the same default keeps API behavior consistent across `compute_distances`, `compute_rmsd_matrix`, `featurize_ca_distances`, etc.
161+
- All PBC-relevant analysis functions sharing the same default keeps API behavior consistent across `compute_distances`, `compute_rmsd_matrix`, `featurize_ca_distances`, etc.
139162
- The optional GPU backends (`[gpu]` extra) must never be required for the common path.
140163

141-
When reviewing or writing code, never change a public function's default backend away from `"mdtraj"`.
164+
When reviewing or writing code, never silently change a public function's default backend away from its current value (`"mdtraj"`, or `"numpy"` for DCCM).
142165

143166
**Uniform signature rule**: every backend registered in a given `BackendRegistry` MUST accept the exact same call signature as the Protocol type parameter on that registry. If one backend needs an extra keyword argument (e.g. `periodic` on mdtraj), every other backend in the same registry MUST also accept that keyword, silently ignoring it if unused (mark `# noqa: ARG001` and document as "accepted for Protocol uniformity, ignored"). This keeps the dispatcher free of per-backend branching and preserves type inference for callers.
144167

@@ -182,16 +205,16 @@ For existing multi-backend functions (e.g. `compute_rmsd_matrix`, pairwise dista
182205
1. Decorate torch/cupy GPU kernels with `@clean_torch_cache` / `@clean_cupy_cache` from `_backends/_imports.py` so pooled memory is released in a `finally` block after the kernel runs. Do **not** apply any cleanup decorator to JAX kernels -- `jax.clear_caches()` trashes JIT compilation caches and forces slow recompiles.
183206
1. If you introduce a new keyword argument, also retrofit every existing backend in the same registry to accept it (silently ignoring when unused, marked `# noqa: ARG001`).
184207
1. Register in the module's `BackendRegistry` at the bottom of the file.
185-
1. Add the backend name to the corresponding `Literal` alias (`DistanceBackend` / `RMSDBackend`) in `_backends/_registry.py`.
208+
1. Add the backend name to the corresponding `Literal` alias (`DistanceBackend` / `RMSDBackend` / `DCCMBackend`) in `_backends/_registry.py`.
186209
1. Add agreement tests in `tests/analysis/test_<kind>.py` guarded by the relevant `requires_*` skip marker and `@pytest.mark.gpu` (if GPU-only).
187-
1. **Do not change the public function's default backend** -- keep it at `"mdtraj"`.
210+
1. **Do not change the public function's default backend** -- keep `compute_distances` / `compute_rmsd_matrix` / `featurize_ca_distances` defaulting to `"mdtraj"` and `compute_dccm` defaulting to `"numpy"`.
188211

189212
## Adding a New Backend Registry
190213

191214
To introduce a registry for a new multi-backend compute function:
192215

193216
1. Create `src/mdpp/analysis/_backends/_<kind>.py` with a `Protocol` class defining the shared call signature.
194-
1. Declare the registry as `<kind>_backends: BackendRegistry[<Kind>BackendFn] = BackendRegistry(default="mdtraj")` -- always parameterise with the Protocol so callers get typed `compute_fn` from `registry.get()`.
217+
1. Declare the registry as `<kind>_backends: BackendRegistry[<Kind>BackendFn] = BackendRegistry(default="mdtraj")` (or another sensible no-optional-dep default if mdtraj has no kernel for that computation -- e.g. `_dccm.py` uses `default="numpy"`). Always parameterise with the Protocol so callers get typed `compute_fn` from `registry.get()`.
195218
1. Add a `Literal` alias to `_backends/_registry.py` (`type <Kind>Backend = Literal["mdtraj", "numba", ...]`) and re-export it from `_backends/__init__.py`.
196219
1. The public wrapper in `src/mdpp/analysis/<kind>.py` imports the registry and delegates via `compute_fn = <kind>_backends.get(backend)`, letting mypy infer the Protocol type at the call site.
197220

0 commit comments

Comments
 (0)