Skip to content

Commit fb6eb6d

Browse files
theo-barfoottheo-barfoot
committed
test: stabilize JOSS suite for 0.2.2
1 parent 5a59722 commit fb6eb6d

29 files changed

Lines changed: 639 additions & 302 deletions

‎.github/workflows/python-package.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ jobs:
1414
fail-fast: false
1515
matrix:
1616
python-version: ["3.10", "3.11", "3.12"]
17-
torch-version: ["2.5.0", "2.9.0"]
17+
torch-version: ["2.5.0", "2.11.0"]
1818
torch-type: ["stable"]
1919
include:
2020
- python-version: "3.12"

‎JOSS_REVIEWER_RESPONSE.md‎

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# Draft JOSS Reviewer Response
2+
3+
Thank you for flagging the test-suite issue. I agree that the previous state made the package harder to validate reproducibly, especially on CUDA-visible machines.
4+
5+
The issue had two parts:
6+
7+
1. The default pytest suite included CUDA memory/performance/OOM experiments that were useful during development but were not appropriate for normal CI or reviewer validation.
8+
2. Several stochastic tests used random tensors without consistent global seeding, and some numerical/statistical assertions were too sensitive to CUDA float32 precision differences. This made a few tests appear flaky depending on the environment and random draw.
9+
10+
I have addressed this in a new release-readiness branch and prepared it for the `0.2.2` patch release.
11+
12+
The default test suite now:
13+
14+
- Uses deterministic global seeding for Python `random`, NumPy, PyTorch CPU RNG, and PyTorch CUDA RNGs.
15+
- Keeps normal CPU tests and lightweight CUDA functional tests enabled when CUDA is available.
16+
- Excludes CUDA memory/performance/OOM experiments unless explicitly requested.
17+
- Removes the previous `pytest.mark.flaky(reruns=...)` markers.
18+
- Uses shared dtype-aware tolerances for solver/backend tests.
19+
20+
The manual CUDA tests are still preserved for package development and memory-advantage validation. They can now be run explicitly with:
21+
22+
```bash
23+
python -m pytest --run-manual-cuda -m manual_cuda -s
24+
```
25+
26+
For normal reviewer validation after the `0.2.2` release, the intended clean-environment commands are:
27+
28+
```bash
29+
uv venv --python 3.12 --seed --managed-python
30+
pip install "torchsparsegradutils[all]==0.2.2"
31+
python -m pytest
32+
```
33+
34+
I also fixed the packaging issue you observed:
35+
36+
```text
37+
WARNING: torchsparsegradutils 0.2.1 does not provide the extra 'all'
38+
```
39+
40+
In `0.2.2`, the `all` extra now lists concrete optional dependencies directly instead of self-referencing the package. The built wheel metadata has been checked and now includes:
41+
42+
```text
43+
Provides-Extra: all
44+
Requires-Dist: cupy-cuda12x>=13.0; extra == "all"
45+
Requires-Dist: jax[cuda12]; extra == "all"
46+
```
47+
48+
I also updated CI to test the current supported stable PyTorch range:
49+
50+
- PyTorch `2.5.0`
51+
- PyTorch `2.11.0`
52+
- PyTorch nightly, allowed to fail
53+
54+
The README badge has been updated accordingly to:
55+
56+
```text
57+
Tested 2.5 / 2.11 / nightly
58+
```
59+
60+
Local verification on a CUDA-visible workspace now passes:
61+
62+
```text
63+
4289 passed, 822 skipped, 324 warnings
64+
```
65+
66+
I also repeated the known flaky target sweep, including:
67+
68+
- `test_sparse_batch_mv[batch_mv_test_data3]`
69+
- `test_bicgstab.py::test_bicgstab_2d_rhs`
70+
- representative linear CG, sparse solve, distribution sampling, CuPy, and JAX tests
71+
72+
That targeted sweep passed:
73+
74+
```text
75+
242 passed, 30 warnings
76+
```
77+
78+
The remaining skipped tests in the default run are intentional: they are the manual CUDA memory/performance/OOM experiments or tests skipped by existing backend/device capability checks. These are now documented and opt-in rather than part of the normal CI/reviewer command.
79+
80+
This should make the package reproducible for JOSS review while still preserving the GPU experiments I use to validate memory advantage over native PyTorch behavior.

‎PR_DESCRIPTION.md‎

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
# Pull Request: JOSS Test Suite and 0.2.2 Release Readiness
2+
3+
## Summary
4+
5+
This PR prepares `torchsparsegradutils` for the JOSS review follow-up and the `0.2.2` patch release. It separates manual CUDA memory/performance experiments from the default pytest suite, makes stochastic tests deterministic, fixes the broken `all` extra, and updates CI/docs for the latest stable PyTorch target.
6+
7+
The default test suite now keeps normal CUDA functional coverage enabled when CUDA is visible, while CUDA memory/performance/OOM-style experiments are preserved behind an explicit opt-in marker.
8+
9+
## What Changed
10+
11+
- Added deterministic global pytest seeding in `torchsparsegradutils/tests/conftest.py`.
12+
- Added seed opt-out environment variables:
13+
- `TSGU_UNLOCK_SEED=true`
14+
- legacy `UNLOCK_SEED=true`
15+
- Added `--run-manual-cuda` pytest option.
16+
- Added and registered `manual_cuda` marker for GPU memory/performance/OOM experiments.
17+
- Added `torchsparsegradutils/tests/test_config.py` for shared device/dtype/layout constants, dtype-aware tolerances, confidence thresholds, and device comparison helpers.
18+
- Removed `@pytest.mark.flaky(reruns=...)` markers after making tests deterministic.
19+
- Marked all of `test_integration_pairwise_sparse_mvn.py` as `manual_cuda`.
20+
- Marked `test_sparse_mm_memory_advantage` and `test_sparse_mm_memory_stability` as `manual_cuda`.
21+
- Kept lightweight CUDA functional tests in the default suite when CUDA is available.
22+
- Updated CI stable PyTorch matrix from `2.5.0` / `2.9.0` to `2.5.0` / `2.11.0`, with nightly still allowed to fail.
23+
- Updated README badge text to `Tested 2.5 / 2.11 / nightly`.
24+
- Bumped package/docs version from `0.2.1` to `0.2.2`.
25+
- Fixed the `all` extra so it expands to concrete optional dependencies instead of self-referencing `torchsparsegradutils[cupy,jax]`.
26+
- Added reviewer-oriented install/test commands to README and docs.
27+
28+
## Context Since `v0.2.1`
29+
30+
This branch builds on the post-`v0.2.1` mainline work:
31+
32+
- JOSS paper and documentation revisions.
33+
- More robust PyTorch version comparison using `packaging.version`.
34+
- Sparse matmul and sparse triangular solve reshape optimizations.
35+
- `sparse_eye` optimization avoiding unnecessary `.coalesce()`.
36+
- CuPy binding and sparse solve updates from the JOSS revision work.
37+
38+
## Reviewer / User-Facing Commands
39+
40+
After the `0.2.2` release:
41+
42+
```bash
43+
uv venv --python 3.12 --seed --managed-python
44+
pip install "torchsparsegradutils[all]==0.2.2"
45+
python -m pytest
46+
```
47+
48+
Manual CUDA memory/performance experiments:
49+
50+
```bash
51+
python -m pytest --run-manual-cuda -m manual_cuda -s
52+
```
53+
54+
## Verification
55+
56+
Run in a CUDA-visible workspace:
57+
58+
```bash
59+
black --check .
60+
isort --check-only --diff .
61+
flake8 . --count --show-source --statistics
62+
python -m pytest -q --ignore=torchsparsegradutils/tests/test_doctests.py
63+
python -m pytest -q -m manual_cuda --collect-only
64+
python -m build --wheel
65+
```
66+
67+
Results:
68+
69+
- `black --check .`: passed
70+
- `isort --check-only --diff .`: passed
71+
- `flake8 . --count --show-source --statistics`: passed
72+
- Default pytest excluding doctests: `4289 passed, 822 skipped, 324 warnings`
73+
- Known flaky target sweep: `242 passed, 30 warnings`
74+
- Manual CUDA collection: `676/5128 tests collected`, `4452 deselected`
75+
- Wheel build: passed
76+
- Wheel metadata confirmed:
77+
- `Version: 0.2.2`
78+
- `Provides-Extra: all`
79+
- `Requires-Dist: cupy-cuda12x>=13.0; extra == "all"`
80+
- `Requires-Dist: jax[cuda12]; extra == "all"`
81+
82+
## Notes
83+
84+
The full manual CUDA suite is intentionally not part of default CI. It remains available for local validation of memory advantage, performance, and OOM behavior on suitable GPU hardware.

‎README.md‎

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# torchsparsegradutils: Sparsity-preserving gradient utility tools for PyTorch
22

3-
[![PyPI](https://img.shields.io/pypi/v/torchsparsegradutils.svg)](https://pypi.org/project/torchsparsegradutils/) [![Python Versions](https://img.shields.io/pypi/pyversions/torchsparsegradutils.svg)](https://pypi.org/project/torchsparsegradutils/) [![Downloads](https://img.shields.io/pypi/dm/torchsparsegradutils.svg)](https://pypi.org/project/torchsparsegradutils/) ![PyTorch 2.5+](https://img.shields.io/badge/PyTorch-2.5%2B-ee4c2c?logo=pytorch) ![Tested 2.5 / 2.9 / nightly](https://img.shields.io/badge/Tested-2.5%20|%202.9%20|%20nightly-ee4c2c?logo=pytorch) [![Build](https://github.com/cai4cai/torchsparsegradutils/actions/workflows/python-package.yml/badge.svg)](https://github.com/cai4cai/torchsparsegradutils/actions/workflows/python-package.yml) [![Docs](https://readthedocs.org/projects/torchsparsegradutils/badge/?version=latest)](https://readthedocs.org/projects/torchsparsegradutils) [![Code Style: Black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) [![License](https://img.shields.io/github/license/cai4cai/torchsparsegradutils)](LICENSE) [![status](https://joss.theoj.org/papers/6da0e92488d06f70c0a03d0a7cbfba7d/status.svg)](https://joss.theoj.org/papers/6da0e92488d06f70c0a03d0a7cbfba7d)
3+
[![PyPI](https://img.shields.io/pypi/v/torchsparsegradutils.svg)](https://pypi.org/project/torchsparsegradutils/) [![Python Versions](https://img.shields.io/pypi/pyversions/torchsparsegradutils.svg)](https://pypi.org/project/torchsparsegradutils/) [![Downloads](https://img.shields.io/pypi/dm/torchsparsegradutils.svg)](https://pypi.org/project/torchsparsegradutils/) ![PyTorch 2.5+](https://img.shields.io/badge/PyTorch-2.5%2B-ee4c2c?logo=pytorch) ![Tested 2.5 / 2.11 / nightly](https://img.shields.io/badge/Tested-2.5%20|%202.11%20|%20nightly-ee4c2c?logo=pytorch) [![Build](https://github.com/cai4cai/torchsparsegradutils/actions/workflows/python-package.yml/badge.svg)](https://github.com/cai4cai/torchsparsegradutils/actions/workflows/python-package.yml) [![Docs](https://readthedocs.org/projects/torchsparsegradutils/badge/?version=latest)](https://readthedocs.org/projects/torchsparsegradutils) [![Code Style: Black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) [![License](https://img.shields.io/github/license/cai4cai/torchsparsegradutils)](LICENSE) [![status](https://joss.theoj.org/papers/6da0e92488d06f70c0a03d0a7cbfba7d/status.svg)](https://joss.theoj.org/papers/6da0e92488d06f70c0a03d0a7cbfba7d)
44

55
A comprehensive collection of utility functions to work with PyTorch sparse tensors, ensuring memory efficiency and supporting various sparsity-preserving tensor operations with automatic differentiation. This package addresses fundamental gaps in PyTorch's sparse tensor ecosystem, providing essential operations that preserve sparsity in gradients during backpropagation.
66

@@ -106,6 +106,22 @@ pip install scipy matplotlib pandas tqdm pytest
106106

107107
> **Note:** The CuPy extra installs `cupy-cuda12x>=13.0`. If you are using a different CUDA version, install the appropriate CuPy package manually (e.g. `pip install cupy-cuda11x`).
108108
109+
### Reviewer Test Environment
110+
111+
For a clean Python 3.12 environment with all optional dependencies after the 0.2.2 release:
112+
113+
```bash
114+
uv venv --python 3.12 --seed --managed-python
115+
pip install "torchsparsegradutils[all]==0.2.2"
116+
python -m pytest
117+
```
118+
119+
The default pytest suite includes CPU tests and lightweight CUDA functional tests when CUDA is available. CUDA memory, performance, and OOM experiments are preserved for manual validation and can be run explicitly:
120+
121+
```bash
122+
python -m pytest --run-manual-cuda -m manual_cuda -s
123+
```
124+
109125
### Requirements
110126

111127
- **Python**: ≥ 3.10
@@ -396,6 +412,9 @@ python -m pytest torchsparsegradutils/tests/test_distributions.py
396412

397413
# Run with coverage
398414
python -m pytest --cov=torchsparsegradutils
415+
416+
# Run CUDA memory/performance experiments manually
417+
python -m pytest --run-manual-cuda -m manual_cuda -s
399418
```
400419

401420
### Running Benchmarks
@@ -662,4 +681,4 @@ dist_stable = SparseMultivariateNormal(
662681
- **No SPD Constraints**: Doesn't require strict positive definiteness
663682
- **Better Conditioning**: Diagonal component can be controlled independently
664683

665-
**Status**: This is a known limitation of the LL^T precision formulation. LDL^T parameterization is the recommended approach for precision matrices.
684+
**Status**: This is a known limitation of the LL^T precision formulation. LDL^T parameterization is the recommended approach for precision matrices.

‎RELEASE_NOTES_0.2.2.md‎

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
# Release Notes: torchsparsegradutils 0.2.2
2+
3+
Patch release focused on JOSS review readiness, deterministic testing, CUDA test separation, packaging metadata, and refreshed PyTorch compatibility validation.
4+
5+
## Highlights
6+
7+
- Default pytest is now suitable for reviewers and CI on CPU or CUDA-visible machines.
8+
- CUDA memory/performance/OOM experiments are preserved but moved behind an explicit manual marker.
9+
- Stochastic tests are deterministic by default.
10+
- The `all` optional dependency extra is fixed.
11+
- CI now tests PyTorch `2.5.0`, `2.11.0`, and nightly.
12+
- README badge updated to `Tested 2.5 / 2.11 / nightly`.
13+
14+
## Testing and CI
15+
16+
- Added global deterministic test seeding for:
17+
- Python `random`
18+
- NumPy
19+
- PyTorch CPU RNG
20+
- PyTorch CUDA RNGs
21+
- Added seed opt-out support:
22+
- `TSGU_UNLOCK_SEED=true`
23+
- `UNLOCK_SEED=true`
24+
- Added `--run-manual-cuda` pytest option.
25+
- Added `manual_cuda` marker for CUDA memory/performance/OOM experiments.
26+
- Added shared test constants and dtype-aware tolerances in `torchsparsegradutils/tests/test_config.py`.
27+
- Removed flaky rerun markers after stabilizing seeding and tolerance behavior.
28+
- Marked pairwise sparse MVN integration tests as manual CUDA.
29+
- Marked sparse matrix multiplication memory advantage and memory stability tests as manual CUDA.
30+
- Kept small CUDA functional tests in the default suite when CUDA is available.
31+
- Updated GitHub Actions stable PyTorch matrix:
32+
- `2.5.0`
33+
- `2.11.0`
34+
- Kept nightly CPU CI on Python `3.12` with `continue-on-error`.
35+
36+
## Packaging
37+
38+
- Bumped version from `0.2.1` to `0.2.2`.
39+
- Fixed `torchsparsegradutils[all]`.
40+
- The `all` extra now expands directly to:
41+
- `cupy-cuda12x>=13.0`
42+
- `jax[cuda12]`
43+
- Wheel metadata now includes `Provides-Extra: all`.
44+
45+
## Documentation
46+
47+
- Added reviewer-oriented installation and test commands:
48+
49+
```bash
50+
uv venv --python 3.12 --seed --managed-python
51+
pip install "torchsparsegradutils[all]==0.2.2"
52+
python -m pytest
53+
```
54+
55+
- Documented manual CUDA validation command:
56+
57+
```bash
58+
python -m pytest --run-manual-cuda -m manual_cuda -s
59+
```
60+
61+
- Updated README badge to `Tested 2.5 / 2.11 / nightly`.
62+
- Updated docs version metadata to `0.2.2`.
63+
64+
## Changes Since `v0.2.1`
65+
66+
In addition to this release-readiness work, the current release includes the post-`v0.2.1` mainline changes:
67+
68+
- JOSS paper revisions and rebuilt paper artifacts.
69+
- Installation and benchmark documentation updates from reviewer feedback.
70+
- More robust PyTorch version comparison using `packaging.version`.
71+
- Sparse matrix multiplication reshape optimization.
72+
- Sparse triangular solve reshape optimization.
73+
- `sparse_eye` optimization using the `is_coalesced` flag instead of forcing `.coalesce()`.
74+
- CuPy binding and sparse solve updates from JOSS review work.
75+
76+
## Verification
77+
78+
Verified in a CUDA-visible workspace:
79+
80+
```bash
81+
black --check .
82+
isort --check-only --diff .
83+
flake8 . --count --show-source --statistics
84+
python -m pytest -q --ignore=torchsparsegradutils/tests/test_doctests.py
85+
python -m pytest -q -m manual_cuda --collect-only
86+
python -m build --wheel
87+
```
88+
89+
Observed results:
90+
91+
- Default pytest excluding doctests: `4289 passed, 822 skipped, 324 warnings`
92+
- Known flaky target sweep: `242 passed, 30 warnings`
93+
- Manual CUDA collection: `676/5128 tests collected`, `4452 deselected`
94+
- Wheel build: passed
95+
- Wheel metadata includes `Provides-Extra: all`
96+
97+
## Upgrade Notes
98+
99+
After publication:
100+
101+
```bash
102+
pip install --upgrade "torchsparsegradutils[all]==0.2.2"
103+
```
104+
105+
Users who do not need CuPy/JAX support can continue installing the base package:
106+
107+
```bash
108+
pip install --upgrade torchsparsegradutils==0.2.2
109+
```
110+
111+
The CuPy extra currently targets CUDA 12 via `cupy-cuda12x`. Users on a different CUDA runtime should install the appropriate CuPy package manually.

‎docs/source/conf.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,8 @@
1515
project = "torchsparsegradutils"
1616
copyright = "2025, CAI4CAI research group"
1717
author = "CAI4CAI research group"
18-
release = "0.2.1"
19-
version = "0.2.1"
18+
release = "0.2.2"
19+
version = "0.2.2"
2020

2121
# -- General configuration ---------------------------------------------------
2222
# https://www.sphinx-doc.org/en/master/usage/configuration.html#general-configuration

‎docs/source/contributing.rst‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -144,6 +144,9 @@ We use pytest for testing. Tests are organized by module:
144144
# Run with coverage
145145
pytest --cov=torchsparsegradutils
146146
147+
# Run CUDA memory/performance experiments manually
148+
pytest --run-manual-cuda -m manual_cuda -s
149+
147150
**Writing Tests**
148151

149152
Follow these guidelines:

‎docs/source/installation.rst‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,26 @@ For additional functionality, you can install optional dependencies:
3434
CUDA version, install the appropriate CuPy package manually
3535
(e.g. ``pip install cupy-cuda11x``).
3636

37+
Reviewer Test Environment
38+
-------------------------
39+
40+
For a clean Python 3.12 environment with all optional dependencies after the
41+
0.2.2 release:
42+
43+
.. code-block:: bash
44+
45+
uv venv --python 3.12 --seed --managed-python
46+
pip install "torchsparsegradutils[all]==0.2.2"
47+
python -m pytest
48+
49+
The default pytest suite includes CPU tests and lightweight CUDA functional
50+
tests when CUDA is available. CUDA memory, performance, and OOM experiments are
51+
preserved for manual validation and can be run explicitly:
52+
53+
.. code-block:: bash
54+
55+
python -m pytest --run-manual-cuda -m manual_cuda -s
56+
3757
Requirements
3858
------------
3959

0 commit comments

Comments
 (0)