Skip to content

Commit 98be238

Browse files
committed
Update dependency compatibility install path
1 parent aa76ea0 commit 98be238

12 files changed

Lines changed: 1215 additions & 65 deletions

File tree

.github/workflows/unit_test.yml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,6 @@ jobs:
1616
output1: ${{ steps.s1.outputs.test }}
1717
output2: ${{ steps.s2.outputs.test }}
1818
if: github.repository_owner == 'deepmodeling'
19-
container: ghcr.io/deepmodeling/deeptb:latest
2019
steps:
2120
- name: Checkout
2221
id: s1
Lines changed: 125 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,125 @@
1+
# Dependency Compatibility Matrix
2+
3+
DeePTB depends on PyTorch, PyG, and the compiled `torch-scatter` extension.
4+
Compatibility testing should therefore verify the exact Python ABI, Torch
5+
version, and PyG binary wheel combination instead of only checking package
6+
version specifiers.
7+
8+
## Local Matrix Tool
9+
10+
Use the matrix runner from the repository root:
11+
12+
```bash
13+
python tools/compat/test_matrix.py --job py312-torch251-cpu
14+
python tools/compat/test_matrix.py --job py312-torch210-cpu
15+
```
16+
17+
The tool creates one isolated uv environment per job under `.compat-envs/`,
18+
installs DeePTB in editable mode, runs an import and `torch_scatter.scatter_add`
19+
smoke check, then runs the default smoke tests:
20+
21+
```bash
22+
python -m pytest dptb/tests -m smoke -q
23+
```
24+
25+
Results are written as JSON files under `tools/compat/results/`.
26+
27+
## Installation Policy
28+
29+
DeePTB now separates the installation surface into two layers:
30+
31+
- `install.sh`: tested installation channel for new machines. It detects or
32+
accepts the backend, pins the corresponding Torch / PyG / `torch-scatter`
33+
combination, and refuses source builds for critical compiled dependencies.
34+
- `pyproject.toml`: developer compatibility range. It allows developers to use
35+
`uv sync`, `pip install -e .`, or an existing compatible Torch environment,
36+
but it does not encode CUDA-driver-specific wheel choices.
37+
38+
The tested installer is the recommended user-facing path. The project metadata
39+
range is intentionally broader so that intermediate Torch releases, such as
40+
Torch 2.7, can be used by developers when a matching `torch-scatter` binary
41+
wheel is available.
42+
43+
CI should exercise the same installer path as users. Do not run `uv sync` after
44+
`install.sh`, because that re-resolves the environment from `uv.lock` /
45+
`pyproject.toml` and can replace the Torch / PyG / `torch-scatter` combination
46+
selected by the installer. For tests that need optional dependencies, use:
47+
48+
```bash
49+
bash install.sh cpu --extra pythtb --test
50+
.venv/bin/python -m pytest ./dptb/tests/
51+
```
52+
53+
The CI runner must also be new enough for the selected binary wheels. The
54+
current CPU installer uses Torch 2.12.1, and the matching Linux
55+
`torch-scatter` wheel requires a newer glibc than the old Ubuntu 20.04-based
56+
test container provides. The unit-test workflow therefore runs on the
57+
`ubuntu-latest` host instead of the legacy `ghcr.io/deepmodeling/deeptb:latest`
58+
container.
59+
60+
Runtime dependencies should avoid exact pins unless the package is tied to a
61+
binary wheel matrix or a known API break. `torch-scatter==2.1.2` is intentionally
62+
fixed because the available PyG wheels define the supported Torch/CUDA
63+
combinations. Test-only packages such as `pytest` and `pytest-order` belong in
64+
the development dependency group, not in the runtime install set.
65+
66+
After relaxing runtime pins, run at least a resolver check and one install/import
67+
check before accepting the change:
68+
69+
```bash
70+
uv lock --dry-run
71+
python tools/compat/test_matrix.py --job py312-torch2121-cpu --no-tests
72+
python tools/compat/test_matrix.py --job py313-torch2121-cpu-modern-cdeps --no-tests
73+
```
74+
75+
## Binary Wheel Rule
76+
77+
`torch-scatter` must be installed from a PyG binary wheel for supported matrix
78+
entries. Do not treat a source build as a supported installation path.
79+
80+
Each job passes a matching PyG wheel page, for example:
81+
82+
```bash
83+
https://data.pyg.org/whl/torch-2.10.0+cpu.html
84+
https://data.pyg.org/whl/torch-2.10.0+cu128.html
85+
```
86+
87+
The runner uses `--only-binary torch-scatter` and fails if the installed
88+
`torch-scatter` distribution metadata is not a platform binary wheel. On Linux
89+
CUDA jobs, the version may include a local suffix such as `2.1.2+pt210cu128`.
90+
On macOS CPU jobs, the wheel can still be binary even when the package version
91+
is just `2.1.2`; inspect the wheel tag instead of relying only on the version
92+
string.
93+
94+
## Suggested Exploration Order
95+
96+
For local macOS CPU checks:
97+
98+
1. `py310-torch251-cpu`: current baseline.
99+
2. `py312-torch251-cpu`: current declared Python upper range with current Torch.
100+
3. `py312-torch210-cpu`: primary Torch upgrade probe.
101+
4. `py313-torch210-cpu`: primary Python and Torch upgrade probe.
102+
5. `py314-torch211-cpu`: experimental only.
103+
104+
For RTX 50 / Blackwell machines, reuse the same runner with CUDA jobs that use
105+
`cu128` or newer PyG wheel pages. The decision order should be:
106+
107+
```text
108+
GPU model -> compute capability -> PyTorch CUDA wheel -> torch-scatter PyG wheel
109+
```
110+
111+
## Tested Support Summary
112+
113+
- macOS CPU: Python 3.12 / 3.13 with Torch 2.10.0 and 2.12.1 passed install,
114+
import, smoke, regression, and `not slow` pytest checks.
115+
- RTX 5090 with driver 570.211.01 / CUDA 12.8: Python 3.12 / 3.13 with
116+
`torch 2.10.0+cu128`, `torch-geometric 2.8.0`, and
117+
`torch-scatter 2.1.2+pt210cu128` passed install, import, smoke, regression,
118+
and `not slow` pytest checks.
119+
- RTX 5090 with the current driver cannot use `torch 2.12.x+cu130/cu132`; the
120+
wheels install, but CUDA is unavailable because the driver is too old for the
121+
CUDA 13 runtime.
122+
- `torch 2.12.x+cu128` is not a supported 5090 path because the PyG index does
123+
not provide a matching `torch-scatter` binary wheel.
124+
- Python 3.14 remains experimental. Torch warns that `torch.jit.script` is not
125+
supported there, so Python 3.14 needs a separate TorchScript migration task.

0 commit comments

Comments
 (0)