Skip to content

Commit c95d9a9

Browse files
ericspodpre-commit-ci[bot]
ericspod
and
pre-commit-ci[bot]
authored
Refactoring min-dep tests (Project-MONAI#8561)
### Description This PR is an effort at refactoring the CICD tests to reduce code volume in the yaml files. Branch rules will have to be temporarily changed to not expect jobs by names which are no longer used, and once accepted changed again to add the jobs with the new names back in. This would be a good opportunity to move branch rules to rule sets which is the newer Github mechanism. This also adds code for removing unused tools in the Linux runners to avoid running out of disk space during testing, this has increasingly become an issue for testing as the image has gotten larger and more libraries get installed. This also removes testing support for Python 3.9 and updates which versions of PyTorch are tested with in line with our support policy. This also bumps versions of some actions and dependencies, specifically black for security reasons. Testing Windows with full dependencies requires a much later version of PyTorch due to a gloo bug, and macOS requires the explicit installation of pybind11 for now. A number of other formatting fixes had to be made to account for newer versions of black and other tools, and a number of imports were moved around to match Python 3.10 usage. A number of type annotations were changed to remove Union and Optional. Windows tests with versions of PyTorch between 2.8 and 2.10 seem to suffer from the gloo bug mentioned in pytorch/pytorch#150381. ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [x] Breaking change (fix or new feature that would cause existing functionality to change). - [ ] New tests added to cover the changes. - [x] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [x] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [ ] In-line docstrings updated. - [ ] Documentation updated, tested `make html` command in the `docs/` folder. --------- Signed-off-by: Eric Kerfoot <17726042+ericspod@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
1 parent 862f3a6 commit c95d9a9

64 files changed

Lines changed: 541 additions & 572 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/build_docs.yml‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
name: build_docs
2+
3+
on:
4+
push:
5+
branches:
6+
- dev
7+
- main
8+
- releasing/*
9+
pull_request:
10+
branches:
11+
- dev
12+
- main
13+
- releasing/*
14+
15+
concurrency:
16+
# automatically cancel the previously triggered workflows when there's a newer version
17+
group: build-docs-${{ github.event.pull_request.number || github.ref }}
18+
cancel-in-progress: true
19+
20+
jobs:
21+
build-docs:
22+
runs-on: ubuntu-latest
23+
env:
24+
# minimum supported version of Python
25+
PYTHON_VER1: '3.10'
26+
steps:
27+
- uses: actions/checkout@v6
28+
- name: Set up Python ${{ env.PYTHON_VER1 }}
29+
uses: actions/setup-python@v6
30+
with:
31+
python-version: ${{ env.PYTHON_VER1 }}
32+
cache: 'pip'
33+
- name: Install dependencies
34+
run: |
35+
python -m pip install --upgrade pip wheel
36+
python -m pip install -r docs/requirements.txt --extra-index-url https://download.pytorch.org/whl/cpu
37+
- name: Make html
38+
run: |
39+
cd docs/
40+
make clean
41+
make html 2>&1 | tee tmp_log
42+
if [[ $(grep -c "ERROR:" tmp_log) != 0 ]]; then echo "Found errors"; grep "ERROR:" tmp_log; exit 1; fi
43+
sed '/WARNING.*pip/d' tmp_log > tmp_log1; mv tmp_log1 tmp_log # monai#7133
44+
if [[ $(grep -c "WARNING:" tmp_log) != 0 ]]; then echo "Found warnings"; grep "WARNING:" tmp_log; exit 1; fi
45+
shell: bash

‎.github/workflows/cicd_tests.yml‎

Lines changed: 317 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,317 @@
1+
name: tests
2+
3+
on:
4+
# quick tests for pull requests and the releasing branches
5+
push:
6+
branches:
7+
- dev
8+
- main
9+
- releasing/*
10+
paths-ignore: # skip if only docs are modified
11+
- '**.md'
12+
- '**.rst'
13+
- 'docs/**'
14+
pull_request:
15+
branches:
16+
- dev
17+
- main
18+
- releasing/*
19+
paths-ignore: # skip if only docs are modified
20+
- '**.md'
21+
- '**.rst'
22+
- 'docs/**'
23+
24+
concurrency:
25+
# automatically cancel the previously triggered workflows when there's a newer version
26+
group: cicd-tests-${{ github.event.pull_request.number || github.ref }}
27+
cancel-in-progress: true
28+
29+
# Supported versions of Python and PyTorch are listed here for use below and as documentation
30+
env:
31+
# supported versions of Python
32+
PYTHON_VER1: '3.10'
33+
PYTHON_VER2: '3.11'
34+
PYTHON_VER3: '3.12'
35+
PYTHON_VER4: '3.13'
36+
# PYTHON_VER5: '3.14' # TODO: not compatible with Torchscript, re-enable once Torchscript removed from MONAI
37+
# supported versions of PyTorch
38+
PYTORCH_VER1: '2.8.0'
39+
PYTORCH_VER2: '2.9.1'
40+
PYTORCH_VER3: '2.10.0'
41+
PYTORCH_VER4: '2.11.0'
42+
TORCHVISION_VER1: '0.23.0' # used for testing with lowest PyTorch version (PYTORCH_VER1), update as needed
43+
44+
# versions of PyTorch/Torchvision to use for full-dep Windows tests, needed to get around a gloo bug
45+
WIN_FULLDEP_VER: '2.11.0'
46+
WIN_FULLDEP_VISION_VER: '0.26.0'
47+
48+
PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION: python # https://github.com/Project-MONAI/MONAI/issues/4354
49+
50+
# These jobs run the CICD tests, type checking, and testing packaging. These use the minimum supported versions of
51+
# Python and PyTorch in many places using the above environment variables but also hard coded where necessary.
52+
# When support is dropped for a version it is important to update these as appropriate.
53+
54+
jobs:
55+
static-checks: # Perform static type and other checks using runtests.sh
56+
runs-on: ubuntu-latest
57+
strategy:
58+
matrix:
59+
opt: ["codeformat", "pytype", "mypy"]
60+
steps:
61+
- name: Clean unused tools
62+
run: |
63+
find /opt/hostedtoolcache/* -maxdepth 0 ! -name 'Python' -exec rm -rf {} \;
64+
sudo rm -rf /usr/share/dotnet
65+
sudo rm -rf /usr/local/lib/android
66+
sudo rm -rf /opt/ghc /usr/local/.ghcup
67+
sudo docker system prune -f
68+
69+
- uses: actions/checkout@v6
70+
- name: Set up Python ${{ env.PYTHON_VER1 }}
71+
uses: actions/setup-python@v6
72+
with:
73+
python-version: ${{ env.PYTHON_VER1 }}
74+
cache: 'pip'
75+
- name: Install dependencies
76+
run: |
77+
python -m pip install --upgrade pip wheel
78+
python -m pip install --no-build-isolation -r requirements-dev.txt
79+
- name: Lint and type check
80+
run: |
81+
# clean up temporary files
82+
$(pwd)/runtests.sh --build --clean
83+
# Github actions have multiple cores, so parallelize pytype
84+
$(pwd)/runtests.sh --build --${{ matrix.opt }} -j $(nproc --all)
85+
86+
min-dep: # Test with minumum dependencies installed for different OS, Python, and PyTorch combinations
87+
runs-on: ${{ matrix.os }}
88+
strategy:
89+
fail-fast: false
90+
matrix:
91+
# Base test cases are to test on all three OSes with the lowest versions of Python and PyTorch, other cases are
92+
# added below as special cases to cover other Python and PyTorch versions under Ubuntu
93+
os: [windows-latest, macOS-latest, ubuntu-latest]
94+
python-version: ['3.10']
95+
pytorch-version: ['2.8.0']
96+
include:
97+
# Test Python versions under Ubuntu with lowest PyTorch version supported
98+
- os: ubuntu-latest
99+
pytorch-version: '2.8.0'
100+
python-version: '3.11'
101+
- os: ubuntu-latest
102+
pytorch-version: '2.8.0'
103+
python-version: '3.12'
104+
- os: ubuntu-latest
105+
pytorch-version: '2.8.0'
106+
python-version: '3.13'
107+
# - os: ubuntu-latest
108+
# pytorch-version: '2.9.1'
109+
# python-version: '3.14'
110+
111+
# Test PyTorch versions under Ubuntu with lowest Python version supported
112+
- os: ubuntu-latest
113+
python-version: '3.10'
114+
pytorch-version: '2.9.1'
115+
- os: ubuntu-latest
116+
python-version: '3.10'
117+
pytorch-version: '2.10.0'
118+
- os: ubuntu-latest
119+
python-version: '3.10'
120+
pytorch-version: '2.11.0'
121+
122+
timeout-minutes: 40
123+
steps:
124+
- if: runner.os == 'Linux'
125+
name: Clean unused tools
126+
run: |
127+
find /opt/hostedtoolcache/* -maxdepth 0 ! -name 'Python' -exec rm -rf {} \;
128+
sudo rm -rf /usr/share/dotnet
129+
sudo rm -rf /usr/local/lib/android
130+
sudo rm -rf /opt/ghc /usr/local/.ghcup
131+
sudo docker system prune -f
132+
- uses: actions/checkout@v6
133+
- name: Set up Python ${{ matrix.python-version }}
134+
uses: actions/setup-python@v6
135+
with:
136+
python-version: ${{ matrix.python-version }}
137+
cache: 'pip'
138+
- name: Prepare pip wheel
139+
run: |
140+
which python
141+
python -m pip install --upgrade pip wheel
142+
python -m pip install --user more-itertools>=8.0
143+
- name: Install the minimum dependencies
144+
run: |
145+
# min. requirements
146+
python -m pip install torch==${{ matrix.pytorch-version }}
147+
python -m pip install -r requirements-min.txt
148+
python -m pip list
149+
BUILD_MONAI=0 python setup.py develop # no compile of extensions
150+
shell: bash
151+
- if: matrix.os == 'linux-gpu-runner'
152+
name: Print GPU Info
153+
run: |
154+
nvidia-smi
155+
python -c 'import torch; print(torch.rand(2,2).to("cuda:0"))'
156+
shell: bash
157+
- name: Run quick tests
158+
run: |
159+
python -c 'import torch; print(torch.__version__); print(torch.rand(5,3))'
160+
python -c "import monai; monai.config.print_config()"
161+
./runtests.sh --min
162+
shell: bash
163+
env:
164+
QUICKTEST: True
165+
NGC_API_KEY: ${{ secrets.NGC_API_KEY }}
166+
NGC_ORG: ${{ secrets.NGC_ORG }}
167+
NGC_TEAM: ${{ secrets.NGC_TEAM }}
168+
169+
full-dep: # Test with full dependencies installed for different OS runners
170+
runs-on: ${{ matrix.os }}
171+
strategy:
172+
fail-fast: false
173+
matrix:
174+
os: [windows-latest, macOS-latest, ubuntu-latest]
175+
timeout-minutes: 120
176+
env:
177+
QUICKTEST: True
178+
steps:
179+
- if: runner.os == 'Linux'
180+
name: Clean unused tools
181+
run: |
182+
find /opt/hostedtoolcache/* -maxdepth 0 ! -name 'Python' -exec rm -rf {} \;
183+
sudo rm -rf /usr/share/dotnet
184+
sudo rm -rf /usr/local/lib/android
185+
sudo rm -rf /opt/ghc /usr/local/.ghcup
186+
sudo docker system prune -f
187+
- if: runner.os == 'windows'
188+
name: Config pagefile (Windows only)
189+
uses: al-cheb/configure-pagefile-action@v1.5
190+
with:
191+
minimum-size: 8GB
192+
maximum-size: 16GB
193+
disk-root: "D:"
194+
- uses: actions/checkout@v6
195+
- name: Set up Python ${{ env.PYTHON_VER1 }}
196+
uses: actions/setup-python@v6
197+
with:
198+
python-version: ${{ env.PYTHON_VER1 }}
199+
cache: 'pip'
200+
- name: Prepare pip wheel
201+
run: |
202+
which python
203+
python -m pip install --upgrade pip wheel
204+
- if: runner.os == 'windows'
205+
name: Install torch cpu from pytorch.org (Windows only)
206+
run: |
207+
python -m pip install torch==${WIN_FULLDEP_VER} torchvision==${WIN_FULLDEP_VISION_VER}+cpu --index-url https://download.pytorch.org/whl/cpu
208+
shell: bash
209+
- if: runner.os != 'windows'
210+
name: Install torch gpu
211+
run: |
212+
# install the lowest version of PyTorch supported
213+
python -m pip install torch==${PYTORCH_VER1} torchvision==${TORCHVISION_VER1}
214+
shell: bash
215+
- if: runner.os == 'Linux'
216+
name: Install itk pre-release (Linux only)
217+
run: |
218+
python -m pip install --pre -U itk
219+
shell: bash
220+
- name: Install the complete dependencies
221+
run: |
222+
python -m pip install --user --upgrade pip wheel pybind11 # TODO: pybind11 added for macOS, may not be needed
223+
#python -m pip install torch==${PYTORCH_VER1} torchvision==${TORCHVISION_VER1}
224+
cat "requirements-dev.txt"
225+
python -m pip install --no-build-isolation -r requirements-dev.txt
226+
python -m pip list
227+
python -m pip install -e . # test no compile installation
228+
shell: bash
229+
- name: Run compiled (${{ runner.os }})
230+
run: |
231+
python -m pip uninstall -y monai
232+
BUILD_MONAI=1 python -m pip install -e . # compile the cpp extensions
233+
shell: bash
234+
- name: Run quick tests
235+
run: |
236+
python -c 'import torch; print(torch.__version__); print(torch.rand(5,3))'
237+
python -c "import monai; monai.config.print_config()"
238+
python -m unittest -v
239+
shell: bash
240+
241+
packaging: # Test package generation
242+
runs-on: ubuntu-latest
243+
env:
244+
QUICKTEST: True
245+
steps:
246+
- name: Clean unused tools
247+
run: |
248+
find /opt/hostedtoolcache/* -maxdepth 0 ! -name 'Python' -exec rm -rf {} \;
249+
sudo rm -rf /usr/share/dotnet
250+
sudo rm -rf /usr/local/lib/android
251+
sudo rm -rf /opt/ghc /usr/local/.ghcup
252+
sudo docker system prune -f
253+
- uses: actions/checkout@v6
254+
with:
255+
fetch-depth: 0
256+
- name: Set up Python ${{ env.PYTHON_VER1 }}
257+
uses: actions/setup-python@v6
258+
with:
259+
python-version: ${{ env.PYTHON_VER1 }}
260+
cache: 'pip'
261+
- name: Install dependencies
262+
run: |
263+
python -m pip install --user --upgrade pip setuptools wheel twine packaging
264+
# install the latest pytorch for testing
265+
# however, "pip install monai*.tar.gz" will build cpp/cuda with an isolated
266+
# fresh torch installation according to pyproject.toml
267+
python -m pip install torch==${PYTORCH_VER1} torchvision --extra-index-url https://download.pytorch.org/whl/cpu
268+
- name: Check packages
269+
run: |
270+
pip uninstall monai
271+
pip list | grep -iv monai
272+
git fetch --depth=1 origin +refs/tags/*:refs/tags/*
273+
set -e
274+
275+
# build tar.gz and wheel
276+
python setup.py check -m -s
277+
python setup.py sdist bdist_wheel
278+
python -m twine check dist/*
279+
- run: echo "pwd=$PWD" >> $GITHUB_OUTPUT
280+
id: root
281+
- run: echo "tmp_dir=$(mktemp -d)" >> $GITHUB_OUTPUT
282+
id: mktemp
283+
- name: Move packages
284+
run: |
285+
printf ${{ steps.root.outputs.pwd }}
286+
printf ${{ steps.mktemp.outputs.tmp_dir }}
287+
# move packages to a temp dir
288+
cp dist/monai* "${{ steps.mktemp.outputs.tmp_dir }}"
289+
rm -r build dist monai.egg-info
290+
cd "${{ steps.mktemp.outputs.tmp_dir }}"
291+
ls -al
292+
- name: Install wheel file
293+
working-directory: ${{ steps.mktemp.outputs.tmp_dir }}
294+
run: |
295+
# install from wheel
296+
python -m pip install monai*.whl --extra-index-url https://download.pytorch.org/whl/cpu
297+
python -c 'import monai; monai.config.print_config()' 2>&1 | grep -iv "unknown"
298+
python -c 'import monai; print(monai.__file__)'
299+
python -m pip uninstall -y monai
300+
rm monai*.whl
301+
- name: Install source archive
302+
working-directory: ${{ steps.mktemp.outputs.tmp_dir }}
303+
run: |
304+
for name in *.tar.gz; do break; done
305+
echo $name
306+
python -m pip install ${name}[all] --extra-index-url https://download.pytorch.org/whl/cpu
307+
python -c 'import monai; monai.config.print_config()' 2>&1 | grep -iv "unknown"
308+
python -c 'import monai; print(monai.__file__)'
309+
- name: Quick test
310+
working-directory: ${{ steps.mktemp.outputs.tmp_dir }}
311+
run: |
312+
# run min tests
313+
cp ${{ steps.root.outputs.pwd }}/requirements*.txt .
314+
cp -r ${{ steps.root.outputs.pwd }}/tests .
315+
ls -al
316+
python -m pip install --no-build-isolation -r requirements-dev.txt --extra-index-url https://download.pytorch.org/whl/cpu
317+
python -m unittest -v

‎.github/workflows/conda.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ jobs:
1919
fail-fast: false
2020
matrix:
2121
os: [ubuntu-latest]
22-
python-version: ["3.9", "3.10"]
22+
python-version: ["3.10", "3.11"]
2323
runs-on: ${{ matrix.os }}
2424
timeout-minutes: 46 # equal to max + 3*std over the last 600 successful runs
2525
env:
@@ -37,7 +37,7 @@ jobs:
3737
run: |
3838
find /opt/hostedtoolcache/* -maxdepth 0 ! -name 'Python' -exec rm -rf {} \;
3939
rm -rf /usr/share/dotnet/
40-
- uses: conda-incubator/setup-miniconda@v3
40+
- uses: conda-incubator/setup-miniconda@v4
4141
with:
4242
auto-update-conda: true
4343
python-version: ${{ matrix.python-version }}

‎.github/workflows/cron-ngc-bundle.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,10 +18,10 @@ jobs:
1818
runs-on: ubuntu-latest
1919
steps:
2020
- uses: actions/checkout@v6
21-
- name: Set up Python 3.9
21+
- name: Set up Python 3.10
2222
uses: actions/setup-python@v6
2323
with:
24-
python-version: '3.9'
24+
python-version: '3.10'
2525
- name: cache weekly timestamp
2626
id: pip-cache
2727
run: echo "datew=$(date '+%Y-%V')" >> $GITHUB_OUTPUT

0 commit comments

Comments
 (0)