Skip to content

Commit b89a1e7

Browse files
pllimpllim
authored
Refactor integration testing system to build a dashboard and run in CI (#44)
* Refactor integration testing system to build a dashboard and run in CI * Add ability to preview page in PRs * Fixes to CI * Simplify CI setup * Fix CI issues * Simplify dashboard set-up * Improvements, including making it possible to run multiple Python versions * Remove old infrastructure * Add affiliated packages and make improvements to infrastructure * Fix more issues and only run first 10 tests in PRs * Tidy up into package * Improve uv cache handling * Cleanup and added pre-commit * More cleanup * Apply suggestions from code review Co-authored-by: P. L. Lim <2090236+pllim@users.noreply.github.com> * Add pyopensci package tier and tidy workflow and docs Add seven pyOpenSci-reviewed affiliated packages (astrodata, astromartini, autogalaxy, GALFITools, petrofit, stingray, zodipy) under a new pyopensci tier, ranked after affiliated in install and dashboard order. Set an explicit pull-requests: none workflow permission, document why setup-python uses a fixed version, and name the CI jobs explicitly in the README. * Pin workflow actions to commit SHAs and add Dependabot cooldown Pin every action in the integration and preview-link workflows to a full release commit SHA with a version comment. Add a 7-day cooldown to the github-actions Dependabot entry so a release has time to settle before a bump is proposed. --------- Co-authored-by: P. L. Lim <2090236+pllim@users.noreply.github.com>
2 parents 2185ff3 + 56e38e9 commit b89a1e7

19 files changed

Lines changed: 1705 additions & 130 deletions

‎.github/dependabot.yml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,9 @@ updates:
99
directory: ".github/workflows" # Location of package manifests
1010
schedule:
1111
interval: "weekly"
12+
cooldown:
13+
# Let a release settle before Dependabot proposes bumping to it.
14+
default-days: 7
1215
groups:
1316
actions:
1417
patterns:

‎.github/workflows/integration.yml‎

Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
name: integration-matrix
2+
3+
on:
4+
schedule:
5+
- cron: '0 6 * * 0' # Sundays at 06:00 UTC
6+
workflow_dispatch:
7+
pull_request:
8+
9+
permissions:
10+
contents: write # to push gh-pages on main / schedule / dispatch
11+
pull-requests: none # explicit: this workflow never touches PRs
12+
13+
concurrency:
14+
# One in-flight run per PR, one per main/dispatch invocation.
15+
# PR runs cancel previous in-progress ones; main runs queue.
16+
group: integration-${{ github.event.pull_request.number || github.ref }}
17+
cancel-in-progress: ${{ github.event_name == 'pull_request' }}
18+
19+
jobs:
20+
variant:
21+
strategy:
22+
fail-fast: false
23+
matrix:
24+
# Keep in sync with `python_versions` in packages.yaml.
25+
# (free-threaded builds: use the "t" suffix, e.g. "3.14t".)
26+
python: ["3.12", "3.14"]
27+
variant: [stable, pre, dev]
28+
runs-on: ubuntu-latest
29+
timeout-minutes: 240
30+
steps:
31+
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
32+
33+
# This Python only runs the `astropy-integration` orchestrator CLI;
34+
# the test venv for `matrix.python` is built separately by uv (see
35+
# the `run` step below), so a fixed version is fine here.
36+
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
37+
with:
38+
python-version: '3.12'
39+
40+
- name: Install uv
41+
uses: astral-sh/setup-uv@caf0cab7a618c569241d31dcd442f54681755d39 # v3.2.4
42+
with:
43+
# We manage the cache ourselves via actions/cache below; turning
44+
# off setup-uv's own caching avoids it saving a duplicate of the
45+
# same directory under its own key.
46+
enable-cache: false
47+
48+
- name: Cache ~/.cache/uv
49+
uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0
50+
with:
51+
path: ~/.cache/uv
52+
# Primary key uses the run id, so every run writes a fresh
53+
# snapshot (after `uv cache prune --ci` slims it to wheels
54+
# actually used in this run). The restore-keys are prefix
55+
# matches against saved keys, so the first one always finds
56+
# the most recent run's cache for the same (os, variant,
57+
# python) and the next run starts warm.
58+
key: uv-${{ runner.os }}-${{ matrix.variant }}-${{ matrix.python }}-${{ github.run_id }}
59+
restore-keys: |
60+
uv-${{ runner.os }}-${{ matrix.variant }}-${{ matrix.python }}-
61+
uv-${{ runner.os }}-${{ matrix.variant }}-
62+
uv-${{ runner.os }}-
63+
64+
- name: Install astropy-integration
65+
run: pip install .
66+
67+
- name: Run ${{ matrix.variant }} variant on Python ${{ matrix.python }}
68+
env:
69+
# PR previews limit each package to the first 10 tests so the
70+
# matrix finishes in minutes; conftest.py at the repo root reads
71+
# this env var and truncates the collected items.
72+
PYTEST_LIMIT_N: ${{ github.event_name == 'pull_request' && '10' || '' }}
73+
run: astropy-integration run --variant ${{ matrix.variant }} --python ${{ matrix.python }}
74+
75+
- name: Prune uv cache before save
76+
if: always()
77+
# Strips pre-built sdists and other entries uv won't reuse, so
78+
# the snapshot the cache step uploads stays lean.
79+
run: uv cache prune --ci
80+
81+
- name: Upload results
82+
if: always()
83+
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
84+
with:
85+
name: results-${{ matrix.variant }}-${{ matrix.python }}
86+
path: results/${{ matrix.variant }}__${{ matrix.python }}.json
87+
88+
dashboard:
89+
needs: variant
90+
if: always()
91+
runs-on: ubuntu-latest
92+
timeout-minutes: 15
93+
steps:
94+
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
95+
96+
- uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
97+
with:
98+
python-version: '3.12'
99+
100+
- name: Install astropy-integration
101+
run: pip install .
102+
103+
- name: Download all variant results
104+
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
105+
with:
106+
pattern: results-*
107+
merge-multiple: true
108+
path: results/
109+
110+
- name: Build dashboard
111+
run: astropy-integration dashboard
112+
113+
# PR path: upload index.html as a non-zipped artifact for the
114+
# in-browser preview (linked from a PR check by preview-link.yml).
115+
- name: Upload preview artifact
116+
if: github.event_name == 'pull_request'
117+
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
118+
with:
119+
name: dashboard-preview
120+
path: site/index.html
121+
archive: false
122+
overwrite: true
123+
124+
# Main / schedule / dispatch path: publish to gh-pages.
125+
- name: Publish to gh-pages
126+
if: github.event_name != 'pull_request'
127+
uses: peaceiris/actions-gh-pages@4f9cc6602d3f66b9c108549d475ec49e8ef4d45e # v4.0.0
128+
with:
129+
github_token: ${{ secrets.GITHUB_TOKEN }}
130+
publish_dir: ./site
131+
force_orphan: true
132+
commit_message: Update integration dashboard

‎.github/workflows/integration_testing.yml‎

Lines changed: 0 additions & 39 deletions
This file was deleted.

‎.github/workflows/preview-link.yml‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
name: preview-link
2+
3+
# Companion to `integration-matrix`. When that workflow finishes on
4+
# a pull request, this one attaches a status check to the source
5+
# commit whose "Details" link goes directly to the dashboard-preview
6+
# artifact (rendered HTML), skipping the artifact summary page.
7+
#
8+
# Must live on the default branch to take effect.
9+
10+
on:
11+
workflow_run:
12+
workflows: ["integration-matrix"]
13+
types: [requested, in_progress, completed]
14+
15+
permissions: {}
16+
17+
jobs:
18+
redirect:
19+
runs-on: ubuntu-latest
20+
permissions:
21+
statuses: write # to attach a status check to the source commit
22+
actions: read # to look up the source workflow's artifact id
23+
steps:
24+
- uses: agriyakhetarpal/github-actions-artifacts-redirector-action@683d25ace2cb0aefe8e6719c39c2ac7f3d22dd8c # v1.0.0
25+
with:
26+
repo-token: ${{ secrets.GITHUB_TOKEN }}
27+
# With `archive: false`, artifact-name is the uploaded file's
28+
# basename without extension (we upload site/index.html).
29+
artifact-name: index
30+
job-title: View dashboard preview

‎.gitignore‎

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
1-
.tox
2-
.hypothesis
3-
result_images
1+
.tmp/
2+
results/
3+
site/
4+
__pycache__/
5+
*.pyc
6+
*.egg-info/
7+
build/

‎.pre-commit-config.yaml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
repos:
2+
- repo: https://github.com/pre-commit/pre-commit-hooks
3+
rev: v6.0.0
4+
hooks:
5+
- id: check-yaml
6+
- id: end-of-file-fixer
7+
- id: trailing-whitespace
8+
9+
- repo: https://github.com/astral-sh/ruff-pre-commit
10+
rev: v0.15.12
11+
hooks:
12+
- id: ruff-check
13+
args: [--fix]
14+
- id: ruff-format

0 commit comments

Comments
 (0)