Skip to content

Commit 1514741

Browse files
committed
refactor(deps): move per-Python lockfiles into lockfiles/pyXX/ subdirs
Each `lockfiles/pyXX/` directory is a separate Dependabot watchpoint, which lets the py39 entry carry a permanent `ignore: urllib3 >= 2.7.0` rule (urllib3 2.7.0 dropped Python 3.9) without holding back the other interpreters. py39 lockfile regenerated with urllib3 2.6.3. Windows lockfile follows the same scheme: `lockfiles/py313-windows/ requirements.txt`. All build/install scripts, GitHub workflows, container fixtures (cpu-usage, strongswan-connections, users) and docs (INSTALL.md, BUILD.md, CONTRIBUTING.md, tox.ini) updated to the new paths.
1 parent 689b62c commit 1514741

61 files changed

Lines changed: 148 additions & 81 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/dependabot.yml

Lines changed: 57 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,64 @@ updates:
88
time: '05:00'
99
timezone: 'Etc/UTC'
1010

11+
# One entry per Python LTS lockfile. Each lockfile lives in its own
12+
# directory under `lockfiles/` so Dependabot can apply per-version
13+
# ignore rules (e.g. urllib3 cannot move past 2.6.x on Python 3.9).
1114
- package-ecosystem: 'pip'
12-
directory: '/'
15+
directory: '/lockfiles/py39'
16+
schedule:
17+
interval: 'weekly'
18+
day: 'friday'
19+
time: '05:00'
20+
timezone: 'Etc/UTC'
21+
ignore:
22+
# urllib3 >= 2.7.0 requires Python 3.10+; the py39 lockfile is
23+
# for RHEL 8 / Debian 11 (Python 3.9) and must stay on 2.6.x.
24+
- dependency-name: 'urllib3'
25+
versions: ['>=2.7.0']
26+
27+
- package-ecosystem: 'pip'
28+
directory: '/lockfiles/py310'
29+
schedule:
30+
interval: 'weekly'
31+
day: 'friday'
32+
time: '05:00'
33+
timezone: 'Etc/UTC'
34+
35+
- package-ecosystem: 'pip'
36+
directory: '/lockfiles/py311'
37+
schedule:
38+
interval: 'weekly'
39+
day: 'friday'
40+
time: '05:00'
41+
timezone: 'Etc/UTC'
42+
43+
- package-ecosystem: 'pip'
44+
directory: '/lockfiles/py312'
45+
schedule:
46+
interval: 'weekly'
47+
day: 'friday'
48+
time: '05:00'
49+
timezone: 'Etc/UTC'
50+
51+
- package-ecosystem: 'pip'
52+
directory: '/lockfiles/py313'
53+
schedule:
54+
interval: 'weekly'
55+
day: 'friday'
56+
time: '05:00'
57+
timezone: 'Etc/UTC'
58+
59+
- package-ecosystem: 'pip'
60+
directory: '/lockfiles/py314'
61+
schedule:
62+
interval: 'weekly'
63+
day: 'friday'
64+
time: '05:00'
65+
timezone: 'Etc/UTC'
66+
67+
- package-ecosystem: 'pip'
68+
directory: '/lockfiles/py313-windows'
1369
schedule:
1470
interval: 'weekly'
1571
day: 'friday'

.github/workflows/lf-build-windows-x86_64.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -82,8 +82,8 @@ jobs:
8282
run: 'python.exe -m pip install --upgrade ordered-set Nuitka'
8383

8484
# install 3rd party libraries for all check plugins
85-
- name: 'python.exe -m pip install --requirement ${{ github.workspace }}\repos\monitoring-plugins\requirements-py313-windows.txt --require-hashes'
86-
run: 'python.exe -m pip install --requirement ${{ github.workspace }}\repos\monitoring-plugins\requirements-py313-windows.txt --require-hashes'
85+
- name: 'python.exe -m pip install --requirement ${{ github.workspace }}\repos\monitoring-plugins\lockfiles/py313-windows/requirements.txt --require-hashes'
86+
run: 'python.exe -m pip install --requirement ${{ github.workspace }}\repos\monitoring-plugins\lockfiles/py313-windows/requirements.txt --require-hashes'
8787

8888
- name: 'Verify Python installation'
8989
run: 'python.exe -m pip list'

BUILD.md

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -176,9 +176,10 @@ Prerequisites:
176176
* **Visual Studio 2022 Build Tools** (supplies MSVC, required by Nuitka
177177
`--msvc=latest`).
178178
* `python3 -m pip install --upgrade ordered-set Nuitka`.
179-
* `python3 -m pip install --require-hashes --requirement requirements-py313-windows.txt`
180-
from the monitoring-plugins repository root. The `pyXX` in the filename
181-
matches the Python version the Windows build is pinned to (3.13 today).
179+
* `python3 -m pip install --require-hashes --requirement lockfiles/py313-windows/requirements.txt`
180+
from the monitoring-plugins repository root. The `pyXX` directory under
181+
`lockfiles/` matches the Python version the Windows build is pinned to
182+
(3.13 today).
182183
* The plugin must carry a `.windows` marker file. Plugins without it are
183184
skipped by `compile-one.sh`.
184185

CHANGELOG.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,8 +21,8 @@ Monitoring Plugins:
2121

2222
Build, CI/CD:
2323

24-
* requirements: one hash-pinned lockfile per supported Python LTS (`requirements-py39.txt` to `-py314.txt`) replaces the single `requirements.txt`. Windows uses `requirements-py313-windows.txt`
25-
* requirements: build scripts auto-detect the Python version and pick the matching file. urllib3 lands at 2.7.0 on Python 3.10+, closing two of the four Dependabot advisories
24+
* requirements: one hash-pinned lockfile per supported Python LTS, each in its own `lockfiles/pyXX/` subdirectory (`py39` to `py314`). Replaces the single `requirements.txt`. Windows uses `lockfiles/py313-windows/requirements.txt`
25+
* requirements: build scripts auto-detect the Python version and pick the matching file. urllib3 lands at 2.7.0 on Python 3.10+, closing two of the four Dependabot advisories; the `py39` lockfile pins urllib3 to 2.6.x via Dependabot ignore (urllib3 2.7.0 requires Python 3.10+)
2626

2727

2828
### Changed

CONTRIBUTING.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -158,7 +158,7 @@ When creating a new plugin, make sure to deliver:
158158
* README file explaining "How?" and "Why?"
159159
* A free, monochrome, transparent SVG icon from <https://simpleicons.org> or <https://fontawesome.com/search?ic=free>, placed in the `icon` directory.
160160
* Optional: `unit-test/run` - the unittest file (see [Unit Tests](#unit-tests))
161-
* Optional: extend the repo-root `requirements.in` with new Python deps; the per-Python lockfiles `requirements-pyXX.txt` are regenerated from it
161+
* Optional: extend the repo-root `requirements.in` with new Python deps; the per-Python lockfiles under `lockfiles/pyXX/requirements.txt` are regenerated from it
162162
* If providing performance data: Grafana dashboard (see [GRAFANA.md](GRAFANA.md)) and `.ini` file for the Icinga Web 2 Grafana Module
163163
* Icinga Director Basket Config for the check plugin (`build-basket`)
164164
* Icinga Service Set in `all-the-rest.json` if appropriate (see [Service Set vs. Service Template](#service-set-vs-service-template))
@@ -1040,7 +1040,7 @@ The canonical distro matrix is the cpu-usage `CONTAINERFILES` list. **Where poss
10401040

10411041
Rules and tips:
10421042

1043-
* **Reuse cpu-usage's `containerfiles/`** as a starting point for a new plugin - the per-distro bootstrap (pacman / apt / dnf / zypper + venv + `pip install -r requirements-pyXX.txt --require-hashes`, where `pyXX` matches the distro's Python LTS) is identical, only the bind-mount path for the plugin script changes.
1043+
* **Reuse cpu-usage's `containerfiles/`** as a starting point for a new plugin - the per-distro bootstrap (pacman / apt / dnf / zypper + venv + `pip install -r lockfiles/pyXX/requirements.txt --require-hashes`, where `pyXX` matches the distro's Python LTS) is identical, only the bind-mount path for the plugin script changes.
10441044
* **`clean_up=False` on `DockerImage`**. Testcontainers' default cleans up the built image and prunes dangling parent layers on exit, which turns every run into a full rebuild. `clean_up=False` keeps the image around so subsequent runs hit podman's layer cache and finish in seconds.
10451045
* **`,Z` on bind mounts**. On SELinux-enforcing hosts (RHEL, Fedora, Rocky) unrelabelled bind mounts are denied by the container runtime. `mode='ro,Z'` relabels the source so the container can read it; without the `Z` flag the plugin inside the container sees "Permission denied" on `import lib`.
10461046
* **Rootless podman caveats** - same as for the service-container pattern: `TESTCONTAINERS_RYUK_DISABLED=true` must be set, `CONTAINER_HOST` / `DOCKER_HOST` must point at the rootless socket. `tools/run-unit-tests` does this automatically.

INSTALL.md

Lines changed: 10 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -125,16 +125,16 @@ and Icinga Director command definitions portable (see
125125

126126
Unlike the RPM and DEB packages (which ship a pre-built venv under
127127
`/usr/lib64/linuxfabrik-monitoring-plugins/venv/`), the source zip only carries source
128-
files. The repository ships one hash-pinned lockfile per supported Python LTS
129-
(`requirements-py39.txt` ... `requirements-py314.txt`). Pick the file that matches the
128+
files. The repository ships one hash-pinned lockfile per supported Python LTS under
129+
`lockfiles/pyXX/requirements.txt` (`py39` ... `py314`). Pick the file that matches the
130130
Python on the target host and run `pip` against it once, as the user that will run the
131131
plugins (`icinga` on RHEL, `nagios` on Debian/Ubuntu):
132132

133133
```bash
134134
PY_TAG="py$(python3 -c 'import sys; print(f"{sys.version_info.major}{sys.version_info.minor}")')"
135135
sudo -u icinga python3 -m pip install --user --upgrade pip
136136
sudo -u icinga python3 -m pip install --user \
137-
--requirement /usr/lib64/nagios/plugins/requirements-${PY_TAG}.txt --require-hashes
137+
--requirement /usr/lib64/nagios/plugins/lockfiles/${PY_TAG}/requirements.txt --require-hashes
138138
```
139139

140140

@@ -170,7 +170,7 @@ for dir in $(find "${plugin_source_dir}" -maxdepth 1 -type d); do
170170
"${dir}/${file}" \
171171
"${remote_user}@${remote_host}:${remote_target_dir}/${file}"
172172
done
173-
scp "${plugin_source_dir}/../requirements-py"*.txt "${remote_user}@${remote_host}:/tmp"
173+
scp -r "${plugin_source_dir}/../lockfiles" "${remote_user}@${remote_host}:/tmp"
174174
```
175175

176176
Once complete, the remote directory looks like this:
@@ -191,7 +191,7 @@ Install the Python dependencies for the user that runs the plugins (`icinga` on
191191
PY_TAG="py$(python3 -c 'import sys; print(f"{sys.version_info.major}{sys.version_info.minor}")')"
192192
sudo -u icinga python3 -m pip install --user --upgrade pip
193193
sudo -u icinga python3 -m pip install --user \
194-
--requirement /tmp/requirements-${PY_TAG}.txt --require-hashes
194+
--requirement /tmp/lockfiles/${PY_TAG}/requirements.txt --require-hashes
195195
```
196196

197197

@@ -273,16 +273,17 @@ installation is required.
273273
### From Source (Python 3.9+)
274274

275275
On Windows, running the `.py` files directly requires a local Python 3.13
276-
installation and the dependencies from `requirements-py313-windows.txt` (the lockfile
277-
matches the version the Windows binary build is pinned to; see `BUILD.md`). Clone the
278-
repository and point the Icinga 2 agent at the `.py` files:
276+
installation and the dependencies from `lockfiles/py313-windows/requirements.txt` (the
277+
lockfile matches the version the Windows binary build is pinned to; see `BUILD.md`).
278+
Clone the repository and point the Icinga 2 agent at the `.py` files:
279279

280280
```powershell
281281
git clone https://github.com/Linuxfabrik/monitoring-plugins.git `
282282
"C:\Program Files\ICINGA2\sbin\linuxfabrik"
283283
python -m pip install --upgrade pip
284284
python -m pip install --requirement `
285-
"C:\Program Files\ICINGA2\sbin\linuxfabrik\requirements-py313-windows.txt" --require-hashes
285+
"C:\Program Files\ICINGA2\sbin\linuxfabrik\lockfiles\py313-windows\requirements.txt" `
286+
--require-hashes
286287
```
287288

288289

build/compile-multiple.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ if ! uname -a | grep -q "_NT"; then
1717
# We are in a container.
1818
source /opt/venv/bin/activate
1919
PY_TAG="py$(python3 -c 'import sys; print(f"{sys.version_info.major}{sys.version_info.minor}")')"
20-
REQS="$REPO_DIR/monitoring-plugins/requirements-${PY_TAG}.txt"
20+
REQS="$REPO_DIR/monitoring-plugins/lockfiles/${PY_TAG}/requirements.txt"
2121
if [ ! -f "$REQS" ]; then
2222
echo "❌ No requirements file for Python ${PY_TAG} at $REQS" >&2
2323
exit 1

build/create-vendor-tarball.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ $LFMP_PYTHON -m pip download \
1717
# as a unpinned dependency (which is already downloaded in the previous step).
1818
# All dependencies including hashes are already listed in our requirements anyway.
1919
PY_TAG="py$($LFMP_PYTHON -c 'import sys; print(f"{sys.version_info.major}{sys.version_info.minor}")')"
20-
REQS="$LFMP_DIR_REPOS/monitoring-plugins/requirements-${PY_TAG}.txt"
20+
REQS="$LFMP_DIR_REPOS/monitoring-plugins/lockfiles/${PY_TAG}/requirements.txt"
2121
if [ ! -f "$REQS" ]; then
2222
echo "❌ No requirements file for Python ${PY_TAG} at $REQS" >&2
2323
exit 1

build/install-vendor.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ $LFMP_VENV_PIP install \
1818
# during vendor-install we are working *inside* the target venv.
1919
LFMP_VENV_PYTHON="$(dirname "$LFMP_VENV_PIP")/python"
2020
PY_TAG="py$($LFMP_VENV_PYTHON -c 'import sys; print(f"{sys.version_info.major}{sys.version_info.minor}")')"
21-
REQS="requirements-${PY_TAG}.txt"
21+
REQS="lockfiles/${PY_TAG}/requirements.txt"
2222
if [ ! -f "$REQS" ]; then
2323
echo "❌ No requirements file for Python ${PY_TAG} at $REQS" >&2
2424
exit 1

check-plugins/cpu-usage/unit-test/containerfiles/archlinux-vlatest

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ RUN pacman-key --init && \
2020
WORKDIR /tmp
2121

2222
# Use ADD to download the requirements.txt file
23-
ADD https://raw.githubusercontent.com/Linuxfabrik/monitoring-plugins/refs/heads/main/requirements.txt /tmp/requirements.txt
23+
ADD https://raw.githubusercontent.com/Linuxfabrik/monitoring-plugins/refs/heads/main/lockfiles/py313/requirements.txt /tmp/requirements.txt
2424

2525
# Set up a Python virtual environment and install dependencies
2626
RUN python -m venv /tmp/venv && \

0 commit comments

Comments
 (0)