Skip to content

Commit f5cb2a4

Browse files
markuslfmarkuslf
committed
test: switch all run files to lib.lftest.attach_tests for accurate test counts
Replace the legacy def test(self): for t in TESTS: with self.subTest(id=t['id']): lib.lftest.run(self, self.check, t) block in 120 plugin run files with a single lib.lftest.attach_tests(TestCheck, TESTS) call. The new helper materialises one real `test_<id>` method per testcase, so unittest reports the actual scenario count instead of collapsing the whole TESTS list into a single "Ran 1 test in ..." line. `./run -v` now lists every scenario by name and `./run`'s test count is accurate. CONTRIBUTING is updated to document the new pattern as the standard. Sixteen plugins are intentionally untouched because they already use a different test shape (apache-httpd-status with cache resets, the testcontainers-based cpu-usage and keycloak-* tests, csv-values which has no `--test` hook, the version checkers that call `lib.version.check_eol` directly, etc.). Those will be migrated case by case.
1 parent 06e368a commit f5cb2a4

File tree

122 files changed

+248
-490
lines changed
  • CONTRIBUTING.md
  • check-plugins
    • apache-httpd-version/unit-test
    • apache-solr-version/unit-test
    • atlassian-statuspage/unit-test
    • cometsystem/unit-test
    • composer-version/unit-test
    • deb-lastactivity/unit-test
    • deb-updates/unit-test
    • dhcp-scope-usage/unit-test
    • diacos/unit-test
    • disk-smart/unit-test
    • disk-usage/unit-test
    • dmesg/unit-test
    • docker-info/unit-test
    • docker-stats/unit-test
    • example/unit-test
    • fail2ban/unit-test
    • file-descriptors/unit-test
    • fs-ro/unit-test
    • githubstatus/unit-test
    • gitlab-health/unit-test
    • gitlab-liveness/unit-test
    • gitlab-readiness/unit-test
    • grafana-version/unit-test
    • grassfish-licenses/unit-test
    • grassfish-players/unit-test
    • grassfish-screens/unit-test
    • graylog-version/unit-test
    • haproxy-status/unit-test
    • hin-status/unit-test
    • huawei-dorado-backup-power/unit-test
    • huawei-dorado-controller/unit-test
    • huawei-dorado-disk/unit-test
    • huawei-dorado-enclosure/unit-test
    • huawei-dorado-fan/unit-test
    • huawei-dorado-host/unit-test
    • huawei-dorado-hypermetrodomain/unit-test
    • huawei-dorado-hypermetropair/unit-test
    • huawei-dorado-interface/unit-test
    • huawei-dorado-power/unit-test
    • huawei-dorado-system/unit-test
    • icinga-version/unit-test
    • infomaniak-events/unit-test
    • infomaniak-swiss-backup-devices/unit-test
    • infomaniak-swiss-backup-products/unit-test
    • ipmi-sel/unit-test
    • ipmi-sensor/unit-test
    • jitsi-videobridge-stats/unit-test
    • jitsi-videobridge-status/unit-test
    • journald-query/unit-test
    • journald-usage/unit-test
    • kemp-services/unit-test
    • kubectl-get-pods/unit-test
    • load/unit-test
    • mailq/unit-test
    • memory-usage/unit-test
    • mod-qos-stats/unit-test
    • mydumper-version/unit-test
    • mysql-version/unit-test
    • needs-restarting/unit-test
    • network-bonding/unit-test
    • network-connections/unit-test
    • nginx-status/unit-test
    • nodebb-cache/unit-test
    • nodebb-database/unit-test
    • nodebb-errors/unit-test
    • nodebb-events/unit-test
    • nodebb-groups/unit-test
    • nodebb-info/unit-test
    • nodebb-users/unit-test
    • nodebb-version/unit-test
    • ntp-chronyd/unit-test
    • ntp-ntpd/unit-test
    • ntp-systemd-timesyncd/unit-test
    • ntp-w32tm/unit-test
    • onlyoffice-stats/unit-test
    • openjdk-redhat-version/unit-test
    • openstack-swift-stat/unit-test
    • openvpn-client-list/unit-test
    • openvpn-version/unit-test
    • php-fpm-ping/unit-test
    • php-fpm-status/unit-test
    • pip-updates/unit-test
    • podman-info/unit-test
    • podman-stats/unit-test
    • postfix-version/unit-test
    • postgresql-version/unit-test
    • redfish-sel/unit-test
    • redis-status/unit-test
    • redis-version/unit-test
    • restic-check/unit-test
    • restic-snapshots/unit-test
    • restic-stats/unit-test
    • rpm-updates/unit-test
    • safenet-hsm-state/unit-test
    • sap-open-concur-com/unit-test
    • sensors-battery/unit-test
    • sensors-fans/unit-test
    • sensors-temperatures/unit-test
    • service/unit-test
    • snmp/unit-test
    • spring-boot-actuator-health/unit-test
    • starface-account-stats/unit-test
    • starface-backup-status/unit-test
    • starface-channel-status/unit-test
    • starface-database-stats/unit-test
    • starface-java-memory-usage/unit-test
    • starface-peer-stats/unit-test
    • starface-status/unit-test
    • statusiq/unit-test
    • statuspal/unit-test
    • strongswan-connections/unit-test
    • swap-usage/unit-test
    • systemd-timedate-status/unit-test
    • systemd-units-failed/unit-test
    • uptimerobot/unit-test
    • uptime/unit-test
    • users/unit-test
    • valkey-version/unit-test
    • veeam-status/unit-test
    • virustotal-scan-url/unit-test
    • whmcs-status/unit-test

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

122 files changed

+248
-490
lines changed

‎CONTRIBUTING.md‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -727,12 +727,12 @@ The expected state is encoded in the testcase `id` instead (see below).
727727

728728
#### Writing tests
729729

730-
Define a `TESTS` list and use `lib.lftest.run()` to execute each testcase. The testcase `id` should lead with the expected state (`ok-`, `warn-`, `crit-`, `unknown-`), followed by a short description of what is being verified. This is what appears in the subtest output and documents the intent of each case.
730+
Define a `TESTS` list and use `lib.lftest.attach_tests()` to materialise one real `unittest` test method per testcase. The testcase `id` should lead with the expected state (`ok-`, `warn-`, `crit-`, `unknown-`), followed by a short description of what is being verified. The id becomes the test method name, so it shows up in `./run -v` output and in the unittest test count.
731731

732732
```python
733733
#!/usr/bin/env python3
734734
import sys
735-
sys.path.append('..')
735+
sys.path.insert(0, '..')
736736

737737
import unittest
738738

@@ -777,16 +777,16 @@ class TestCheck(unittest.TestCase):
777777

778778
check = '../my-check'
779779

780-
def test(self):
781-
for t in TESTS:
782-
with self.subTest(id=t['id']):
783-
lib.lftest.run(self, self.check, t)
780+
781+
lib.lftest.attach_tests(TestCheck, TESTS)
784782

785783

786784
if __name__ == '__main__':
787785
unittest.main()
788786
```
789787

788+
The reason for `attach_tests()` over a plain `for` loop with `subTest()` is reporting accuracy. A plain `for ... subTest()` loop runs every testcase but unittest only counts the surrounding method, so `./run` reports `Ran 1 test in ...s` regardless of how many fixtures the file actually exercises. `attach_tests()` materialises one `test_<id>` method per entry in TESTS, so `./run` reports the real count and `./run -v` lists every scenario by name. Failures still surface in either pattern, but the count is misleading without the helper.
789+
790790
Naming rules for the testcase `id`:
791791

792792
* Lead with the expected state: `ok-`, `warn-`, `crit-`, `unknown-`.

‎check-plugins/apache-httpd-version/unit-test/run‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -77,10 +77,8 @@ class TestCheck(unittest.TestCase):
7777

7878
check = '../apache-httpd-version'
7979

80-
def test(self):
81-
for t in TESTS:
82-
with self.subTest(id=t['id']):
83-
lib.lftest.run(self, self.check, t)
80+
81+
lib.lftest.attach_tests(TestCheck, TESTS)
8482

8583

8684
if __name__ == '__main__':

‎check-plugins/apache-solr-version/unit-test/run‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -75,10 +75,8 @@ class TestCheck(unittest.TestCase):
7575

7676
check = '../apache-solr-version'
7777

78-
def test(self):
79-
for t in TESTS:
80-
with self.subTest(id=t['id']):
81-
lib.lftest.run(self, self.check, t)
78+
79+
lib.lftest.attach_tests(TestCheck, TESTS)
8280

8381

8482
if __name__ == '__main__':

‎check-plugins/atlassian-statuspage/unit-test/run‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -76,10 +76,8 @@ class TestCheck(unittest.TestCase):
7676

7777
check = '../atlassian-statuspage'
7878

79-
def test(self):
80-
for t in TESTS:
81-
with self.subTest(id=t['id']):
82-
lib.lftest.run(self, self.check, t)
79+
80+
lib.lftest.attach_tests(TestCheck, TESTS)
8381

8482

8583
if __name__ == '__main__':

‎check-plugins/cometsystem/unit-test/run‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -108,10 +108,8 @@ class TestCheck(unittest.TestCase):
108108

109109
check = '../cometsystem'
110110

111-
def test(self):
112-
for t in TESTS:
113-
with self.subTest(id=t['id']):
114-
lib.lftest.run(self, self.check, t)
111+
112+
lib.lftest.attach_tests(TestCheck, TESTS)
115113

116114

117115
if __name__ == '__main__':

‎check-plugins/composer-version/unit-test/run‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -73,10 +73,8 @@ class TestCheck(unittest.TestCase):
7373

7474
check = '../composer-version'
7575

76-
def test(self):
77-
for t in TESTS:
78-
with self.subTest(id=t['id']):
79-
lib.lftest.run(self, self.check, t)
76+
77+
lib.lftest.attach_tests(TestCheck, TESTS)
8078

8179

8280
if __name__ == '__main__':

‎check-plugins/deb-lastactivity/unit-test/run‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -40,10 +40,8 @@ class TestCheck(unittest.TestCase):
4040

4141
check = '../deb-lastactivity'
4242

43-
def test(self):
44-
for t in TESTS:
45-
with self.subTest(id=t['id']):
46-
lib.lftest.run(self, self.check, t)
43+
44+
lib.lftest.attach_tests(TestCheck, TESTS)
4745

4846

4947
if __name__ == '__main__':

‎check-plugins/deb-updates/unit-test/run‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -114,10 +114,8 @@ class TestCheck(unittest.TestCase):
114114

115115
check = '../deb-updates'
116116

117-
def test(self):
118-
for t in TESTS:
119-
with self.subTest(id=t['id']):
120-
lib.lftest.run(self, self.check, t)
117+
118+
lib.lftest.attach_tests(TestCheck, TESTS)
121119

122120

123121
if __name__ == '__main__':

‎check-plugins/dhcp-scope-usage/unit-test/run‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -54,10 +54,8 @@ class TestCheck(unittest.TestCase):
5454

5555
check = '../dhcp-scope-usage'
5656

57-
def test(self):
58-
for t in TESTS:
59-
with self.subTest(id=t['id']):
60-
lib.lftest.run(self, self.check, t)
57+
58+
lib.lftest.attach_tests(TestCheck, TESTS)
6159

6260

6361
if __name__ == '__main__':

‎check-plugins/diacos/unit-test/run‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -36,10 +36,8 @@ class TestCheck(unittest.TestCase):
3636

3737
check = '../diacos'
3838

39-
def test(self):
40-
for t in TESTS:
41-
with self.subTest(id=t['id']):
42-
lib.lftest.run(self, self.check, t)
39+
40+
lib.lftest.attach_tests(TestCheck, TESTS)
4341

4442

4543
if __name__ == '__main__':

0 commit comments

Comments
 (0)