Add CI/CD pipeline with container-based testing (Feature Request ID 9393 in Mantis)#820
Conversation
|
Added Note to https://gramps-project.org/bugs/view.php?id=9393 requesting testing and feedback of this PR. |
|
Corrected to conform to the agents.md guidelines, including using unittest instead of pytest |
|
Can you provide the commands to test the CI workflow. And a usage of the test harness. |
|
How did the code work without the imports you have added |
|
As per comment on another PR, the CLAUDE.md file should not be here as it just duplicates Agents.md in the main Gramps repository meaning any changes etc. will have to be made in two place and it will be hard to keep the files in step. |
|
Thanks for fixing so many bugs and problems, but there are too many completely different changes in a single commit. Should they not be made in several commits(or even separate PRs). |
|
Yeah, I wanted more at one go, I can see that it might not be the right approach. @kulath , there are many Lint issues and I chose to fix them them instead of dropping the Lint. I thought this might get some pushback. We can do it the other way around - I'll remove the Linting and we can make issues out of them as preperation for activating them. |
517129d to
b677454
Compare
Introduce GitHub Actions CI inside a shared Docker image on ghcr.io, plus a native Windows runner for cross-platform unit-test coverage, and a shared unittest harness with Gramps-backed fixtures that verifies every addon registers, loads, and exposes valid plugin metadata. CI infrastructure ----------------- - .github/docker/gramps-ci/Dockerfile — Python 3.12 + Gramps 6.0 (pip) + PyGObject + GTK typelibs + xvfb/xauth + ruff, dbf, intltool, gettext, git. GTK lives in the base so addon modules that do `from gi.repository import Gtk` at load time are importable; xvfb and xauth are bundled for tests that actually render. - .github/workflows/docker-build.yml — rebuilds the image on .github/docker/** changes or via workflow_dispatch. - .github/workflows/ci.yml — seven jobs: lint (ruff E9/F63/F7/F82 + trailing whitespace), addon-structure (every addon has po/template.pot), compile-check (py_compile on every .py), unit-test-linux (container), unit-test-windows (native, conda+pip), integration-test (container with --init so xvfb-run does not hang), build (make.py gramps60 build all). - .github/environment.yml — hybrid conda+pip env for Windows. Gramps is not on conda-forge, so pygobject/gtk3 come from conda and gramps/orjson/dbf come from pip. Shared test harness ------------------- - tests/__init__.py — GPL header. - tests/gramps_test_env.py — sys.path / GRAMPS_RESOURCES bootstrap and two unittest base classes: GrampsTestCase (session-cached plugin manager + registry via setUpClass) and GrampsDbTestCase (same plus a fresh in-memory SQLite DB per test). - tests/test_plugin_registration.py — four unittest.TestCase classes covering plugin registration, subprocess-isolated module loading (crash-safe), required metadata (gramps_target_version=6.0, valid id/name/version), and import/export entry-function smoke tests. Gate policy ----------- All seven jobs run on every push and PR. Four are marked continue-on-error: true so they surface issues without blocking merges while the existing tree is cleaned up: - lint (~79 pre-existing ruff E9/F63/F7/F82 errors) - addon-structure (4 addons missing po/template.pot) - unit-test-linux (some addon test modules fail to import today) - unit-test-windows (same) compile-check, integration-test, and build are blocking from day one. Each non-blocking gate will be flipped to blocking in the same follow-up PR that fixes its underlying issues, so the tightening is incremental and visible in history. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
b677454 to
774a9ac
Compare
|
I've significantly reduced the scope of this PR based on @kulath's feedback about "too many completely different changes." It's now strictly CI infrastructure — 7 files, one commit. What's in the PR
Everything else from the earlier revision (lint fixes across ~29 addons, po/template.pot stubs for the four addons missing them, TMGimporter tests, the SurnameMappingGramplet.grp.py → .gpr.py rename, CLAUDE.md) has been removed and will be submitted as separate PRs. Gate Policy All seven jobs run on every push and PR. To avoid showering innocent contributors with red CI for pre-existing tree state, four jobs are marked continue-on-error: true so they surface issues without blocking merges: lint — non-blocking (~79 pre-existing ruff E9/F63/F7/F82 errors) In order to activate all gates, there will be some rework of existing addons needed |
|
@GaryGriffin — here are the commands for each CI job, lifted straight from ci.yml. Run from a checkout of addons-source with Gramps importable (either PYTHONPATH / GRAMPS_RESOURCES pointing at a Gramps checkout, or inside the CI container image). Lint — syntax/import errors plus trailing whitespace ruff check --select=E9,F63,F7,F82 --no-fix --exclude='.gpr.py' . Addon structure — every addon must have po/template.pot for gpr in /.gpr.py; do Compile check — py_compile on every .py find . -name '.py' ! -name '.gpr.py' ! -path './.git/' ! -path '/pycache/*' Per-addon unit tests — matches both the Linux and Windows jobs export PYTHONPATH=. Integration tests — plugin registration plus per-addon integration export PYTHONPATH=. per-addon integration uses the same loop as the unit block above |
Sorry, what I meant was, is there a way to invoke the CI process (not the commands that are invoked by the CI process) manually to test the github actions. I did try the invoked commands: Lint: I dont have ruff on my Mac. How would I invoke the CI lint process within github as an action. Addon Structure: if I manually invoke the commands on my Mac that is in the comment, it fails (due to escaping special chars needed for comments). I need to use:
Results: 4 issues, as you stated. I do not know if it is a requirement that the template.pot exists. If there are no translated strings, then there shouldn't need to be one, I think. Maybe I am wrong. Compile Check: same issue with escaping special chars. I think the gpr.py should actually be included. I need to use:
Results: 4 issues - Themes, Query, HouseTimelineGramplet, and lxml Integration Tests: failed with error |
|
Can you tell me what you are trying to do or understand? I'm not sure if I'm giving you the right answers. The tests run in containers stored on ghcr.io containing all the necessary tools. It's optimized for steady-state but is kind of a pain for this specific PR, because you can't really test it in this environment. The upside is that the image is stable across hundreds of normal PRs. It only churns when someone explicitly wants the CI environment to change.
You will have to make the images publically available for the PRs to work, but I don't think that's an issue. There will be some maintainance work needed to be done once you flip over to branch61 as default, I'd see what can be done to make it a bit easier then, assuming you like the current process. In order for you to verify how it works, take a poke at my forked repo where I've merged the PR. This is how you can do it
|
I should have been more explicit. I am trying to test the implementation before merging/publishing this PR. Understanding the resources needed and the frequency of activity is part of that. And making sure that it functions completely and as expected. I was hoping for something like a github command to activate the action so I can see it work. Given your complete description (much appreciated), I think I am going to have to yield to @Nick-Hall to review/merge/publish this one since it impacts the repo actions in ways far above my knowledge of github. For instance, I dont know if these are the right blocking/non-blocking decisions. Or if the scope should just be the Addon impacted by the PR rather than all Addons. |
|
@GaryGriffin - I thought of adding a manual button for everything, but that only works if it's merged with the default branch, lol |
The Dockerfile bakes in only `dbf`, but addons declare a wider set of Python deps in their .gpr.py `requires_mod` lists (networkx, psycopg2, pygraphviz, lxml, svgwrite, boto3, litellm, life_line_chart, psycopg). Without these installed, per-addon unit tests and the plugin- registration subprocess load fail with ImportError/NameError. Add a pre-test step to unit-test-linux, unit-test-windows, and integration-test that globs every *.gpr.py, extracts the requires_mod union via ast.literal_eval, and pip-installs each package one at a time. Per-package install (not batched) keeps a single build failure (pygraphviz without graphviz-dev, psycopg2 without libpq-dev) from aborting the rest — the affected addon's tests will skip or fail in isolation without blocking others. Mirrors Gramps' Addon Manager install path (gramps/gui/plug/_windows.py __on_install_clicked → req.install → gen/utils/requirements.py), keeping .gpr.py files as the single source of truth for addon deps. New addon deps do not need a parallel update to the Dockerfile or this workflow.
With ci.yml's auto-derive step in place (previous commit), dbf is installed at CI runtime from TMGimporter's .gpr.py requires_mod list. Keeping it baked into the Dockerfile and environment.yml in parallel would defeat the "single source of truth = .gpr.py" goal and drift the moment a new addon declares an additional dep. Remove dbf from both; leave the stable base (PyGObject, pycairo, Gramps, orjson, ruff) since those are not addon deps. Add a comment pointing readers at the auto-derive step so future edits do not re-bake runtime deps back in.
Root cause of "Unit Tests (Linux)" and "Integration Tests (Gramps)"
failures was not broken test modules — the steps never invoked
unittest. The container's default shell is /bin/sh (dash on
python:3.12-slim), and the inline scripts use bash-only parameter
expansions (${f%.py}, ${mod//\//.}) to build the dotted module list.
Dash fails with "Bad substitution" on the first such line; the rest
of the script never runs. continue-on-error: true masked this as a
generic job failure for two CI rounds.
Add "shell: bash" explicitly to:
- unit-test-linux / Run per-addon unit tests (bashisms)
- integration-test / Run per-addon integration tests (bashisms)
- integration-test / Run plugin registration tests (no bashisms today,
but consistent and future-proof)
Compile Check already sets shell: bash. Windows jobs inherit bash
via defaults.run at the job level. No other steps affected.
The Windows unit-test job hung on TMGimporter's DB-backed tests because
make_database("sqlite").load(":memory:", None) deadlocks under the
conda-forge GTK + pip Gramps combination. Rather than patch the hang,
introduce a filename convention so per-addon authors can declare OS
scope up front:
test_*.py general (every OS)
test_linux_*.py Linux-only
test_windows_*.py Windows-only
test_integration_*.py Linux-only, full-pipeline/DB-backed (pre-existing)
unit-test-linux skips test_windows_* and test_integration_*;
unit-test-windows skips test_linux_* and test_integration_*.
Applied to TMGimporter: the 13 DB-backed classes in tests/test_libtmg.py
move to tests/test_linux_libtmg.py (along with the _Rec/_table/_make_db/
_add_person/_MockUser helpers they use). The 7 pure-logic classes
(TestStripTmgCodes, TestTmgDateToGrampsDate, TestNumTo{Month,Date},
TestParseDate, TestRepoTypeFromName, TestUrlFromName) stay in
test_libtmg.py and will run on every OS.
Locally all 175 tests still pass via run-addon-unit.sh TMGimporter.
|
I just reworked the CI approach a bit to be able to differentiate between Base, Linux & Windows test specifically. Also added automated dependency loading for future unit tests. The Unit Tests currently fail because of an issue in Websearch, but that needs to be adressed seperately. |
The addon CI workflow inlined the same requires_mod derivation as an identical Python heredoc in three jobs (unit-test-linux, unit-test-windows, integration-test) and the is_active() bash filter verbatim across six job steps, with the find_spec name-gate triplicated alongside. A one-line change to any of them meant a three- to six-site edit, and the copies could silently diverge. Move the requires_mod derivation and the find_spec validator into a single .github/scripts/addon_python_deps.py that every job calls (--install-list / --check-resolves), and the is_active() filter into .github/scripts/active_addons.sh that each filtering step sources. The .gpr.py files stay the single source of truth, so the derived module list and the active-addon set are unchanged. The module also centralises the import-to-distribution install map (PIL -> Pillow) on the install side only; the find_spec gate keeps validating the raw declared import name, exactly as Gramps does at runtime via check_mod(). No requires_mod=["PIL"] exists in the tree today, so the derived install list is byte-identical to the old heredoc output and the change is behaviour-preserving. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Note that if/when the mini-installer from gramps-project/gramps#2308 is merged, that it uses slightly different names (pip names rather than import names). |
|
I plan to review #2308 next and it will probably be merged. |
…#2308
The CI install step maps requires_mod import names to PyPI distribution names
(e.g. PIL→Pillow). gramps PR #2308 ("PyPI wheel installer for addon module
dependencies") adds the authoritative table _IMPORT_TO_PYPI in
gramps/gen/utils/pypi.py with ~12 entries; this copy had one. Once #2308 merges,
an addon declaring e.g. cv2 / yaml / sklearn in requires_mod installs correctly
in Gramps but this CI's install union would pip install the bare import name and
fail the addon-unit step.
Mirror the 11 missing entries now (all current, correct PyPI names — valid
independent of #2308) so CI installs the same distribution Gramps does. Closes
the drift flagged on PR 820 (upstream PR 2308). No live addon declares an
at-risk name today, so this is pre-emptive. When #2308 merges, single-source the
table from gramps' module instead of hand-mirroring it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
@dsblank - I've added some stuff on that account to PR 820 now |
The unit/integration jobs discovered tests with a one-level glob (`*/tests/test_*.py`) and the runner loaded them by dotted name from the repo root, never putting the addon's own dir on sys.path. So a nested-package addon's suites (tests under `<Addon>/tests/<subpkg>/`) were not found, and addon code importing a top-level package (`from name_processor… import`, WebSearch's `from models import`) could not resolve. - ci.yml: enable `shopt -s globstar` and recurse the three discovery loops (`*/tests/**/test_*.py`, and `**/test_integration*.py`). Existing guards (is_active, the basename filters, the Sqlite skip, the dotted conversion) are unaffected. - run_addon_tests.py: in the worker, APPEND the addon's own directory to sys.path before loading — mirroring Gramps' plugin loader (gramps/gen/plug/_manager.py). Append (not prepend) so the repo-root shared `tests` Gramps-emulation env still wins; the module is still loaded by full dotted name from the repo root, so package-relative imports keep working. `--root` is threaded into the worker. Strict superset of today's behaviour: flat addons unchanged. Validated: the full NameSuite nested suite (upstream PR 941) — 20 modules, 234 tests — discovers, loads, and passes as-is; WebSearch's `from models import` now resolves (its remaining Gtk/Gdk-version error is the local GTK4 env / the separate #38 Gdk-pin gap, not this change). Out of scope: dsblank's `test/`+`*_test.py`+pytest convention is intentionally not matched (tracked separately). tests/test_run_addon_tests_paths.py covers a nested Model-B addon (top- level lib import + shared-env-not-shadowed) and a flat Model-B addon. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
How far away are we from having even a basic CI test framework ready? |
|
Noting that gramps-project/gramps#2308 has been merged. |
Resolves add/add conflicts on the four shared test files by adopting the upstream versions (tests/__init__.py, tests/gramps_test_env.py, tests/test_plugin_load_gate.py, tests/test_plugin_registration.py): upstream's hardened suite already contains this branch's changes (the load-failure gate, the VERSION_TUPLE-derived target version, and the include_in_listing filter) plus later adversarial-review follow-ups. The PR-specific test files (test_addon_system_deps.py, test_requires_mod_dedup.py, test_run_addon_tests_paths.py) are kept — they cover CI-pipeline concerns upstream's suite does not.
The merge of upstream maintenance/gramps60 brought in two requires_mod declarations the dependency drift gate had never seen: PIL (EditExifMetadata) and pymongo (MongoDB). Neither was classified in WHEEL_ONLY_MODS or MOD_BUILD_PACKAGES, so addon_system_deps.py --unmapped exited 1, failing the 'Validate addon system deps are mapped' CI step and three tests in tests/test_addon_system_deps.py. Both ship binary wheels on every CI platform and need no system package, so they belong in WHEEL_ONLY_MODS. PIL is declared by import name per the requires_mod contract; the install side already maps PIL -> Pillow via addon_python_deps.py.
…s PR #2308)
gramps 6.1 now ships the authority this machinery was hand-mirroring:
gramps/gen/utils/pypi.py (gramps PR #2308, 7f94428b13 — not backported
to 6.0) with the import->distribution table _IMPORT_TO_PYPI, and a
Requirements.check_mod that really imports the module after find_spec.
Wire addon_python_deps.py to both, per lane, at lookup time:
- _distribution_map() prefers the installed gramps' _IMPORT_TO_PYPI so
6.1+ lanes install exactly what Gramps' own installer would, new
upstream entries included; the local _IMPORT_TO_DISTRIBUTION becomes
a fallback mirror for lanes where gramps is absent or predates 6.1
(the gramps60 image, the conda-forge 6.0.x Windows lane, bare
runners). The table dict is read directly rather than via
resolve_pypi_name(), whose per-name LOG.warning advises declaring
the PyPI name — the opposite of the import-name contract the gate
enforces.
- --check-resolves delegates to the installed gramps'
Requirements().check_mod, so the gate matches whichever series the
lane ships: find_spec-only on 6.0, find_spec plus a real import on
6.1+. Stdlib find_spec remains the fallback where gramps is not
importable (never a CI lane).
- Fix a latent probe bug the PIL declaration makes live: installed-ness
was probed with 'pip show <raw import name>', but pip only knows the
distribution name ('pip show PIL' fails with Pillow installed), so
the one name the mapping machinery exists for was silently skipped,
never validated. Probe by the mapped distribution name; keep judging
by the raw import name (the install-only-map invariant).
All gramps imports are lazy and guarded with (Exception, SystemExit) —
a half-installed gramps raises SystemExit from ResourcePath at import,
not ImportError — so the module stays importable pure-stdlib.
tests/test_addon_python_deps.py pins the two seams and the probe fix
hermetically (fake gramps module trees via mock.patch.dict), plus a
sync-guard asserting mirror == authority wherever gramps >= 6.1 is
importable (the gramps61 lanes; skips elsewhere, where the mirror is
inert). ci.yml changes are comment-only.
|
@dsblank — PR #2308 is now accounted for (commits 427f8c3 + ad98e1d):
One small upstream observation from wiring this up: |
…to the raw name
Two gaps the adversarial review found in the test module added for gramps
PR #2308:
1. The {"gramps": None} overlay does not force ImportError when a gramps
submodule is already cached — `from gramps.gen.utils.requirements import
Requirements` short-circuits on the cached submodule and never consults
the None-ed parent. unittest discovery imports test_plugin_registration,
which caches gramps.gen.utils.requirements at module level, so on the
integration lane test_stdlib_fallback took the DELEGATED path and
false-red'd (verified: label 'gramps Requirements().check_mod' instead of
'stdlib find_spec'). Replace the constant with _no_gramps_overlay(), which
also None-outs every already-cached gramps.* key. Both fallback tests now
seed a cached fake gramps tree first, then overlay — so they prove the
overlay defeats a pre-cached submodule rather than merely an absent gramps.
2. CheckResolvesGate's stub checkers were constant lambdas, so a mutant that
judged the mapped DISTRIBUTION name (check(dist)) instead of the raw import
name survived — in production that would make gramps 6.1's
check_mod("Pillow") fail a correct requires_mod=["PIL"]. The checkers now
record their argument and assert they were asked about "PIL".
Also corrects the sync-guard docstring: the guard is latent until these
workflows land on maintenance/gramps61 (no current lane imports gramps >= 6.1),
not something 'those lanes catch' today.
Two adversarial-review findings in the dedup test:
1. test_no_requires_mod_heredoc_remains was a tautology. Its guard,
re.findall(r"requires_mod\s*=\s*\(\[", text), can never match the old
heredoc's own source line
pat = re.compile(r"requires_mod\s*=\s*(\[[^\]]*\])")
because that line contains the LITERAL characters \s* while the guard's
\s* matches whitespace. Pasting the removed heredoc back into ci.yml kept
the test green. Replace it with assertNotIn of the heredoc's literal
fragments (re.compile(r"requires_mod and requires_mod\s*), both of which
appear verbatim in the pre-dedup ci.yml (6x) and nowhere in the current
file — so a revert now reds.
2. test_install_list_matches_old_heredoc's oracle hardcodes _INSTALL_MAP=
{PIL: Pillow}. Since #2308, production install_list() consults gramps'
authoritative _IMPORT_TO_PYPI when a gramps >= 6.1 is importable, so on a
6.1 lane an upstream table entry for any declared mod would flip this
oracle red for a non-regression. Pin _distribution_map to the local mirror
inside the comparison; comment the two-step fix (GrampsTableSync reds ->
re-sync mirror -> extend _INSTALL_MAP).
…thon3-dev
The build-toolchain assertions were line-based substring checks and let three
regressions through (adversarial-review mutants, all verified surviving):
- M4c: a multi-line `RUN apt-get purge -y \` with gcc/python3-dev on the
continuation lines — invisible to a per-physical-line scan.
- M4d: `apt-get -y purge gcc` (flag before the verb) — the old
'apt-get purge' substring test missed it.
- M4e: dropping python3-dev from the install — test_toolchain_is_installed
only checked gcc and pkg-config.
Join backslash continuations into logical lines first; match any apt-get
removal verb (purge/remove/autoremove) in any flag order via regex; require
each of gcc, pkg-config AND python3-dev to appear as a whole token on an
actual `apt-get install` line (not merely somewhere in the file — a comment
or a purge line must not satisfy it). The real Dockerfile still passes.
Three security/robustness fixes from the adversarial review: - Add top-level 'permissions: contents: read' to ci.yml. The workflow runs addon-influenced code (pip package hooks from requires_mod; addon import-time code in the load/test lanes) and had no permissions block, so on push runs that code inherited the ambient default token scope. Pulling the public GHCR image needs no packages scope. (docker-build.yml already scopes its token.) - Add the '--unmapped' drift gate to unit-test-windows, before its pip step. The gate previously existed only on unit-test-linux; unit-test-windows 'needs: setup' alone (not unit-test-linux), so a novel requires_mod name reached 'pip install' unguarded on the Windows lane — arbitrary package execution — even though Linux blocked it. Both lanes now gate independently. - Pin ruff==0.15.22 in the CI image. Unpinned, a ruff release could flip the lint verdict (new rule in the E9/F63/F7/F82 selection) on any rebuild with no repo change; the pin makes the gate reproducible. Bump deliberately. Dedup pins (exactly-3 --install-list/--check-resolves, is_active sourcing) unaffected: the new step calls addon_system_deps.py, and permissions is not a job step.
…stalled
The dep-resolution gate skipped ('~') every requires_mod whose distribution
pip never installed, treating install failure as an environment gap. For
source-built modules (pygraphviz, psycopg2 — need a system -dev package) that
is right. But a wheel-only module (WHEEL_ONLY_MODS: PIL, boto3, litellm,
networkx, pymongo, svgwrite, dbf, life_line_chart) ships a pure/binary wheel
that installs on every CI platform, so a miss there is a real provisioning
regression that the previous 'skip' let scroll by green — the fail-open hole
the review flagged for the addons that have no tests of their own.
check_resolves now splits the not-installed branch on the classification set
(imported from the sibling addon_system_deps): wheel-only misses FAIL with a
distinct ::error::, source-built misses stay advisory skips. Tests exercise
both against the real WHEEL_ONLY_MODS (not mocked), and assert a never-
installed wheel is never even handed to the dep checker.
…xonomy, zero-test gate Four adversarial-review findings in the per-addon test runner: F1 — the per-module timeout was defeated by a grandchild. proc.kill() reaped only the worker; the follow-up proc.communicate() then blocked until any process that inherited the stdout pipe exited, so a test that spawned a long-lived child hung the run to the job cap (verified: a 2s timeout took 16s). The worker now runs in its own process group (start_new_session, POSIX) and a timeout os.killpg's the group, with a bounded follow-up communicate(). A new posix-only test spawns a sleep-120 child inheriting stdout and asserts the run returns in < 45s. F2/F3 — the load-failure taxonomy was inverted AND never fired. A module-level ImportError (the common missing-dep shape) was not raised by loadTestsFromName — since Python 3.5 it is swallowed into a _FailedTest that errors at run time — so it reached the parent as an anonymous broke=1 and hard-failed with no satisfiability check, while a SyntaxError on an unsatisfiable platform was *excused* as a dep skip. The worker now probes the import explicitly (importlib.import_module) and tags it kind=dep|other; the parent excuses only dep-shaped failures on an unsatisfiable platform, and fails a non-dep-shaped load error (SyntaxError, an import-time code bug) on every platform. F4 — a module that loaded but collected zero tests reported ok (unittest exits 0 on an empty suite). It now FAILS with a clear message. F10 — a non-integer RUN_ADDON_TESTS_TIMEOUT crashed the runner at import with a raw ValueError; it is now ignored with a stderr note and the default used. Tests driven via subprocess on synthetic addon trees, using GooCanvas (apt-provisioned, conda:None) as the satisfiable/unsatisfiable lever.
An author copying the requires_gi tuple shape into requires_mod
(requires_mod=[("psycopg2", ">=2")]) writes a valid literal that both dep
CLIs then choke on: the tuple reaches sorted() mixing tuple and str
(TypeError), or a list entry is unhashable for a set .add() — replacing the
--unmapped drift gate's readable diagnostic with a raw traceback, and
violating addon_python_deps' own 'skip tolerantly, don't abort the batch'
contract.
Guard on isinstance(entry, str) in both scanners (addon_python_deps._declared_
mods and addon_system_deps._scan / addon_requirements): a non-string entry is
skipped with a stderr note, like a non-literal value already is. Tests in both
modules feed a tuple entry and assert the scan yields only the string names
and does not raise; the real tree's --unmapped stays clean.
active_addons.sh's is_active() grepped each .gpr.py for include_in_listing: an =True anywhere, or the absence of the flag, meant active. Two latent bugs the adversarial review flagged: (1) it is file-granular, so an addon with one register(include_in_listing=False) plus a sibling register() that omits the flag was read INACTIVE, though make.py builds/releases it (it reads include_in_listing per registration, default True) — CI would skip lint, structure, compile and tests for an addon that ships; (2) grep has no comment awareness, so a flag mentioned only in a # comment flips the result. Move the rule into a new pure-stdlib active_addons.py that parses each gpr with ast and classifies per register() call: active iff any register omits the flag or sets it to anything but the literal False; an unparsable or register-less gpr is tolerantly active (never silently drop an addon). active_addons.sh now calls it once at source time (python3, python fallback for the conda Windows lane) and is_active() is a membership test — one interpreter per sourcing step, not one per addon. ci.yml is untouched, so the dedup invariants hold. Verified behaviour-identical to the old grep across all 146 current addon dirs; test_active_addons pins the semantics and embeds that whole-tree oracle, so the first gpr to exercise the difference trips the test for human review rather than silently changing CI's gated set.
Two silent test exclusions the review flagged as unexplained: - The unit-test glob is */tests/**/test_*.py, which never matches DynamicWeb/ test_dynamicweb.py (a root-level module). That is deliberate: it is a nose-era dev harness (nose is gone on py3.12, it asserts a USER_PLUGINS install and drives Gramps.py from a source checkout) that would hard-fail if run, and its CI-shaped sibling DynamicWeb/tests/test_dwr_tree_names.py IS matched. State it at the glob so it does not read as an accidental gap. - Sqlite/tests/test_sqlite.py is hard-excluded in both unit lanes with no reason given. Record it: the test needs GRAMPS_RESOURCES + example/gramps/ example.gramps from a gramps SOURCE checkout (the wheels ship only gramps/ + share) and writes fixed /tmp paths; un-excluding is a follow-up. Comment-only; no behaviour change.
…-forge) Comments in environment.yml and ci.yml claimed the conda Windows lane gets gramps from conda-forge and that the pin 'self-heals' when 6.1 reaches conda-forge. Both are wrong: gramps is installed from PyPI via environment.yml's pip: block, and the pin 'gramps>=6.0,<6.1' can never resolve 6.1 regardless of conda-forge. On a gramps61+ branch the lane therefore validates addons against 6.0.x until BOTH 6.1 is published on PyPI AND the pin is bumped by hand. Correct the comments to name PyPI as the source and the two conditions; the ci.yml 'Report gramps-vs-branch series' step already warns on the divergence (logic unchanged). CI-MAINTAINER.md's 'no workflow edits required' for a new branch now notes the one file that IS a manual per-branch bump: environment.yml. Docs/comments only; no behaviour change.
The container image ref embedded ${{ github.repository }} verbatim. GHCR
references must be lowercase and docker-build's metadata-action lowercases
what it pushes, so a fork whose owner has uppercase letters (e.g. EdUralph)
pushed a lowercase image but ci.yml tried to pull the mixed-case ref — an
invalid reference that fails every container job at init. Lowercase it with
${GITHUB_REPOSITORY,,} in the setup step (no new step, so the is_active
dedup pins are untouched). Upstream (all-lowercase owner) is unaffected.
…e, weekly refresh The image lifecycle was the review's weakest subsystem. Three additions to docker-build.yml, none changing the normal push path: - pull_request trigger scoped to .github/docker/** and this workflow: builds the image to validate a Dockerfile change before merge, WITHOUT pushing (the login step is skipped and push: is false via github.event_name), so a fork PR never touches the packages:write token. params derives the series from github.base_ref on a PR. - workflow_dispatch 'no-cache' boolean input wired to build-push-action, for a from-scratch rebuild that picks up base-image/apt and new gramps patch releases the buildx cache key would otherwise skip. - weekly schedule + a weekly-rebuild fan-out job that dispatches a no-cache rebuild per maintenance branch (permissions: actions: write; existing job guarded off schedule). Documented as inert until the workflows reach the default branch, since scheduled runs fire only from there. CI-MAINTAINER.md gains an 'Image lifecycle' section covering all of the above plus the known one-push image lag (ci.yml pulls the moving tag while the rebuild runs, so an image-affecting push tests against the previous image — re-run the affected jobs after the build; the Dockerfile ruff pin lands one push late for the same reason).
|
Ran an adversarial review of this PR (four independent passes: workflow logic, helper scripts, test adequacy via mutation testing, and CI security) and pushed fixes for everything that survived verification. Summary of what changed: Blocking (CI couldn't pass its own first run):
Test blind spots (mutation-tested):
Security:
Correctness / fail-open:
Lifecycle / docs:
13 commits total. All five CI-owned test modules pass locally (49 tests, ruff clean). Next I'm validating the full container pipeline on both the 6.0 and 6.1 lanes on my fork before considering this ready. |
The import-probe added for the load-failure taxonomy used importlib.import_module(modname), which is fragile: the worker appends the addon's own directory to sys.path so the addon's tests can import its top-level modules, and for an addon whose main module shares the addon's directory name (e.g. CalculateEstimatedDates/CalculateEstimatedDates.py) that appended entry shadows the namespace-package directory, so a dotted import_module resolves CalculateEstimatedDates to the .py MODULE and 'CalculateEstimatedDates.tests' then raises 'not a package'. 11 real addons failed this way on CI (gramps-independent — the fresh image would fail too). unittest's own loadTestsFromName resolves the package correctly, so load via it and classify the load failure from whichever form it takes: a RAISED exception (some Python versions raise SyntaxError) or a DEFERRED _FailedTest placeholder (ImportError, wrapped as an ImportError whose message embeds the original traceback). _dep_shaped inspects the terminal exception in that embedded traceback so a wrapped SyntaxError is still classified as a code bug, not a dependency skip. Verified across the F2/F3/F4 fixtures.
What should we do here? Is just changing the warning text enough? Or does something need to change for a proper check? Also noting relationship to: gramps-project/gramps#2250 |
This is part of the answer to issue 9393, using the "X framebuffer" approach (Option 1 of the feature request).
Scope
This PR is strictly CI infrastructure. After review feedback from @kulath (2026-04-18) — "too many completely different changes in a single commit" — all per-addon code/lint/structure work that was originally bundled here has been split into one-PR-per-addon submissions; see the Companion PRs section at the bottom. This PR adds only the workflow files, the container image, the shared CI scripts, and a small shared test harness.
Thanks also to @GaryGriffin for the early testing feedback on bug 9393 and for asking the questions that led to the standalone per-job reproduction commands below.
What's in the PR
CI infrastructure
.github/docker/gramps-ci/Dockerfile— Python 3.12 + Gramps 6.0 + PyGObject + GTK typelibs (incl.gir1.2-gexiv2-0.10for EditExifMetadata / PhotoTaggingGramplet) + xvfb/xauth + ruff, intltool, gettext, git. GTK lives in the base image so addon modules that dofrom gi.repository import Gtkat module load are importable; xvfb/xauth are bundled for tests that actually render. Gramps is installed from PyPI for released series, or from a SHA-pinnedgramps-project/gramps@maintenance/grampsNNsnapshot for an unreleased branch (the series is parameterised via theGRAMPS_SERIESbuild arg)..github/workflows/docker-build.yml— rebuilds the image on.github/docker/**changes or viaworkflow_dispatch, deriving the image tag and Gramps series from the branch ref. Default-branch-only; gates on GHCR write access..github/workflows/ci.yml— eight jobs:make.pyseries suffix from the branch ref (maintenance/grampsNN), so the rest of the matrix is branch-neutral.ruff --select=E9,F63,F7,F82+ trailing-whitespace check (usesgrep -Pso\tmatches a real tab, not a literalt— see commit205b21cfor the regex bug fix).po/template.pot.python3 -m py_compileon every.py.<addon>/tests/test_*.py(skipstest_windows_*/test_integration_*) viarun_addon_tests.py(GI bootstrap + per-module timeout + honest skip accounting).tests/test_plugin_registration.py+<addon>/tests/test_integration_*.py, Linux-only, container--initsoxvfb-rundoesn't hang.make.py gramps60 build all..github/environment.yml— hybrid conda+pip environment for Windows. Gramps isn't on conda-forge, sopygobject/gtk3come from conda; only the stable base (gramps+orjson) comes from pip. Addon runtime deps (dbf,networkx,lxml, …) are no longer pinned here —ci.ymlauto-derives and installs them at runtime from each.gpr.py'srequires_mod(single source of truth;dbfdropped in commit8d2654a).Shared CI scripts
.github/scripts/addon_system_deps.py— single source of truth for addon system dependencies. Maps eachrequires_gitypelib /requires_exeexecutable declared in a.gpr.pyto its package on each CI platform (apt vs conda), and scans the tree soci.ymlderives the install list from one place instead of a hand-kept list. Records platform asymmetry (the GTK 3 libs — goocanvas, osm-gps-map, gexiv2 — exist on apt but not conda-forge). Pure stdlib..github/scripts/run_addon_tests.py— per-addon unit/integration runner that replaces a barepython -m unittest: it (1) bootstraps the GI versions the Gramps GUI launcher pins (Pango/PangoCairo/Gtk) before anygramps.guiimport, (2) runs each module in a subprocess with a wall-clock timeout so a hung test can't stall the job, and (3) FAILs a wholly-skipped module unless the addon's declared system deps are genuinely unavailable on the platform — so a silent skip can't read as a pass..github/scripts/gi_bootstrap/sitecustomize.py— the same GI version pin as asitecustomizeshim, placed onPYTHONPATHfor the discover-/subprocess-loading steps (e.g. plugin registration) so every spawned interpreter inherits the bootstrap.Shared test harness
tests/gramps_test_env.py—GrampsTestCase/GrampsDbTestCasebase classes (stdlibunittest, mirroring upstream Gramps' own test style).tests/test_plugin_registration.py— registers every addon in a subprocess (crash-safe), verifiesgramps_target_version=6.0plus valid id/name/version, smoke-tests import/export entry functions.tests/__init__.py.Docs
CONTRIBUTING.md— a short note that PRs now run automated CI checks, and that a green check on an unreleasedmaintenance/grampsNNbranch means the addon works against that branch tip (a SHA-pinned snapshot), not against a tagged release..github/CI-MAINTAINER.md— operational runbook for agramps-project/addons-sourcemaintainer: one-time setup when this PR merges, and how to stand up a new maintenance branch. The pipeline is otherwise self-driving.Gate policy
PR-blocking checks should only fire on issues the PR's own diff can cause. Every test here runs gramps or addon code, so the policy is:
continue-on-error: false): Lint, Compile Check, Unit Tests (Linux), Unit Tests (Windows), Integration Tests, Build.continue-on-error: true): Addon Structure — the only advisory job.Lint and both Unit-test jobs were advisory while their pre-existing backlog was being cleared by the companion PRs; that backlog is now cleared on
maintenance/gramps60, so they were flipped to blocking (lint ind265612, both unit-test jobs in0dd3f1b). The single remaining advisory is Addon Structure, which stays non-blocking until the four addons missingpo/template.potare fixed in a follow-up PR — flip it off in that PR.Commits
774a9acc6aa10erequires_modin.gpr.py.8d2654adbffrom image/env — installed via auto-derive now.28febdcshell: bashon unit-test-linux + integration-test steps (fixes DashBad substitution).715e71dtest_linux_*/test_windows_*/test_integration_*).dd0fd38gir1.2-gexiv2-0.10typelib to image (companion to #878, #880).205b21c[ \t]to PCRE. The previous BRE pattern matched any line ending int/\/[/]/space (~3 000 false positives across ~430 files); PCRE (-P) recognises\tas a tab — real count 597 lines across 21 files.f5907af.gpr.pydeclaresinclude_in_listing=False.ff215acrequires_modnames againstfind_spec(Pillow/PIL trap).d265612maintenance/gramps60.0dd3f1b9a91d89make.pyargument from the branch ref.abefdbe9927626GRAMPS_SERIESbuild arg.894e48473d75ed3b2a947.github/CI-MAINTAINER.md).ad2815cdepends_on.bc8df217f8a839146649106b95bcLocal reproduction
This PR can't be exercised against
gramps-project/addons-sourcedirectly (workflows live in the PR), but the per-job commands are reproducible standalone:A complete end-to-end test is also available by forking to a personal repo and pushing a branch — the workflows fire because the workflow files come along with the fork. See the comment thread for the step-by-step walkthrough.
Companion PRs
Code/lint/structure work that was originally bundled here is now submitted as one-PR-per-addon. Tracker:
displayer, JSON: log repr(obj) on unrecognized object instead of undefined data #871 JSONdata, LifeLineChartView: fix tuple-in-if outer condition for tooltip cache #872 LifeLineChartView tuple-in-if, libaccess: add missing value parameter to Person.gramps_id setter lambda #873 libaccess lambda (thanks to @GaryGriffin for catching that the first version of libaccess: add missing value parameter to Person.gramps_id setter lambda #873 was missing the actual code edit), lxml: drop self.uistate.window from module-level ErrorDialog calls #874 lxml module-levelself.webreport/common.pyno-ICU fallback) — both required to unblock plugin-registration smoke tests in this PR.As of HEAD the lint and unit-test backlogs are cleared and those gates are blocking; once the remaining
po/template.potgaps land, Addon Structure can be promoted to blocking too.🤖 Generated with Claude Code