Skip to content

ImportMerge: fix AttributeError when adding/merging Tag objects (bug 0014056)#826

Merged
GaryGriffin merged 5 commits into
gramps-project:maintenance/gramps60from
eduralph:fix/importmerge-tag-gramps-id-14056-upstream
Apr 21, 2026
Merged

ImportMerge: fix AttributeError when adding/merging Tag objects (bug 0014056)#826
GaryGriffin merged 5 commits into
gramps-project:maintenance/gramps60from
eduralph:fix/importmerge-tag-gramps-id-14056-upstream

Conversation

@eduralph

Copy link
Copy Markdown
Contributor

Summary

Fixes bug 0014056Import and Merge tool crashes with AttributeError: 'SQLite' object has no attribute 'has_tag_gramps_id' when the user selects Add (or hits a GID conflict) on a Tag row.

Tag is a table object and, unlike primary objects, does not have a gramps_id field — there is no has_tag_gramps_id / find_next_tag_gramps_id on the database. The generic getattr(self.db1, 'has_' + obj_type.lower() + '_gramps_id') lookup in do_commits therefore raised before the commit could complete.

Changes

  • Guard both GID-conflict branches in ImportMerge/importmerge.py (S_ADD and S_DIFFERS) so they skip Tag.
  • Bump version in importmerge.gpr.py (0.0.250.0.26).
  • Add an integration test exercising both branches via the real Gramps SQLite backend; the test fails without the guard and passes with it.

Confirmed by upstream maintainers (Nick_H, dstraub) in the bug thread: tags are table objects without a Gramps ID, so the generic lookup is invalid for them.

Test plan

  • python3 -m pytest ImportMerge/tests/test_integration_importmerge.py -v passes
  • Reverting the guard makes the new tests fail with the original AttributeError
  • Manual verification with the steps from the bug report (Gramps Sample tree, sort by Object descending, Add a Tag row, Done, Save) — completes without crash

🤖 Generated with Claude Code

@GaryGriffin

Copy link
Copy Markdown
Member

no need to bump version.

Tag is a table object without a gramps_id field, so the generic
has_<obj_type>_gramps_id / find_next_<obj_type>_gramps_id lookups in
do_commits raised AttributeError when the user selected Add on a Tag
row. Guard both the S_ADD and S_DIFFERS GID-conflict blocks so they
skip Tag.

Adds integration tests covering both branches; verified they fail
without the guard and pass with it.

Fixes bug 0014056

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@eduralph
eduralph force-pushed the fix/importmerge-tag-gramps-id-14056-upstream branch from 8f28bf2 to b639a8c Compare April 18, 2026 00:42
@eduralph

Copy link
Copy Markdown
Contributor Author

Undid the versioning

@eduralph

Copy link
Copy Markdown
Contributor Author

The tests require following

  1. Python 3 + pytest
  2. Gramps importable as a Python package (python3 -c "import gramps" must work)
  3. PyGObject (python3 -c "import gi")
  4. GTK 3.0 typelib (gir1.2-gtk-3.0 / gir1.2-gdk-3.0)
  5. PYTHONPATH=. from the addons-source repo root (so ImportMerge is importable)

then you can runs the tests like so

PYTHONPATH=. GRAMPS_RESOURCES=$(python3 -c "import os, gramps; print(os.path.dirname(os.path.dirname(gramps.file)))") python3 -m pytest ImportMerge/tests/test_integration_importmerge.py -v

PR 820 fulfills all the requirements, so you don't have to take care of that normally. That's why the tests are classified as integration tests you can find at test_integration_importmerge.py. I ran into a gap while testing on my dev machine and will correct that in the tests.

On systems with both GTK3 and GTK4 installed, PyGObject defaults to
GTK4, and importing ImportMerge pulls in gramps.gui which crashes on
Gtk.IconSize.MENU (a GTK3-only enum). Mirror the pin that
gramps.grampsapp performs at startup so reviewers can run the test
without environment tweaks.
@Nick-Hall

Copy link
Copy Markdown
Member

Tests must use the unittest framework — do not use pytest.

See the Testing section in the AGENTS.md file.

AGENTS.md requires the unittest framework for tests. Convert the
pytest-based integration test (fixtures + assert statements) to a
unittest.TestCase with setUp/tearDown and self.assert* calls. GTK
availability is now checked with a module-level try/except raising
unittest.SkipTest, which also makes the separate GTK-pin step
redundant (still applied here before the gramps import).
@eduralph

Copy link
Copy Markdown
Contributor Author

Sorry, the agent caches the content, so it doesn't pick up on changes. changed that, should now be conform. I also adjusted it to using unittest.

Pre-reqs are

  • Python 3.10+
  • Gramps — installed and importable as a Python package (python3 -c "import gramps" must succeed)
  • PyGObject (the Python gi bindings)
  • GTK 3.0 typelib — gir1.2-gtk-3.0 and gir1.2-gdk-3.0 on Debian/Ubuntu (ImportMerge is a GUI tool whose module-level from gi.repository import Gtk runs at import time)
  • The addons-source repo checked out, with ImportMerge/ on disk

Command is

PYTHONPATH=. GRAMPS_RESOURCES=$(python3 -c "import os, gramps; print(os.path.dirname(os.path.dirname(gramps.file)))") python3 -m unittest ImportMerge.tests.test_integration_importmerge -v

I'll now double-check the other PRs

Addresses AGENTS.md rule requiring Black-formatted Python. Collapses
three multi-line function calls that fit on a single line.
Addresses AGENTS.md requirements:
- Type hints on all helpers and test methods using Python 3.10+
  syntax (``X | None``, ``tuple[X, Y]``).
- Sphinx-style ``:param:`` / ``:returns:`` docstring markers on the
  helper functions.
- ``# ------`` class-header divider above the TestCase so it's easy
  to locate.

No behavioural change.
@GaryGriffin

Copy link
Copy Markdown
Member

Unittest fails on Mac. Traceback is below. It appears to still want to use pytest, among other issues.

[jhbuild]:  % PYTHONPATH=. GRAMPS_RESOURCES=$(python3 -c "import os, gramps; print(os.path.dirname(os.path.dirname(gramps.file)))") python3 -m unittest ImportMerge.tests.test_integration_importmerge -v
Traceback (most recent call last):
  File "<string>", line 1, in <module>
    import os, gramps; print(os.path.dirname(os.path.dirname(gramps.file)))
                                                             ^^^^^^^^^^^
AttributeError: module 'gramps' has no attribute 'file'
test_integration_importmerge (unittest.loader._FailedTest.test_integration_importmerge) ... ERROR

======================================================================
ERROR: test_integration_importmerge (unittest.loader._FailedTest.test_integration_importmerge)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_integration_importmerge
Traceback (most recent call last):
  File "/Users/GrampsTest/gtk/inst/lib/python3.13/unittest/loader.py", line 137, in loadTestsFromName
    module = __import__(module_name)
  File "/Volumes/Storage/Development/addons-source/ImportMerge/tests/test_integration_importmerge.py", line 44, in <module>
    import pytest
ModuleNotFoundError: No module named 'pytest'


----------------------------------------------------------------------
Ran 1 test in 0.000s


@eduralph

Copy link
Copy Markdown
Contributor Author

Yes, you're right - I'm guessing it also has something to do with the environment I've been testing it in. Let me get a bit more structured and define a proper baseline and run the tests against that.

@eduralph

eduralph commented Apr 19, 2026

Copy link
Copy Markdown
Contributor Author

@Nick-Hall , part of the reason why the system wasn't honoring AGENTS.md was because it's not in the maintainance/gramps60 line. It is in master and maintainance/gramps61 line of gramps though.

@eduralph

Copy link
Copy Markdown
Contributor Author

Sorry for the churn earlier, @GaryGriffin — the traceback you got was hitting a pre-conversion snapshot where import pytest was still at the top of the file. That was fixed a few commits later in a37f52ed (ImportMerge: rewrite integration test with unittest framework), and the branch has since also picked up Black formatting (ac81d5f) and type hints + class header (72281fb) per the gramps AGENTS.md conventions. The current tip of the branch has no pytest reference anywhere.

To avoid any messes with any kind of mix-up, I put together a reproducible Docker-based harness at eduralph/gramps-testbed. Clean Ubuntu 24.04 run against this PR branch.

Any comments on that are welcomed of course.

@GaryGriffin
GaryGriffin merged commit 52433a9 into gramps-project:maintenance/gramps60 Apr 21, 2026
GaryGriffin added a commit that referenced this pull request Apr 21, 2026
GaryGriffin added a commit that referenced this pull request Apr 21, 2026
eduralph added a commit to eduralph/addons-source that referenced this pull request May 8, 2026
* Wrap label vaue to ensure it is a string

The argument `label` to Gtk.Label must be of type string. In recent versions of Python and/or PyGObject, this check has become stricter and as a result is throwing an exception instead of silently converting to a string. So now we explicitly convert to string.

Fixes #14181

* TimelinePedigreeView: fix crash on second right-click in context menu (bug 0012387)

The settings submenu builder appended the same SeparatorMenuItem twice,
which GTK rejected with "Can't set a parent on widget which has a parent".
The corrupt parent linkage caused a segfault when the previous menu was
garbage-collected on the next right-click. Removing the stray duplicate
append fixes all four context-menu paths (background canvas, person,
relation, missing parent) and silences the related GTK_IS_WIDGET
assertion warnings. Also resolves duplicate report 0013463.

* Merge TimelinePedigree PR 819, 823

* DataEntryGramplet: fix crash when adding person with no Family Tree open (bug 0012691)

Clicking Add (or Save after a dirty edit) when no tree was loaded
raised AttributeError: 'DummyDb' object has no attribute 'get_undodb'
from DbTxn. Guard both mutating callbacks on dbstate.is_open() and
surface a clear ErrorDialog instead.

Add unit tests for the closed-db guards, pre-existing input guards,
and .gpr.py registration metadata so future refactors can't silently
break the bug-12691 fix.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* DataEntryGramplet: convert tests to stdlib unittest (bug 0012691)

Gramps' own test suite uses unittest, not pytest, so addon tests that
ship alongside the codebase should follow the same convention to stay
contributable upstream without rewriting.

Replace pytest fixtures, monkeypatch, and module-level test functions
with unittest.TestCase classes and mock.patch.object. Guard the
module-level addon import with a try/except that raises SkipTest so
collection is quiet on environments without the GUI stack, and pin
Gtk to 3.0 before gramps imports to avoid the GTK4 fallback crash on
Gtk.IconSize.MENU.

* DataEntryGramplet: qualify test import to pick up the class not the module

When unittest loads this file as DataEntryGramplet.tests.test_..., the
outer DataEntryGramplet is already a namespace package in sys.modules,
so `from DataEntryGramplet import DataEntryGramplet` binds the submodule
and DataEntryGramplet.NO_REL (class attr) raises AttributeError. Fix by
importing the class via its fully-qualified path.

* Merge DataEntryGramplet: fix crash when adding person with no Family Tree open (bug 0012691) gramps-project#824

* CalculateEstimatedDates: handle ancestry-loop DatabaseError per-person (bug 0007898)

probably_alive_range raises DatabaseError when it detects loops in
ancestor or descendant chains. Previously this propagated out of the
removal, selection, and apply loops and tore down the entire tool,
leaving signals disabled and the progress dialog stuck open.

Wrap each per-person iteration with try/except so a single bad record
is logged and skipped, and add outer try/finally blocks so signals are
re-enabled and the progress dialog is closed even on unexpected
failures. Surface a "Skipped N people due to errors" message to the
user when any rows were skipped.

Add unit tests covering get_modifier branches, calc_estimates happy
path, DatabaseError propagation from calc_estimates, and .gpr.py
registration metadata. The addon module is loaded lazily inside a
fixture so pytest collection succeeds even when the GUI stack cannot
import.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* CalculateEstimatedDates: convert tests to stdlib unittest (bug 0007898)

Gramps' own test suite uses unittest, not pytest, so addon tests that
ship alongside the codebase should follow the same convention to stay
contributable upstream without rewriting.

Replace pytest fixtures, monkeypatch, and pytest.raises with
unittest.TestCase, mock.patch.object, and assertRaisesRegex. Guard the
module-level addon import with a try/except that raises SkipTest so
collection is quiet on environments without the GUI stack, and pin Gtk
to 3.0 before gramps imports to avoid the GTK4 fallback crash on
Gtk.IconSize.MENU.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Merge CalculateEstimatedDates: handle ancestry-loop DatabaseError (bug 0007898)gramps-project#825

* ImportMerge: fix AttributeError when adding/merging Tag objects

Tag is a table object without a gramps_id field, so the generic
has_<obj_type>_gramps_id / find_next_<obj_type>_gramps_id lookups in
do_commits raised AttributeError when the user selected Add on a Tag
row. Guard both the S_ADD and S_DIFFERS GID-conflict blocks so they
skip Tag.

Adds integration tests covering both branches; verified they fail
without the guard and pass with it.

Fixes bug 0014056

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* ImportMerge: pin Gtk to 3.0 in integration test

On systems with both GTK3 and GTK4 installed, PyGObject defaults to
GTK4, and importing ImportMerge pulls in gramps.gui which crashes on
Gtk.IconSize.MENU (a GTK3-only enum). Mirror the pin that
gramps.grampsapp performs at startup so reviewers can run the test
without environment tweaks.

* ImportMerge: rewrite integration test with unittest framework

AGENTS.md requires the unittest framework for tests. Convert the
pytest-based integration test (fixtures + assert statements) to a
unittest.TestCase with setUp/tearDown and self.assert* calls. GTK
availability is now checked with a module-level try/except raising
unittest.SkipTest, which also makes the separate GTK-pin step
redundant (still applied here before the gramps import).

* ImportMerge: apply Black formatting to integration test

Addresses AGENTS.md rule requiring Black-formatted Python. Collapses
three multi-line function calls that fit on a single line.

* ImportMerge: add type hints and class header to integration test

Addresses AGENTS.md requirements:
- Type hints on all helpers and test methods using Python 3.10+
  syntax (``X | None``, ``tuple[X, Y]``).
- Sphinx-style ``:param:`` / ``:returns:`` docstring markers on the
  helper functions.
- ``# ------`` class-header divider above the TestCase so it's easy
  to locate.

No behavioural change.

* Merge ImportMerge: fix AttributeError when adding/merging Tag objects (bug 0014056) gramps-project#826

* Form: fix crash and surface clear errors for malformed XML (bug 0011707)

A family section whose title lacked the expected 'X/Y' separator caused
a ValueError: not enough values to unpack when the form editor opened,
crashing the Forms gramplet. The underlying issue was that the addon
trusted the XML definitions and had no schema validation or user-facing
error reporting for broken files.

Split the validation out of form.py into a pure-Python form_validator
module (no GTK/Gramps imports) so it can be unit-tested without a GUI.
The Form loader now:

* parses each file defensively (ExpatError -> ErrorDialog),
* runs the validator before loading (invalid files -> ErrorDialog with
  the file path, offending form id, and the rule that failed),
* skips any <form> element that fails validation while still loading
  sibling well-formed forms from the same file.

split_family_title() in form_validator belt-and-braces the FamilySection
constructor so a missing separator no longer raises, even if validation
is bypassed.

Also adds diagnostic logging:

* INFO log of forms loaded per file,
* DEBUG trace of each file parsed and each form id loaded/skipped,
* DEBUG when EditForm opens (event/citation handles),
* WARNING in FamilySection if its title lacks 'X/Y'.

Tests:

* Form/tests/test_form_validator.py -- 32 pure-Python unit tests,
  covers split_family_title, every validation branch, parse_and_validate
  file handling, and a sanity check that every shipped form_*.xml passes
  validation.
* Form/tests/test_integration_form.py -- unittest integration tests that
  patch ErrorDialog to verify the loader surfaces syntax errors, invalid
  family titles, missing role, and invalid section types; partially
  broken files still load their valid forms; shipped files trigger no
  dialogs.

Partially addresses bug 0011010 (request for user error dialog for
unsupported elements) by covering its core ask: clear errors for
invalid section types, missing/empty role, missing/empty type, and XML
syntax errors.

Fixes #11707.
Refs #11010.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* Form: detect empty definition files and warn on column-size mismatches (bug 0011010)

A form definition file whose <forms> root contained no <form> elements
used to load silently — the loader iterated zero <form> children and
returned, leaving the user with no feedback.  validate_form_dom now
reports this as an error so the ErrorDialog wiring added for bug 11707
surfaces the problem on load.

get_form_warnings is a new non-fatal check for sections whose <column>
<size> values do not sum to 100.  78 shipped definition files violate
this rule today without breaking rendering (the size field is parsed
but never read by the layout code), so treating it as an error would
flag correctly-working forms.  The loader logs the warnings via
LOG.warning instead, making authoring mistakes in user-authored
custom.xml files diagnosable without harassing users of the built-in
forms.

Unit coverage: rejected empty <forms>, forms-root-with-only-comments,
and every branch of the sized-column check (no columns, no sizes,
summing to 100, summing to other totals, partial sizing, independent
of errors, multiple sections).  Integration coverage: empty-forms
triggers ErrorDialog; size mismatch is logged but does not block the
form from loading.  Tests use stdlib unittest to match Gramps' own
conventions.

* Merge Form: gramps-project#821 and (bug 11010)gramps-project#822

to 6.1 branch

* WebSearch: fix bare imports in test_filetable for dotted-path loading

`WebSearch/tests/test_filetable.py` imports `models`, `constants` and
`db_file_table` without a package prefix. Those resolve only when
`WebSearch/` itself is on sys.path — i.e. when the test is loaded via
`unittest discover` from inside `tests/`. Under the dotted-path form
that addons-source's own ci.yml uses (`python3 -m unittest
WebSearch.tests.test_filetable` from the addons-source root), the
imports look for a top-level `models` module and the test fails to
load:

    ImportError: Failed to import test module: test_filetable
    ModuleNotFoundError: No module named 'models'

Add the same `sys.path.insert(0, …parent dir…)` prologue that
TMGimporter and Form already use for the same pattern, so the test
loads under either form. No behavioural change beyond the imports.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Merge WebSearch: fix bare imports in test_filetable for dotted-path loading gramps-project#833

---------

Co-authored-by: Himanshu Gohel <1551217+hgohel@users.noreply.github.com>
Co-authored-by: GaryGriffin <genealogy@garygriffin.net>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
@eduralph
eduralph deleted the fix/importmerge-tag-gramps-id-14056-upstream branch June 7, 2026 16:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants