Add NameSuite Addon (tool + gramplet) to fix, infer and audit given and patronymic names#941
Conversation
…nd patronymic names
|
Thanks for the submission. Couple comments
When I tried to run, I got: Running unittests: |
|
Thanks for the feedback. |
|
I ran the unittest from the addons-source directory. By default (without a MANIFEST) only Package your Addon files in the Addon directory are put into the download file. |
|
A few questions:
|
You should use unittest instead of pytest for the Addons repo.
imports are relative to the test file location.
Every python file should have the license info in the header. Be sure to include your copyright info.
If you do not want them in the download, then they are not needed in Github. They can remain if you want - your call. They could be the basis for the help_url.
It will be needed in every directory that has a test. If you flatten and only have a single tests directory, then there. If you keep the multi-level directory structure, you will need in each of the sub directorie that have tests. |
|
@GaryGriffin updated the PR
My email in GPR is real, it's the GitHub-specific email GitHub creates for devs. |
|
All tests worked. Tried Addon and got warnings (errors): Audit Given and Patronymic Names
Patronymic Suggestion
You need to include the name_processor files in the MANIFEST. Currently only the python files in the NameSuite directory are in the download, you need to add the following to the MANIFEST
After adding that, the Audit works. The Suggestion fails with:
Comments:
|
|
In Always importing it fixes the issue and the Addon works. |
|
Changed my naming convention to put Patronymic in Surname (Origin=Patronymic) instead of Given Name. Audit Gramplet worked great. |
Confusingly, Gramps considers Patronymic names (e.g. Petrovich in Ivan Pertovich) a type of surname. For East Slavs it's not really a surname, but this historical hack works pretty well to the point Gramps documents it in the wiki. They later built a dedicated name format to visualize them properly and the hack became a feature. Typing pat names manually and adjusting the surname type is time-consuming and far from fun. That's why I've built the gramplet. For the large DBs (I have 2k+), manually copy-pasting pat names creates a lot of typos: names with the wrong gender, grandfather's names being accidentally used instead of the father's... It also helps to catch children attached to the wrong parents, thanks to pat names serving as a second check.
It's a pat names corpus for tests. I have a quite advanced morphology engine to derive the correct endings for pat names. It requires testing on a corpus of real historical East Slavic names, as, to my knowledge, no complete set of linguistic rules exists, and some names are the official exceptions.
That is easy to change. I have a script that adds licenses to files.
Yes, good catch. That's
I thought the Addon must be live first?
I see. This part is quite hard to figure out. The other addons I've checked consist of just one file, so they aren't a good example. When I test by copying Python files to my Gramps plugin dir, everything works fine with no warnings or errors. |
|
@GaryGriffin I think I figured it out. Updated the repo. Here's the |
|
Thanks for the better description of Patronymic. Some answers:
Open Issues:
Specifically, views/gramplet.py : line 71:
This causes the Patronymic Suggestion Addon to fail. When I deleted line 36 and adjusted spacing on line 37, it worked. Cleanup |
Thanks for the details, as I wasn't able to reproduce it with Windows Gramps 6.0.8. Let me figure out the most compatible solution. Do you know the Python version used to build your Gramps? Don't worry if you don't, it's not a problem. |
Mac version of Gramps 6.0.6 - 3.13 Unless you disagree, my intention is to cherry pick this addon (once published on 6.0) to 6.1 |
My Win Gramps uses 3.14.4, so that's why I wasn't able to see it. Good catch! I'm going to update the code to make it compatible with all 3.10+. Bear with me. |
|
As another point of reference, Mac Gramps 5.1 and 5.2 use python 3.11 . |
|
A new guidelines wiki page for addon developers is being written. It is still very preliminary. Your comments would be appreciated. |
|
One suggestion. In the Audit Patronymic --> Audit Settings, it would be useful to add a Person Filter option. This would limit the part of the database that is searched. I have over 10K+ people but < 100 are Slavic names. In my case, one person married someone from Russia and limiting the database search to only their ancestors and relations would reduce the scope of the search. |
Since I've gotten back to the codebase, I'm doing another pass, refactoring to make it better decoupled. This way, I'll be able to have normal E2E tests with minimal impact from Gramps imports (GTK, DB). I wish Gramps were more testable. Not being able to run headless Gramps (no GTK) to test DB access with fixtures is a major downside. Not having type annotation in Gramps is another safety downside. Seems like a low-hanging fruit, given the AI automation. |
I need to read the new one, but the comments on the old one: P0: Lack of pointers to good examples. 90% of developers don't have time to read extensive docs. They would take and modify someone else's existing code. It was already like that pre-AI, now that ship had sailed. If developers find a bad/outdated example, it can spread bad practices across the codebase. P1: Some data was outdated. AFAIR on "addons" repository, and on "requesting write access" to the Gramps repo, instead of forking it. P1: The part on how addons are imported and what namespace is used kept me puzzled. All the other addons were simple one-file Python scripts that weren't representative. P2: The MANIFEST-related part was confusing, and how the P2: Moreover, I'd fine-tune the docs to be 50/50 Human/LLM parsable, provide clear must not guidelines that could be added to the P2: Since the guidelines mentioned above are... guidelines, and not enforcement. I'd develop tests, lints, etc., that would actually enforce validation on the These are a few from the top of my head. I'll check the updated docs, but it may take me some time. Need to balance my addon refactoring with other things. |
I may need your feedback on the speed of the audit. How long does it take on your DB? I'm considering adding a caching layer to speed it up. |
I have 17,500 people and it took 1 min 5 sec on an M3 Mac.
I believe this is possible. Look at some of the recent changes that have unittests. SQLiteWithSelect may be a good example of test cases.
I wrote a Sample addon a few years ago but it was more about using the API and not about guidelines. You bring up a good point.
I updated the docs to clarify.
Yours is one of the first that has subdirectories with files where this is an issue.
Does Package your addon make this clear, or is this where the confusion lies?
We are moving to a CI/CD process for the addons-source repo, so guidelines are likely to be more rigorous. Lint and black are still suggestions. But there are testing processes for CI/CD that we are now running (not in production yet). |
|
@eduralph let me know where to post the feedback to your GERP, and we can move it there. TL;DR: IMHO, what you're describing is the Gramps Public API. Taking that perspective may straighten the whole GERP.
Access to private internals ends up being the core maintainer's problem on every migration. I'd restrict it. If some functionality is missing, it'd be added to the API.
This all makes sense. Semver is standard in Python packages. This section could be slimmed down. If I'm not missing any internal details, the message is
If you ask me, I'd approach this one differently. Python has Protocols that were designed specifically to declare APIs/Interfaces. ABCs are the historical option, but Protocols are superior. They provide static typing (that Gramps is very past due on), and allow Addons to be isolated from the Gramps internals. Having a whitelist of imports is Pros: Gramps already has a DB API that can and should be converted to Protocols anyway.
Great point, I'd even extend this tooling section. With the modern speed of development, the guardrails are a must.
Don't have a background, not going to comment on it. These things are highly opinionated and tend to consume time in discussions. I prefer not to go there unless there is a clear pain point everyone agrees on.
Oh my, pls replace Wiki with the GitHub code-based (or some automated) solution. I know it's out of scope for this GERP, just throwing it here.
Hide the entire concept of
For web services, libraries, etc., the Public API defines the main (and only) version, as that's what customers face. For Gramps as a desktop app, it could be different. Users may see a major change in functionality without a change in the API. I'm less of an expert there, but the independent Gramps & API versioning makes sense.
If agreement is slow to reach, I'd kick it into a separate low-priority proposal.
Do what Python does: support old versions :-)
I'd raise priority on this one heavily. Registration is a pain point for addon developers. It's a Python file being non-Python. No tests, linting, or type hinting. If I, as an addon dev, mess it up, I can't test it within the Hope it helps |
@eduralph Is this complete for the CI/CD process? The template.pot seems reasonable and I am merging your make.py changes so that point is doubly covered. |
|
@bryndin — small but important: get_addon_translator(file) loads the catalog from dirname(file)/locale, so binding _ in the name_processor/views/* modules makes it look under name_processor/views/locale, which doesn't exist — the .mo files live at NameSuite/locale/. I tested it against real Gramps with a compiled ru/LC_MESSAGES/addon.mo: bound in a nested module, a Russian user gets the untranslated English string (the ValueError falls through to the core catalog); anchored at the addon root it returns the translation. So the idiom needs to be anchored at the addon root, not each module's file. Cleanest: bind _ once in a new name_processor/i18n.py: import os # this file is /name_processor/i18n.py → the addon root (which holds locale/) from name_processor.i18n import _ |
|
@eduralph
While I can do it to unblock this PR, the better options could be: Option 1: Make Gramps
|
I was thinking the same thing. We have the foundation of our own mini-installer now; we could use that infrastructure... or something even easier. |
Thanks for your ideas, I think there is some really good ideas to include in the GEPS - I'll revise it. |
|
@GaryGriffin If you are thinking about changing the way addon translations are handled, then you should also consider the possibility of compiling and distributing the aggregated translations instead of extracting them. This would have the benefit of separating the translation files from the individual addons. Users wouldn't see a new addon version for just a translation update. |
|
Should the translation topic move to a separate discussion rather than this PR? It is above my Python knowledge to suggest a path forward. I think the appropriate people ( @bryndin @eduralph @dsblank at a minimum ) need to drive to a plan for how to handle addon translations, as they seem already involved with it. |
|
The translation question should be made in "discussions" in the gramps repo. |
|
@bryndin - your Option 2 is the thing I proposed — both bind _ once, anchored at the addon root (the dir that holds locale/), and both let the view modules just from … import _. The reason it can't be a per-module file binding isn't a stylistic choice on my end: get_addon_translator(filename) derives the catalog dir as os.path.dirname(os.path.abspath(filename)) + "/locale" (gen/utils/grampslocale.py). So a binding in name_processor/views/base_tab.py looks under name_processor/views/locale/, which doesn't exist, and a Russian user silently gets English. Anchoring at the addon root is what makes it resolve NameSuite/locale/…/addon.mo. |
|
@GaryGriffin @eduralph @dsblank @Nick-Hall I jammed on the options for the i18n design, here's a high-level RFC (it's human-readable):
If you are time restricted, check the best option:
We can move this discussion elsewhere. |
|
Your Translation Workflow for Addons was correct up to v5.2 . However, version 6.0 translations moved to a Weblate process (Gramps core moved to a Weblate process during 5.x), found at https://hosted.weblate.org/projects/gramps-project/addons/ . Both Gramps core and Addons have separate Weblate projects for translations. The current translation process for Addons is:
|
|
Thanks @GaryGriffin , I'll update the RFC. The "best solution" should stay the same, but the details need a rewrite. |
|
@eduralph updated the i18n logic, it resides in the dedicated NameSuite/name_processor/views/i18n.py |
|
"Gramps Addons - Current l10n process" This is the summary of the current process, as far as I understand it. @GaryGriffin is the process correct? Let me know if you have better documentation. |
|
Here's the updated RFC: https://gist.github.com/bryndin/b0fbd268b4785d0e6600b8270daca75e Description of the current L10n process in Gramps: |
|
Suggest you start a discussion at https://github.com/gramps-project/gramps/discussions as this will require gramps core changes. This also gives a wider discussion area. |
|
@eduralph does the PR align with the CI proposed process and translations? From your perspective, it is ready to merge? |
_trans` = glocale.get_addon_translator(os.path.join(_ADDON_ROOT, "name_processor.gpr.py")) would do. |
@eduralph should there be a repository-wide test for that, so that contributors like me can self-test it? |
No, unfortunately not but is a good idea for a rule in a precommit hook. Definitely something to add to it once we get a CI process in place. |
My understanding, a direct implementation would require all the Gramps dependencies, which is a no-go for the addon contributors. |
@eduralph updated, can you pls check? |
|
@bryndin - all green, no more nit-picks |
|
I've posted the RFC: Redesigning the L10n Architecture for Gramps Addons |



See https://github.com/bryndin/namesuite