Make bag-based translations more freeform?

[HalfWhitt]

There are a number of places where the template and translations get pretty awkward, because some languages may need to put things in different orders than others. For example, Chinese is (so far) the only language that needs to put some text after the link to the PyCon video in the sprint description. So we end up with something like this:

<p>{{ "sprint_description"|trans }} <a href="https://www.youtube.com/watch?v=hOtKgFaFcz0">{{ "sprint_pycon_video"|trans }}</a>{% if post %} {% endif %}{{ post }}.</p>

(Which still isn't quite right, as John pointed out.)

I propose that these chunks of text get grouped together into a single variable each.

This PR demonstrates one way that we could do this while still keeping URLs stored in only one place, using a second pass of Jinja template-processing. Usage is that in the template, the URL is declared:

{% set pycon_link = link_to("https://www.youtube.com/watch?v=hOtKgFaFcz0") %}

And then the strings in translate.ini call the variable:

Code sprints are events held normally after a conference where attendees can use the conference venue to work in small groups on various projects. Learn more about how sprints work with the {{ pycon_link("PyCon US 'What is a sprint?' video") }}.

代码冲刺是通常在会议结束后举行的活动，与会者可以使用会议场地分组处理各种项目。用{{ pycon_link("PyCon US 什么是冲刺视频") }}了解更多关于冲刺是如何工作的。

This way, each language can arrange things in whatever order works best, without being constrained by the template.

Alternatively, we could simply set the URL in the template without generating a callable, but then we'd need to either type out the <a> tags in the strings or make them all markdown.

I'd like to check in on what others (@freakboy3742? @kattni? @johnzhou721?) think before I go and convert everything. Does this seem like a useful workflow? Is there anything about it I should reconsider?

PR Checklist:

• All new features have been tested
• All new features have been documented
• I have read the CONTRIBUTING.md file
• I will abide by the code of conduct

[johnzhou721]

Definitely useful workflow; however, before expanding on it, I'd like this to be in a more commonly recognized format and the format strings to be more standard such that the formatting placeholders is able to be recognized by Weblate and translated there; in return, we get free warnings if some of those strings doesn't include the required placeholders (aka. mismatched for English).

That said, we can simplify all that by using..... (drumroll).... GNU GETTEXT! Haven't looked much into that yet but you'd need to add | safe to avoid HTML being escaped... but does Weblate spot mismatched HTML markup? Also does the templates like {% ... %} work?

If not this might be the best approach, given that using GNU Gettext needs more effort.

[johnzhou721]

On the other hand I have no objections w/ this current approach; we're giving translators more context to work with which means better translations (as demonstrated by #652 -- I looked at the context).

[HalfWhitt]

I know nothing about gettext, other than the fact that Jinja has built-in support for working with it. I'm pretty hazy on the history of our translation mechanisms, what we've already previously used, and why we're no longer using them...

[github-actions]

Visit the preview URL for this PR (updated for commit 7e6f52f):

https://beeware-org--pr668-double-process-8xisvu73.web.app

_{(expires Tue, 24 Jun 2025 02:41:41 GMT)}

_{🔥 via Firebase Hosting GitHub Action 🌎}

_{Sign: b0da44bc067e7d9a4255c77cb2c5fce572218cec}

[HalfWhitt]

@freakboy3742 Have we ever used or looked at the i18n Jinja extension? From a cursory glance, it looks like it's pretty flexible.

[HalfWhitt]

...All this said, it would be nice to reinvent as few wheels (ours or third-party) as possible. I should probably hold off making any structural changes like this till I've looked at how we integrate with Weblate, so I have a better idea what will be more compatibility.

[freakboy3742]

This is definitely clever - but m impression here is that we're pushing lektor and database further than they're intended. In particular, when we get to long form text like the sprint description, it feels like we should be leaning more on what Weblate and gettext can do, rather than rolling our own solutions.

I haven't done anything with jinja-i18n - but I'd be shocked if there isn't a gettext integration that can work with jinja templates at some level.

[johnzhou721]

Hint: monolingual gettext

[HalfWhitt]

Alrighty, I'll do some reading up. 👍

[johnzhou721]

For jjnja i18n you can only have expressions (technically variables but it provides facilities to assign the expression into an usable variable) but I don’t think you could call functions with parameters that gets translated… therefore the best bet is probably to use our own plugin.

My proposed way to do this:

• keep the translate filter
• At begin build load the POs
• use Monolingual gettext (b/c the current model uses a key for the messages and not the English message as the ID)
• use pretty much the current process but when we’re getting the translation, get it from a po file of the current language or English if unavailable else nothing
• keep track of the message ID that needs to be written into the pot file in an array in memory
• after build all hook: write pot with all the keys, merge pot with existing messages and update po files

However if we do this it’d take an extra lektor run for us to add a new string, and also we have to modify the po file by hand to add new messages… and it’d take a while before Weblate have those new messages for us to translate which is slow… and also we need to have consistent order of the messages that probably shouldn’t be alphabetical… and that’d be hard to handle.

I don’t have familiarity with POlib so can’t contribute a change.

@HalfWhitt @freakboy3742 what do you think about having our own plugin do it like this? Any other approaches I might have missed?

[freakboy3742]

@HalfWhitt @freakboy3742 what do you think about having our own plugin do it like this? Any other approaches I might have missed?

I'm afraid I don't have a clear enough understanding of what you're proposing to pass comment. You say "our own plugin"... our own plugin for what? Lektor? Jinja? Both?

We're in a space where I'm not familiar with what Jinja can do, and I'm not familiar with how Lektor can be configured to use Jinja, and I'm only partially familiar with the broader processes around translation. I'd need to have a lot more specific detail to sign off on any approach.

And, if we're going to continue this as a technical discussion, it probably shouldn't be on a closed pull request, but on an new issue that has a high level "problem statement" that we can resolve with a specific technical fix.

[johnzhou721]

@HalfWhitt @freakboy3742 what do you think about having our own plugin do it like this? Any other approaches I might have missed?

I'm afraid I don't have a clear enough understanding of what you're proposing to pass comment. You say "our own plugin"... our own plugin for what? Lektor? Jinja? Both?

Using our existing beeware lektor plugin. Closing the loop on this PR for now; when I have more time, I will open a new issue w/ a reference implementation, but that might be a few months away.

[johnzhou721]

Never mind… jinja i18n should be good but you can’t translate functions isn’t that much of a big deal, but in the case of pycon_link we can do link = pycon_link(_(“whatever”)) in the top of the trans block and {{ link }} inside .