Skip to content

Commit bb12bb7

Browse files
oguzkocerjkmassel
andauthored
Add a daily translation-sync job (#23001)
* Add daily translation-sync job (Buildkite + fastlane) New update_translations lane downloads the latest WordPress & Jetpack translations from GlotPress and opens/refreshes a single rolling translations/daily-update PR via release-toolkit's find_or_create_pull_request. A new scheduled Buildkite pipeline runs it daily. * Fix translation-sync git identity and use proper git helpers - download-translations.sh: 'source use-bot-for-git' (like the release committing lanes) so the agent has a git identity/auth; drop configure_apply (not needed for translations). - update_translations lane: use GitHelper.create_branch / point_to_same_commit? and push_to_git_remote instead of raw git sh calls, mirroring release.rb. * Run translation-sync on mac-metal (where use-bot-for-git is available) The job commits, pushes and opens a PR, which needs the bot git identity from use-bot-for-git — only available on mac-metal agents, same as the release code-freeze/finalize lanes. (android only builds, so release-builds.yml stays there.) * Disable translation-sync failure notifications Comment out the notify block; the channel is pre-set to #wpmobile for if/when it is re-enabled. * Prune orphaned translations as a separate commit in update_translations After downloading, prune keys no longer in the source strings (main/res vs main; jetpack/res vs main+jetpack) via android_prune_orphaned_translations and commit it separately, so the PR shows exactly what was pruned vs downloaded. * Send translation-sync failure notifications to the test Slack channel Re-enable the notify block, routed to #test-wpmobile-slack-integration while this flow is validated, mirroring the beta-promotion pipeline. Moves to #wpmobile once it ships to real use. * Align download-translations.sh with sibling command-script style Use `#!/usr/bin/env bash` + `set -eu` (the more robust shebang recommended in review) and the `:rubygems: Setting up Gems` header, matching build-trunk-internal.sh, gather-beta-candidates.sh and promote-to-beta.sh. * Update daily translation lane to warn against manual changes Co-authored-by: Jeremy Massel <1123407+jkmassel@users.noreply.github.com> * Update daily translation lane to abort if the branch can't be checked out Co-authored-by: Jeremy Massel <1123407+jkmassel@users.noreply.github.com> * Use modifier unless for checkout_and_pull guard (RuboCop Style/IfUnlessModifier) --------- Co-authored-by: Jeremy Massel <1123407+jkmassel@users.noreply.github.com>
1 parent b7a8374 commit bb12bb7

3 files changed

Lines changed: 106 additions & 0 deletions

File tree

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
#!/usr/bin/env bash
2+
3+
set -eu
4+
5+
echo "--- :robot_face: Use bot for git operations"
6+
source use-bot-for-git
7+
8+
echo "--- :rubygems: Setting up Gems"
9+
install_gems
10+
11+
echo "--- :globe_with_meridians: Download translations and open/update the PR"
12+
bundle exec fastlane update_translations
Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
# yaml-language-server: $schema=https://raw.githubusercontent.com/buildkite/pipeline-schema/main/schema.json
2+
---
3+
4+
# Runs on a daily schedule (configured in the Buildkite UI, ~06:00 UTC) to sync the latest
5+
# translations from GlotPress into a single rolling `translations/daily-update` PR.
6+
#
7+
# Runs on `mac-metal` (like the release code-freeze/finalize lanes) because this job commits,
8+
# pushes and opens a PR, which needs the bot git identity from `use-bot-for-git` — that command
9+
# is only available on those agents.
10+
11+
agents:
12+
queue: "mac-metal"
13+
14+
steps:
15+
- label: ":globe_with_meridians: Download Translations"
16+
command: .buildkite/commands/download-translations.sh
17+
plugins: [$CI_TOOLKIT]
18+
19+
# Slack backstop for a failed sync — its unique value is failures that abort before the lane runs
20+
# (gems/secrets) and can't self-report. Channel intentionally stays the test channel while this flow
21+
# is validated (mirrors promote-beta.yml + the Fastfile `send_slack_message` default); moves to
22+
# #wpmobile once it ships to real use.
23+
notify:
24+
- slack:
25+
channels:
26+
- "#test-wpmobile-slack-integration"
27+
message: "Failed to sync translations from GlotPress."
28+
if: build.state == "failed"

fastlane/lanes/localization.rb

Lines changed: 66 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -375,6 +375,72 @@
375375
)
376376
end
377377

378+
TRANSLATIONS_SYNC_BRANCH = 'translations/daily-update'
379+
380+
#####################################################################################
381+
# update_translations
382+
# -----------------------------------------------------------------------------------
383+
# Downloads the latest WordPress & Jetpack translations from GlotPress and opens (or
384+
# refreshes) a single rolling Pull Request, so `trunk` stays continuously localized.
385+
# Intended to run on a daily schedule.
386+
#
387+
# Each run resets `translations/daily-update` to `trunk` and re-downloads, so the PR
388+
# always shows the complete current translation delta against `trunk` (no accumulation).
389+
# If GlotPress has nothing new, no commit is made and the lane exits without a PR.
390+
# -----------------------------------------------------------------------------------
391+
# Usage:
392+
# bundle exec fastlane update_translations
393+
#####################################################################################
394+
lane :update_translations do
395+
# `checkout_and_pull` swallows git errors and only signals failure through its return value, so
396+
# abort explicitly rather than silently building the sync branch from a stale/wrong `trunk`.
397+
UI.user_error!("Could not check out and pull #{DEFAULT_BRANCH}; aborting translation sync.") unless Fastlane::Helper::GitHelper.checkout_and_pull(DEFAULT_BRANCH)
398+
399+
# Reset the rolling branch to the tip of `trunk` so each run produces a clean delta against it.
400+
Fastlane::Helper::GitHelper.delete_local_branch_if_exists!(TRANSLATIONS_SYNC_BRANCH)
401+
Fastlane::Helper::GitHelper.create_branch(TRANSLATIONS_SYNC_BRANCH, from: DEFAULT_BRANCH)
402+
403+
download_translations
404+
405+
# `download_translations` commits when GlotPress has updates; if nothing changed, HEAD still
406+
# points at `trunk` and there is nothing to open a PR for.
407+
if Fastlane::Helper::GitHelper.point_to_same_commit?(DEFAULT_BRANCH, 'HEAD')
408+
UI.important('No new translations from GlotPress today; nothing to sync.')
409+
next
410+
end
411+
412+
# Prune translations whose key is no longer in the source strings (GlotPress can still serve them),
413+
# which would otherwise fail Lint's `ExtraTranslation` check. Done as a separate commit on top of the
414+
# download so the PR shows exactly what was pruned vs. what was downloaded.
415+
main_res = File.join('WordPress', 'src', 'main', 'res')
416+
jetpack_res = File.join('WordPress', 'src', 'jetpack', 'res')
417+
android_prune_orphaned_translations(res_dir: main_res)
418+
android_prune_orphaned_translations(
419+
res_dir: jetpack_res,
420+
additional_source_strings_paths: [File.join(main_res, 'values', 'strings.xml')]
421+
)
422+
Fastlane::Helper::GitHelper.commit(message: 'Prune orphaned translations', files: :all)
423+
424+
push_to_git_remote(remote_branch: TRANSLATIONS_SYNC_BRANCH, tags: false, force: true, set_upstream: true)
425+
426+
# `find_or_create_pull_request` resolves the GitHub token the standard way (GITHUB_TOKEN) and only
427+
# opens a PR when none is already open; the force-push above already refreshed any existing one.
428+
pr_url = find_or_create_pull_request(
429+
repository: GITHUB_REPO,
430+
title: 'Update translations',
431+
body: <<~BODY,
432+
Automated daily translation sync from GlotPress. Opened by the `download-translations` scheduled pipeline.
433+
434+
⚠️ This branch is reset to `trunk` and force-pushed daily — do not push commits here;
435+
they will be overwritten. Fix translations in GlotPress instead.
436+
BODY
437+
head: TRANSLATIONS_SYNC_BRANCH,
438+
base: DEFAULT_BRANCH,
439+
labels: ['Localization']
440+
)
441+
UI.success("Translations PR: #{pr_url}")
442+
end
443+
378444
# Updates the `.po` file at the given `po_path` using the content of the `sources` files,
379445
# interpolating `release_version` where appropriate.
380446
# Internally, this calls the `gp_update_metadata_source` release toolkit action and adds Git management to it.

0 commit comments

Comments
 (0)