Skip to content

Commit 586df38

Browse files
committed
Add String Catalog plural pipeline (Plurals.xcstrings ⇄ GlotPress)
Plurals are authored in WordPress/Classes/Plurals.xcstrings (English one/other) and carried through the main app GlotPress project as flat `<key>|==|plural.<cldr-category>` originals — the same id `xcodebuild -exportLocalizations` uses — so every locale, including Welsh (6 forms), is covered. Forward (no build): catalog → flat `.strings` originals for the category union, merged into Localizable.strings (like MANUALLY_MAINTAINED_STRINGS_FILES) so they upload with the app strings; the lane fails loudly if a plural is missing its English `other`. Reverse (no build): the flat keys are read back out of the downloaded Localizable.strings and folded into the catalog JSON via a committed per-locale CLDR category map — each cell `human ?? AI ?? English`, with AI / English-fallback cells flagged `needs_review`. The map is regenerated from Apple's exporter by `refresh_plural_categories`, the one build-backed lane. `.strings` reading is delegated to the release toolkit's `read_strings_file_as_hash` (plutil); the flat keys stay in Localizable.strings as harmless, unused-at-runtime entries. Wired into generate_strings_file_for_glotpress / download_localized_strings: runs in parallel with the app-strings pipeline before cutover (failsafe — a plural error is logged, never raised), with nothing consuming the result at runtime yet. PluralStrings is pure Ruby and unit-tested.
1 parent 45e3058 commit 586df38

6 files changed

Lines changed: 516 additions & 1 deletion

File tree

.gitignore

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -136,3 +136,11 @@ WordPress/Frameworks/*.tar.gz
136136
WordPress/Frameworks/react-native-bundle-source-map
137137

138138
Tests/AgentTests/results
139+
140+
# Generated during localization, regenerated each CI run — not committed:
141+
# - Plurals.strings: transient flat plural originals (derived from Plurals.xcstrings, merged into
142+
# Localizable.strings at code freeze, then discarded).
143+
# - Localizable.xcstrings: the build-free String Catalog (parity-verified by CI), shelved until the cutover
144+
# commits it as the runtime backing store at a build-member path.
145+
WordPress/Resources/Plurals.strings
146+
WordPress/Resources/Localizable.xcstrings
Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
{
2+
"sourceLanguage" : "en",
3+
"strings" : {
4+
"blogging.reminders.weeklyCount" : {
5+
"comment" : "Number of blogging reminders per week. %1$d is the count. Initial migration from BloggingRemindersScheduleFormatter; see https://github.com/wordpress-mobile/WordPress-iOS/issues/6327",
6+
"localizations" : {
7+
"en" : {
8+
"variations" : {
9+
"plural" : {
10+
"one" : {
11+
"stringUnit" : { "state" : "translated", "value" : "%1$d time a week" }
12+
},
13+
"other" : {
14+
"stringUnit" : { "state" : "translated", "value" : "%1$d times a week" }
15+
}
16+
}
17+
}
18+
}
19+
}
20+
},
21+
"editor.textCounter.wordCount" : {
22+
"comment" : "Number of words in the editor. %1$ld is the count. Initial migration from AztecPostViewController; see https://github.com/wordpress-mobile/WordPress-iOS/issues/6327",
23+
"localizations" : {
24+
"en" : {
25+
"variations" : {
26+
"plural" : {
27+
"one" : {
28+
"stringUnit" : { "state" : "translated", "value" : "%1$ld word" }
29+
},
30+
"other" : {
31+
"stringUnit" : { "state" : "translated", "value" : "%1$ld words" }
32+
}
33+
}
34+
}
35+
}
36+
}
37+
}
38+
},
39+
"version" : "1.0"
40+
}
Lines changed: 148 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,148 @@
1+
{
2+
"ar": [
3+
"zero",
4+
"one",
5+
"two",
6+
"few",
7+
"many",
8+
"other"
9+
],
10+
"bg": [
11+
"one",
12+
"other"
13+
],
14+
"cs": [
15+
"one",
16+
"few",
17+
"many",
18+
"other"
19+
],
20+
"cy": [
21+
"zero",
22+
"one",
23+
"two",
24+
"few",
25+
"many",
26+
"other"
27+
],
28+
"da": [
29+
"one",
30+
"other"
31+
],
32+
"de": [
33+
"one",
34+
"other"
35+
],
36+
"en-AU": [
37+
"one",
38+
"other"
39+
],
40+
"en-CA": [
41+
"one",
42+
"other"
43+
],
44+
"en-GB": [
45+
"one",
46+
"other"
47+
],
48+
"es": [
49+
"one",
50+
"other"
51+
],
52+
"fr": [
53+
"one",
54+
"other"
55+
],
56+
"he": [
57+
"one",
58+
"two",
59+
"many",
60+
"other"
61+
],
62+
"hr": [
63+
"one",
64+
"few",
65+
"other"
66+
],
67+
"hu": [
68+
"one",
69+
"other"
70+
],
71+
"id": [
72+
"other"
73+
],
74+
"is": [
75+
"one",
76+
"other"
77+
],
78+
"it": [
79+
"one",
80+
"other"
81+
],
82+
"ja": [
83+
"other"
84+
],
85+
"ko": [
86+
"other"
87+
],
88+
"nb": [
89+
"one",
90+
"other"
91+
],
92+
"nl": [
93+
"one",
94+
"other"
95+
],
96+
"pl": [
97+
"one",
98+
"few",
99+
"many",
100+
"other"
101+
],
102+
"pt": [
103+
"one",
104+
"other"
105+
],
106+
"pt-BR": [
107+
"one",
108+
"other"
109+
],
110+
"ro": [
111+
"one",
112+
"few",
113+
"other"
114+
],
115+
"ru": [
116+
"one",
117+
"few",
118+
"many",
119+
"other"
120+
],
121+
"sk": [
122+
"one",
123+
"few",
124+
"many",
125+
"other"
126+
],
127+
"sq": [
128+
"one",
129+
"other"
130+
],
131+
"sv": [
132+
"one",
133+
"other"
134+
],
135+
"th": [
136+
"other"
137+
],
138+
"tr": [
139+
"one",
140+
"other"
141+
],
142+
"zh-Hans": [
143+
"other"
144+
],
145+
"zh-Hant": [
146+
"other"
147+
]
148+
}

fastlane/lanes/localization.rb

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -181,14 +181,30 @@
181181

182182
# Merge various manually-maintained `.strings` files into the previously generated `Localizable.strings` so their extra keys are also imported in GlotPress.
183183
# Note: We will re-extract the translations back during `download_localized_strings_and_metadata` (via a call to `ios_extract_keys_from_strings_files`)
184+
# Generate the flat plural originals and merge them into Localizable.strings so they ride the main app
185+
# GlotPress project alongside everything else.
186+
paths_to_merge = MANUALLY_MAINTAINED_STRINGS_FILES.dup
187+
run_plural_step('forward') do
188+
generate_plural_strings_for_glotpress
189+
paths_to_merge[PLURALS_FLAT_STRINGS] = '' # flat keys are self-qualified; no prefix
190+
end
191+
184192
ios_merge_strings_files(
185-
paths_to_merge: MANUALLY_MAINTAINED_STRINGS_FILES,
193+
paths_to_merge: paths_to_merge,
186194
destination: File.join(WORDPRESS_EN_LPROJ, 'Localizable.strings')
187195
)
188196

189197
git_commit(path: [WORDPRESS_EN_LPROJ], message: 'Update strings for localization', allow_nothing_to_commit: true) unless skip_commit
190198
end
191199

200+
# Runs a plural-pipeline step, never fatally — a failure is logged so the parallel plural run (whose result
201+
# isn't consumed at runtime until cutover) can't break a release.
202+
def run_plural_step(label)
203+
yield
204+
rescue StandardError => e
205+
UI.error("Plural pipeline (#{label}) failed; continuing: #{e.message}")
206+
end
207+
192208
def generate_strings_file(gutenberg_path:, derived_data_path:)
193209
ios_generate_strings_file_from_code(
194210
paths: [
@@ -391,6 +407,9 @@ def generate_strings_file(gutenberg_path:, derived_data_path:)
391407
message: 'Update app translations – Other `.strings`',
392408
allow_nothing_to_commit: true
393409
)
410+
411+
# Fold plural translations from the downloaded Localizable.strings into Plurals.xcstrings.
412+
run_plural_step('reverse') { download_localized_plurals }
394413
end
395414

396415
# Downloads the localized metadata (for App Store Connect) from GlotPress for the WordPress app.
Lines changed: 132 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,132 @@
1+
# frozen_string_literal: true
2+
3+
require_relative 'plural_strings_helper'
4+
5+
#################################################
6+
# Plurals: String Catalog ⇄ GlotPress ("all-flat")
7+
#
8+
# Plurals are authored in `WordPress/Classes/Plurals.xcstrings` (English one/other) and carried through the
9+
# MAIN app GlotPress project as flat strings keyed `<key>|==|plural.<cldr-category>` — the same id Apple's
10+
# `xcodebuild -exportLocalizations` uses — so every locale (incl. Welsh) is covered. The forward merges these
11+
# flat originals into `Localizable.strings` (like MANUALLY_MAINTAINED_STRINGS_FILES); the reverse reads them
12+
# back out of the downloaded `Localizable.strings` and folds them into the catalog JSON (build-free) via the
13+
# committed per-locale category map. The flat keys stay in `Localizable.strings` as harmless, unused-at-runtime
14+
# entries — exactly like the merged `infoplist.*` keys. The exporter is consulted only by
15+
# `refresh_plural_categories` when the ship-locale list changes.
16+
#################################################
17+
18+
# Lives in a synchronized source folder (WordPress/Classes) so it auto-joins the WordPress target.
19+
# `WordPress/Resources` is an explicitly-referenced (non-synchronized) group, so a catalog placed
20+
# there is NOT a target member and would be skipped by `-exportLocalizations`.
21+
PLURALS_CATALOG = File.join(PROJECT_ROOT_FOLDER, 'WordPress', 'Classes', 'Plurals.xcstrings')
22+
PLURALS_FLAT_STRINGS = File.join(PROJECT_ROOT_FOLDER, 'WordPress', 'Resources', 'Plurals.strings') # transient merge input (not committed)
23+
# Per-locale CLDR category map ({ "<lproj>" => ["one","other",…] }): drives the build-free reverse (which slots
24+
# each locale needs) and the forward union (which originals to upload). Captured from Apple's exporter (CLDR)
25+
# over the 33 ship locales; regenerate with `refresh_plural_categories` when the supported-locale list changes.
26+
PLURAL_CATEGORIES_JSON = File.join(PROJECT_ROOT_FOLDER, 'WordPress', 'Resources', 'plural-categories.json')
27+
PLURALS_SCHEME = 'WordPress'
28+
29+
platform :ios do
30+
# FORWARD (no build): Plurals.xcstrings (English) -> flat "<key>|==|plural.<cat>" originals (a transient
31+
# `.strings` that `generate_strings_file_for_glotpress` merges into Localizable.strings for the main project).
32+
#
33+
# Called by generate_strings_file_for_glotpress (its originals merge into Localizable.strings).
34+
desc 'Generates the flat plural originals (.strings) merged into Localizable.strings for GlotPress'
35+
lane :generate_plural_strings_for_glotpress do
36+
catalog = JSON.parse(File.read(PLURALS_CATALOG))
37+
categories = PluralStrings.union_categories(JSON.parse(File.read(PLURAL_CATEGORIES_JSON)))
38+
39+
missing = PluralStrings.plural_keys_missing_other(catalog)
40+
unless missing.empty?
41+
UI.user_error!("Plurals.xcstrings: plural(s) missing a non-empty English `other` form (CLDR requires it — without it they upload empty originals): #{missing.join(', ')}")
42+
end
43+
44+
originals = PluralStrings.flat_originals(catalog, categories)
45+
File.write(PLURALS_FLAT_STRINGS, PluralStrings.serialize_legacy_strings(originals))
46+
UI.message("Generated #{originals.size} flat plural originals from #{catalog['strings'].size} catalog keys → #{PLURALS_FLAT_STRINGS}")
47+
end
48+
49+
# REVERSE (no build): pull the flat plural translations back out of the already-downloaded app
50+
# `Localizable.strings` (they rode the main GlotPress project) and fold them straight into Plurals.xcstrings
51+
# JSON, using the committed per-locale category map. Each cell is human ?? AI ?? English source; machine and
52+
# English-fallback cells are flagged needs_review.
53+
#
54+
# Called by download_localized_strings, after the app strings are downloaded.
55+
desc 'Folds plural translations from the downloaded Localizable.strings into Plurals.xcstrings'
56+
lane :download_localized_plurals do
57+
catalog = JSON.parse(File.read(PLURALS_CATALOG))
58+
missing = PluralStrings.plural_keys_missing_other(catalog)
59+
UI.user_error!("Plurals.xcstrings: plural(s) missing a non-empty English `other` form (CLDR requires it): #{missing.join(', ')}") unless missing.empty?
60+
categories_by_locale = JSON.parse(File.read(PLURAL_CATEGORIES_JSON))
61+
62+
written = PluralStrings.fold_translations!(
63+
catalog,
64+
categories_by_locale: categories_by_locale,
65+
translations_by_locale: plural_translations_by_locale(File.join(PROJECT_ROOT_FOLDER, 'WordPress', 'Resources')),
66+
ai_translator: method(:ai_translate_plural)
67+
)
68+
File.write(PLURALS_CATALOG, "#{JSON.pretty_generate(catalog)}\n")
69+
UI.message("Folded plural translations from Localizable.strings into #{File.basename(PLURALS_CATALOG)} (#{written} locale variations).")
70+
71+
git_commit(path: [PLURALS_CATALOG], message: 'Update plural translations from GlotPress', allow_nothing_to_commit: true)
72+
end
73+
74+
# Regenerate the per-locale CLDR category map (`plural-categories.json`) from Apple's exporter over the ship
75+
# locales. Run only when the supported-locale list changes — the one place the exporter is used. Build-backed.
76+
desc 'Refreshes plural-categories.json (per-locale CLDR sets) from the exporter over the ship locales'
77+
lane :refresh_plural_categories do
78+
categories_by_locale = export_plural_skeletons(GLOTPRESS_TO_LPROJ_APP_LOCALE_CODES.values.uniq) do |skeleton_dir|
79+
paths = Dir.glob(File.join(skeleton_dir, '*.xcloc', 'Localized Contents', '*.xliff'))
80+
PluralStrings.categories_by_locale_from_skeletons(paths)
81+
end
82+
UI.user_error!('No plural categories found — is there a plural in Plurals.xcstrings?') if categories_by_locale.empty?
83+
84+
File.write(PLURAL_CATEGORIES_JSON, "#{JSON.pretty_generate(categories_by_locale)}\n")
85+
UI.success("Wrote plural categories for #{categories_by_locale.size} locales: #{categories_by_locale.keys.sort.join(', ')}")
86+
end
87+
88+
#################################################
89+
# Helpers
90+
#################################################
91+
92+
# Exports the per-locale plural skeletons (one build) into a temp dir and yields it, removing the dir when
93+
# the block returns. Returns whatever the block returns.
94+
#
95+
# `SUPPORTS_MACCATALYST=NO` constrains the string-extraction build to iOS. Without it,
96+
# `-exportLocalizations` builds every supported destination incl. Mac Catalyst, which fails
97+
# when a binary dependency (e.g. a Zendesk xcframework) ships no `maccatalyst` slice.
98+
def export_plural_skeletons(xcode_locales)
99+
Dir.mktmpdir do |out|
100+
sh(
101+
'xcodebuild', '-exportLocalizations',
102+
'-workspace', WORKSPACE_PATH,
103+
'-scheme', PLURALS_SCHEME,
104+
'-localizationPath', out,
105+
'SUPPORTS_MACCATALYST=NO',
106+
*xcode_locales.flat_map { |loc| ['-exportLanguage', loc] }
107+
)
108+
yield out
109+
end
110+
end
111+
112+
# Pulls the flat plural keys out of each locale's downloaded `Localizable.strings`, returning
113+
# { "<lproj>" => { "<flat-plural-id>" => value } }. Decoding (escapes like `\n`/`\U…`, encoding/BOM) is
114+
# delegated to `L10nHelper.read_strings_file_as_hash` — Apple's `plutil` — rather than a hand-rolled parser.
115+
def plural_translations_by_locale(dir)
116+
Dir.glob(File.join(dir, '*.lproj', 'Localizable.strings')).each_with_object({}) do |path, acc|
117+
locale = File.basename(File.dirname(path), '.lproj')
118+
translations = Fastlane::Helper::Ios::L10nHelper.read_strings_file_as_hash(path: path)
119+
acc[locale] = translations.select { |key, _| PluralStrings.plural_key?(key) }
120+
end
121+
end
122+
123+
# Machine-translation floor for the reverse fold: invoked for every plural slot with no human translation.
124+
# Returns nil until wired to a translation service, leaving such slots to fall back to the English source
125+
# (flagged needs_review). The named `category` + dev `note` let the prompt request the correct grammatical
126+
# form (e.g. "give me the Polish *few* form of …").
127+
# rubocop:disable Lint/UnusedMethodArgument -- keyword names are the documented call contract
128+
def ai_translate_plural(id:, source:, category:, note:, locale:)
129+
nil # TODO: call the translation service.
130+
end
131+
# rubocop:enable Lint/UnusedMethodArgument
132+
end

0 commit comments

Comments
 (0)