Skip to content

Create multilingual way stubs for core languages #102

Description

@aaronsb

Context

PR #101 shipped dual-model embedding support. The infrastructure is complete:

  • embed_model frontmatter field routes ways to EN or multilingual model
  • Dual corpus generation (ways-corpus-en.jsonl + ways-corpus-multi.jsonl)
  • Scanner queries both corpora and merges results
  • languages.json defines 52 supported languages
  • test-multilingual.sh validates cross-language similarity
  • ways language shows coverage report

What's missing: actual language stubs. All 85+ ways are English-only (embed_model: en). The multilingual corpus is empty.

Test data from this session

Same-language matching with native stubs dramatically outperforms cross-language:

Lang EN model × EN desc Multi model × EN desc Multi model × native desc
en 0.76 0.68 0.68
ja -0.03 0.69 0.93
ko -0.02 0.72 0.86
zh -0.03 0.59 0.89
de 0.08 0.62 0.82
ru 0.03 0.76 0.85
ar 0.04 0.40 0.96
el -0.01 0.59 0.81

Full report: docs/architecture/system/multilingual-model-evaluation.md

Stub format

Each stub is ~5 lines of frontmatter, no body:

---
description: セキュリティ脆弱性スキャンと監査
vocabulary: セキュリティ 脆弱性 CVE 監査 認証 暗号化
embed_model: multilingual
embed_threshold: 0.25
---

File naming: {wayname}.{lang}.md (e.g., security.ja.md)

Recommended starting languages

Cover the architecture tiers with 4 languages:

Language Code BM25 Why
Japanese ja impossible CJK, highest same-language delta (+0.96), large user base
German de available BM25 compound noun challenges, tests both engines
Spanish es available Romance language, clean BM25 fit, large user base
Arabic ar impossible Non-concatenative morphology, highest same-language score (0.96)

Team execution plan

This is ideal for an agent team (4 teammates, one per language):

Lead: Creates task list, validates dual corpus after stubs land, runs test-multilingual.sh

Per-language teammate:

  1. For each of 85+ ways, create {wayname}.{lang}.md with:
    • description: — natural expression in the target language (not mechanical translation)
    • vocabulary: — domain terms in the target language
    • embed_model: multilingual
    • embed_threshold: — tuned per-way (start at 0.25, adjust based on test results)
  2. Run ways corpus to regenerate dual corpora
  3. Run bash tools/way-embed/test-multilingual.sh to validate
  4. Run ways language --filter {lang} to verify coverage

Reviewer: Verify native descriptions are natural, not mechanical translations

Acceptance criteria

  • All 85+ ways have stubs for each target language
  • ways-corpus-multi.jsonl is non-empty after ways corpus
  • test-multilingual.sh passes for all target languages
  • ways language shows correct coverage
  • Per-way embed_threshold tuned (no false positives, reasonable recall)
  • ways lint --global passes with no errors

References

  • ADR-107 (rewritten): docs/architecture/system/ADR-107-way-match-corpus-batch-mode-and-locale-support.md
  • Language guide: docs/hooks-and-ways/languages.md
  • Evaluation report: docs/architecture/system/multilingual-model-evaluation.md
  • Test suite: tools/way-embed/test-multilingual.sh

Metadata

Metadata

Assignees

No one assigned

    Labels

    contentWay content, stubs, vocabulary, and test fixturesi18nMulti-language support and localizationteam-taskDesigned for agent team execution

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions