Context
PR #101 shipped dual-model embedding support. The infrastructure is complete:
embed_model frontmatter field routes ways to EN or multilingual model
- Dual corpus generation (
ways-corpus-en.jsonl + ways-corpus-multi.jsonl)
- Scanner queries both corpora and merges results
languages.json defines 52 supported languages
test-multilingual.sh validates cross-language similarity
ways language shows coverage report
What's missing: actual language stubs. All 85+ ways are English-only (embed_model: en). The multilingual corpus is empty.
Test data from this session
Same-language matching with native stubs dramatically outperforms cross-language:
| Lang |
EN model × EN desc |
Multi model × EN desc |
Multi model × native desc |
| en |
0.76 |
0.68 |
0.68 |
| ja |
-0.03 |
0.69 |
0.93 |
| ko |
-0.02 |
0.72 |
0.86 |
| zh |
-0.03 |
0.59 |
0.89 |
| de |
0.08 |
0.62 |
0.82 |
| ru |
0.03 |
0.76 |
0.85 |
| ar |
0.04 |
0.40 |
0.96 |
| el |
-0.01 |
0.59 |
0.81 |
Full report: docs/architecture/system/multilingual-model-evaluation.md
Stub format
Each stub is ~5 lines of frontmatter, no body:
---
description: セキュリティ脆弱性スキャンと監査
vocabulary: セキュリティ 脆弱性 CVE 監査 認証 暗号化
embed_model: multilingual
embed_threshold: 0.25
---
File naming: {wayname}.{lang}.md (e.g., security.ja.md)
Recommended starting languages
Cover the architecture tiers with 4 languages:
| Language |
Code |
BM25 |
Why |
| Japanese |
ja |
impossible |
CJK, highest same-language delta (+0.96), large user base |
| German |
de |
available |
BM25 compound noun challenges, tests both engines |
| Spanish |
es |
available |
Romance language, clean BM25 fit, large user base |
| Arabic |
ar |
impossible |
Non-concatenative morphology, highest same-language score (0.96) |
Team execution plan
This is ideal for an agent team (4 teammates, one per language):
Lead: Creates task list, validates dual corpus after stubs land, runs test-multilingual.sh
Per-language teammate:
- For each of 85+ ways, create
{wayname}.{lang}.md with:
description: — natural expression in the target language (not mechanical translation)
vocabulary: — domain terms in the target language
embed_model: multilingual
embed_threshold: — tuned per-way (start at 0.25, adjust based on test results)
- Run
ways corpus to regenerate dual corpora
- Run
bash tools/way-embed/test-multilingual.sh to validate
- Run
ways language --filter {lang} to verify coverage
Reviewer: Verify native descriptions are natural, not mechanical translations
Acceptance criteria
References
- ADR-107 (rewritten):
docs/architecture/system/ADR-107-way-match-corpus-batch-mode-and-locale-support.md
- Language guide:
docs/hooks-and-ways/languages.md
- Evaluation report:
docs/architecture/system/multilingual-model-evaluation.md
- Test suite:
tools/way-embed/test-multilingual.sh
Context
PR #101 shipped dual-model embedding support. The infrastructure is complete:
embed_modelfrontmatter field routes ways to EN or multilingual modelways-corpus-en.jsonl+ways-corpus-multi.jsonl)languages.jsondefines 52 supported languagestest-multilingual.shvalidates cross-language similarityways languageshows coverage reportWhat's missing: actual language stubs. All 85+ ways are English-only (
embed_model: en). The multilingual corpus is empty.Test data from this session
Same-language matching with native stubs dramatically outperforms cross-language:
Full report:
docs/architecture/system/multilingual-model-evaluation.mdStub format
Each stub is ~5 lines of frontmatter, no body:
File naming:
{wayname}.{lang}.md(e.g.,security.ja.md)Recommended starting languages
Cover the architecture tiers with 4 languages:
Team execution plan
This is ideal for an agent team (4 teammates, one per language):
Lead: Creates task list, validates dual corpus after stubs land, runs
test-multilingual.shPer-language teammate:
{wayname}.{lang}.mdwith:description:— natural expression in the target language (not mechanical translation)vocabulary:— domain terms in the target languageembed_model: multilingualembed_threshold:— tuned per-way (start at 0.25, adjust based on test results)ways corpusto regenerate dual corporabash tools/way-embed/test-multilingual.shto validateways language --filter {lang}to verify coverageReviewer: Verify native descriptions are natural, not mechanical translations
Acceptance criteria
ways-corpus-multi.jsonlis non-empty afterways corpustest-multilingual.shpasses for all target languagesways languageshows correct coverageembed_thresholdtuned (no false positives, reasonable recall)ways lint --globalpasses with no errorsReferences
docs/architecture/system/ADR-107-way-match-corpus-batch-mode-and-locale-support.mddocs/hooks-and-ways/languages.mddocs/architecture/system/multilingual-model-evaluation.mdtools/way-embed/test-multilingual.sh