fix: correct 6 diacritization bugs found in 83K poem corpus#74

Open

lesmartiepants wants to merge 8 commits into

linuxscout:masterfrom

lesmartiepants:fix/tashkeel-bugs

Commits on Mar 7, 2026

fix(tashkeel): increase default word limit from 1000 to 20000

Siraj Farage
and
claude
committed
fix(vocalization): remove duplicate diacritics and impossible hamza+sukun

Siraj Farage
and
claude
committed
fix(vocalization): fix missing diacritics in vocalize_foreign for unknown words

Siraj Farage
and
claude
committed
fix(vocalization): correct spurious shadda on function words

Siraj Farage
and
claude
committed
fix(vocalization): add sun letter assimilation after definite article

Siraj Farage
and
claude
committed
test(vocalization): add 25-test suite verifying all 5 tashkeel fixes

Siraj Farage
and
claude
committed
feat(pipeline): add tashkeel diacritization pipeline and investigation report

Siraj Farage
and
claude
committed
docs(pipeline): add INSTRUCTIONS.md with setup, usage, and common cases

Siraj Farage
and
claude
committed