Transliterate Liquid Filter

[urbanit]

Add transliterate Liquid Filter

Based on https://github.com/orgs/OrchardCMS/discussions/10748 and fixes #4151 and fixes #4874

Summary

Introduces a new transliterate Liquid filter to the OrchardCore.Liquid module. It converts Unicode text (e.g. Greek,
Cyrillic, Arabic, Chinese, etc.) into its closest ASCII/Latin representation using the
https://github.com/anyascii/anyascii library.

Changes

• Added AnyAscii (v0.3.2) package to Directory.Packages.props
• Added TransliterateFilter.cs implementing ILiquidFilter
• Registered the filter as transliterate in Startup.cs

Usage

Apply the filter to any string value in a Liquid template:

{{ "Ελληνικά" | transliterate }}
{# Output: Ellinika #}

{{ "Привет мир" | transliterate }}
{# Output: Privet mir #}

{{ "日本語" | transliterate }}
{# Output: Ri Ben Yu #}

Chaining with slugify

A common use case is generating URL slugs from non-Latin content — transliterate first, then slugify:

{{ ContentItem.DisplayText | transliterate | slugify }}

This is especially useful for auto-generating URL aliases for content items with titles in non-Latin scripts.

[hishamco]

You remind me of the work that I did in Orchard Core Contrib (OCC) https://github.com/OrchardCoreContrib/OrchardCoreContrib/tree/main/src/OrchardCoreContrib.ContentLocalization.Abstractions/Transliteration 2 years ago

Please show me some usage examples before I review this

[Piedone]

See the PR description for examples.

[hishamco]

See the PR description for examples.

Thanks Zoltan, seems I'm blind at the beginning :)

@urbanit, how has the script been specified in your case? If you look into the related repo or the demo https://www.youtube.com/watch?v=MEmNL5tzezA, you will notice that we used a sort of script to transliterate

[sebastienros]

He's back! (never left our hearts though)

[urbanit]

Added:

1. Slugify filter enriched with transliteration option
2. MediaSlufigy functionality enriched with transliteration option

[hishamco]

I was waiting for @urbanit to answer my question #19009 (comment) since last month

[Piedone]

Ah, sorry!

[urbanit]

See the PR description for examples.

Thanks Zoltan, seems I'm blind at the beginning :)

@urbanit, how has the script been specified in your case? If you look into the related repo or the demo https://www.youtube.com/watch?v=MEmNL5tzezA, you will notice that we used a sort of script to transliterate

Used https://github.com/anyascii/anyascii library

[hishamco]

Regardless of the library, what I mean is how to transilterate from Cyrillic to Latin ot vice versa, Arabic to Latin. I have never seen any option allow this, like what I mentioned in the above link

I remembered Seb commenting about this 2 years ago

[hishamco]

Seems the library is doing Unicode to ASCII

This is a simple use case that allows doing transilteration from one script to another https://www.lexilogos.com/keyboard/russian_conversion.htm

[urbanit]

Actually yes, target is always ASCIi/Latin, independent of the source, since it the most dominant script for URLs

[hishamco]

IMHO, this might work, but it's not extensible, another thing the transliterate as a service should be a part of the localization module

I will wait for @sebastienros comment, then we can enhance if we want to go with this

Thanks

[urbanit]

If you have a Greek site, without localization enabled, you need to transliterate the url otherwise it's not SEO friendly. So, my proposal is that it should be part of slug rather than localization.

[hishamco]

Let me correct my phrase; it should be a part of the localization APIs, not a module