Support Unicode block codes

[kdmccormick]

Background

Block codes (aka slugs, aka block_ids, aka url_names, aka component_codes or container_codes) have historically been restricted to ascii slug characters (A-Za-z0-9_-). But, we recently discovered that we have some partial support for non-ascii characters in block codes in V2 content libraries: #38402 (comment). Specifically, non-ascii alphanumeric unicode characters, i.e. the character class accepted by re.match(r'\w', unicode=True) in Python: https://docs.python.org/3/library/re.html**

Currently, as far as we know, you can only get a block with a non-ascii code by doing one of the following:

• Creating a container (unit, subsection, section) with an initial title that includes non-ascii characters (in a content library)
• Editing a component or container's key with the library backup ZIP, and then restoring that backup.
• Creating a component or container with a non-ascii alphanumeric character (e.g., "θ") in the title and then copy-pasting that component into a content library. The original item won't have a non-ascii code, but the pasted item will.

In the long term we'd like to support such non-ascii codes, but in the short term we're concerned about downstream effects of those codes making into systems which were designed without unicode in mind, like:

• modulestore
• courseware student module
• analytics

Acceptance criteria

• Research the effect of non-ascii alphanumeric chars on those downstream systems
• Decide which systems we feel are safe for such characters
• Document this decision
• Loosen UX-level validation as to allow for them
• Do the same for: library codes, course codes, org codes. Or create ticket(s) for them.

[kdmccormick]

fyi @ormsbee , @bradenmacdonald

[bradenmacdonald]

@kdmccormick I added one more very simple way to get this in the current UI:

Creating a container (unit, subsection, section) with an initial title that includes non-ascii characters

It seems that v2 content library codes/slugs were clearly designed to support unicode codes as well, but it isn't quite working in the Authoring MFE's "Create Library" UI for some reason, despite the error message saying it should work.

openedx-platform/openedx/core/djangoapps/content_libraries/models.py

Line 72 in 91d3174

slug = models.SlugField(allow_unicode=True)

openedx-platform/openedx/core/djangoapps/content_libraries/tests/test_content_libraries.py

Lines 83 to 85 in 91d3174

	lib = self._create_library(
	slug="téstlꜟط", title="A Tést Lꜟطrary", description="Just Téstꜟng", license_type=CC_4_BY,
	)

[bradenmacdonald]

@kdmccormick Also, I don't really see a need to research "modulestore" and CSM support for these? Libraries using openedx_content already work perfectly with unicode slugs, and have long had at least a couple test cases verifying that. When we move courses to openedx_content, we can support unicode slugs in courses too, but I don't think we need to update modulestore.

[bradenmacdonald]

Somewhat related: openedx/frontend-app-authoring#3015

[kdmccormick]

@bradenmacdonald yeah, I am less-so thinking about unicode block codes for modulestore blocks, and moreso thinking about situations where library usage keys end up as moduelstore metadata, like the upstream field.

Honestly, it might be completely fine. I'd just like us to take a moment to think it through and try to imagine edge cases which our smoke-testing won't necessarily hit.