refactor: Upgrade to openedx-core 0.44.0 (for OEP-68) by kdmccormick · Pull Request #38402 · openedx/openedx-platform

kdmccormick · 2026-04-21T18:24:21Z

Description

This incorporates a major set of renamings and key semantic changes,
agreed upon in OEP-68 [1] and released in openedx core v0.43.0 [2]

[1] https://docs.openedx.org/projects/openedx-proposals/en/latest/best-practices/oep-0068-bp-content-identifiers.html
[2] https://github.com/openedx/openedx-core/releases/tag/v0.43.0

Additionally, we now show unknown / unknown in the restore UI when
a learning package's package_ref cannot be parsed as a library key.
This is because we are no longer assuming any specific format for
package_refs. In the future, we may want a more graceful way handling
un-parseable package_refs, especially if we have learning packages
of courses instead of libraries.

Part of: openedx/openedx-core#322

Status

CI passes ✅
Claude wrote most of this, but I've reviewed all of it and made fixes (fix: various fixes to Claude's output...) ✅
Manually tested ✅

Testing

In libraries, I tested:

Backup
Restore
Restore with un-parseable package ref (unknown / unknown)
Creating components with non-ascii title (code is ascii)
Copying components with non-ascii title (code is non-ascii)
Creating containers with non-ascii title (code is non-ascii)
Creating collections with non-ascii title (code is ascii)
Publishing containers
Editing text with static assets
Editing problems
Editing ORA
Editing videos

I didn't test:

Courses
loncapa/python problems -- Codejail is not available on the sandbox. We'll want to be sure this is tested in the Verawood testing process.

AI Usage

Claude Plan: https://gist.github.com/kdmccormick/7316d5fe490763cf12e9687e6fb24db8

I basically gave Claude the release notes (linked above) and had it write the plan from that.

I additionally gave it the instructions to avoid any breaking changes to REST APIs or the libraries search index. That'll be handled later in #38406

kdmccormick · 2026-04-22T01:46:48Z

        admin = UserFactory.create(username="Admin", email="admin@example.com", is_staff=True)  # noqa: F841

-        lib = self._create_library(slug="téstlꜟط", title="A Tést Lꜟطrary", description="Tésting XBlocks")
+        lib = self._create_library(slug="testlib", title="A Tést Lꜟطrary", description="Tésting XBlocks")


@ormsbee @bradenmacdonald , our new code_field validation makes these non-ascii library and block slugs illegal. Is that fine?

I recall adding these non-ascii test slugs myself, way back in the Blockstore->LearningCore switchover. I don't know exactly what I was thinking--maybe I was just thinking that we'd made no statements about what could and couldn't be in a LearningCore slug, so we may as well test non-ascii slugs in order to see if they ever break.

@ormsbee @bradenmacdonald , our new code_field validation makes these non-ascii library and block slugs illegal. Is that fine?

Isn't it already illegal because it's put into an opaque key, and those don't support Unicode?

FWIW, in general I would prefer to have the ability to have non-ascii in the URLs, especially given our international reach. But that's something we can add in later in a backwards compatible way if we get support through all the plumbing.

Isn't it already illegal because it's put into an opaque key, and those don't support Unicode?

I don't know. This test passed, and it involves creating the library block, editing its OLX, and rendering it.

FWIW, in general I would prefer to have the ability to have non-ascii in the URLs, especially given our international reach. But that's something we can add in later in a backwards compatible way if we get support through all the plumbing.

Sounds good.

I just tested manually on master and we it seems that libraries code at least is perfectly happy to accept unicode slugs - I think it's something we tried to allow for. See these lines in the library opaque-keys code.

Would it be too difficult to update the new code fields to be unicode compatible?

Yeah, and now trying to migrate forward to test your PR, I get a check constraint violation.

django.db.utils.IntegrityError: (3819, "Check constraint 'oel_component_code_regex' is violated.")

@bradenmacdonald While I agree in principal with supporting unicode codes, I'm worried about the downstream effects of broadly allowing them. Just because they ostensibly work in openedx_content doesn't mean they wouldn't quietly break elsewhere (modulestore content, student state storage, analytics, so on). I wish we'd been more intentional about allowing them in the first place.

I think we should allow unicode in component_codes, container_codes, and entity_refs, since that's what the UI will allow you to do. But I think we should consider this support "provisional" rather than something we actively encourage, we should keep the existing ascii-only UI validation in place, and we should not yet allow unicode in library_codes or package_refs.

@kdmccormick Yeah, I'm not asking to broadly allow them, just to continue supporting them in any of the new areas of the system (i.e. v2 libraries) where we currently allow them.

Weirdly, the UI messaging indicates that unicode slugs should be allowed for library codes too, but it doesn't actually allow them. So it's fine to not allow that for now, I guess.

Also: this reflects our terrible lack of default content. Why doesn't every tutor devstack include sample content libraries with components that use unicode slugs? Then we'd be a lot more confident that they work.

https://github.com/openedx/openedx-core/releases/tag/v0.44.0 loosens the component and container constraints to allow non-ascii letters, and I've reverted the changes to this test so that we're testing non-ascii library slugs and block_ids again.

kdmccormick

Notes / questions / self-review

kdmccormick · 2026-04-22T15:18:43Z

    name: ${{ matrix.shard_name }}(py=${{ matrix.python-version }},dj=${{ matrix.django-version }},mongo=${{ matrix.mongo-version }})
    runs-on: ${{ matrix.os-version }}
    strategy:
+      fail-fast: false


Suggested change

fail-fast: false

Remove this before merging

kdmccormick · 2026-04-22T15:20:50Z

-            description=description,
-            created_by=created_by,
-        )
-    except IntegrityError as err:


Note for reviewers: Duplicates now raise a ValidationError instead of an IntegrityError. Catching ValidationError felt overly broad for this purpose, so I converted it into an if ...exists(): raise ...

It might be nice to introduce a more specific ValidationError subclass into core in the future, like CodeAlreadyExists, as I generally like the try...except pattern better than check-exists + create.

kdmccormick · 2026-04-22T15:57:39Z

+    def get_org(self, obj) -> str:
+        """
+        The org code/slug, as parsed from archive_package_ref, or "unknown" if unparseable.
+        """
+        return obj["lp_restored_data"]["archive_org_code"] or "unknown"
+
+    def get_slug(self, obj) -> str:
+        """
+        The library code/slug, as parsed from archive_package_ref, or "unknown" if unparseable.
+        """
+        return obj["lp_restored_data"]["archive_package_code"] or "unknown"
+


@ormsbee @bradenmacdonald , this sort of thing is necessary because we are no longer assuming in openedx-core that backup_restore always loads a library with LibraryLocatorV2 as its package_ref. So, archive_org_code and archive_package_code will come in as None if that package_ref parsing fails. I'm not sure what the right way to handle this is, but this here was the lowest-touch thing I think of. The only way to trigger this in practice would be to manually edit the value of package_ref in the archive.

I think the proper way to handle this, which may become relevant as we move Courses into Learning Packages, would be an error message that says "The package you're trying to restore is not a Content Library"

kdmccormick · 2026-04-22T16:02:08Z

        admin = UserFactory.create(username="Admin", email="admin@example.com", is_staff=True)  # noqa: F841

-        lib = self._create_library(slug="téstlꜟط", title="A Tést Lꜟطrary", description="Tésting XBlocks")
+        lib = self._create_library(slug="testlib", title="A Tést Lꜟطrary", description="Tésting XBlocks")


Isn't it already illegal because it's put into an opaque key, and those don't support Unicode?

I don't know. This test passed, and it involves creating the library block, editing its OLX, and rendering it.

FWIW, in general I would prefer to have the ability to have non-ascii in the URLs, especially given our international reach. But that's something we can add in later in a backwards compatible way if we get support through all the plumbing.

Sounds good.

bradenmacdonald · 2026-04-22T17:31:47Z

I'm getting an error when trying to save changes to a multiple choice problem component using the visual editor on this branch. Although the error looks unrelated, the issue does not occur on platform master 5935899 + core 10eae2da8d5e8cfc2d5b9a451687eaa9b4f40530 (before your rename changes)

cms-1       |   File "/openedx/edx-platform/openedx/core/djangoapps/xblock/rest_api/views.py", line 348, in post
cms-1       |     block.save()
cms-1       |   File "/mnt/xblock/xblock/core.py", line 515, in save
cms-1       |     self.runtime.save_block(self)
cms-1       |   File "/openedx/edx-platform/openedx/core/djangoapps/xblock/runtime/openedx_content_runtime.py", line 290, in save_block
cms-1       |     serialized = serialize_modulestore_block_for_openedx_content(block)
cms-1       |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
cms-1       |   File "/openedx/edx-platform/openedx/core/lib/xblock_serializer/api.py", line 35, in serialize_modulestore_block_for_openedx_content
cms-1       |     return XBlockSerializer(block, write_url_name=False, write_copied_from=False)
cms-1       |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
cms-1       |   File "/openedx/edx-platform/openedx/core/lib/xblock_serializer/block_serializer.py", line 41, in __init__
cms-1       |     self.olx_node = self._serialize_block(block)
cms-1       |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
cms-1       |   File "/openedx/edx-platform/openedx/core/lib/xblock_serializer/block_serializer.py", line 80, in _serialize_block
cms-1       |     olx = self._serialize_normal_block(block)
cms-1       |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
cms-1       |   File "/openedx/edx-platform/openedx/core/lib/xblock_serializer/block_serializer.py", line 124, in _serialize_normal_block
cms-1       |     block.add_xml_to_node(olx_node)
cms-1       |   File "/openedx/edx-platform/xmodule/xml_block.py", line 480, in add_xml_to_node
cms-1       |     for attr in sorted(own_metadata(self)):
cms-1       |                        ^^^^^^^^^^^^^^^^^^
cms-1       |   File "/openedx/edx-platform/xmodule/modulestore/inheritance.py", line 340, in own_metadata
cms-1       |     return block.get_explicitly_set_fields_by_scope(Scope.settings)
cms-1       |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
cms-1       | AttributeError: 'ProblemBlockWithMixins' object has no attribute 'get_explicitly_set_fields_by_scope'
cms-1       | [22/Apr/2026 17:27:11] "POST /api/xblock/v2/xblocks/lb:BradenX:LC1:problem:35cabc6a-6e20-494a-83ee-0107f68ded68/fields/ HTTP/1.1" 500 337505

kdmccormick · 2026-04-22T17:35:31Z

@bradenmacdonald hm, I'd bet that it's this commit: a82cd98

I'll have the Aximprovements team look at that now. I opened #38412

bradenmacdonald · 2026-04-22T17:38:39Z

@kdmccormick Ah, you're right, I was testing with a slightly old master missing that commit, and it was working. It's also weird that our openedx_content runtime relies on a function called serialize_modulestore_block_for_openedx_content when there's no modulestore blocks involved, but that hasn't changed lately.

bradenmacdonald · 2026-04-22T17:46:37Z

I read through all the code changes and didn't see any issues.

I just want to make sure we allow unicode component slugs, since we allowed them before, and it is possible to create them using the UI. Otherwise, I can do a bit more testing once that other bug is resolved and the unicode slugs are allowed, but I think it's looking great.

bradenmacdonald

Found no issues in my smoke tests with pre-release 0.44.0.

Conditional approval if you merge the unicode fix and bump this to 0.44+

Thanks for taking all this on! Consistent terminology is a huge win ❤️

While upgrading platform from core 0.39.2 to core 0.43.0, we realized that we've quietly supported non-ascii component and container codes in Content Libraries V2 since their release. We do not advertise our support for non-ascii codes within Libraries (and we don't want to, yet [2]), but in order to avoid breaking sites that stumbled upon non-ascii codes, we need to loosen the validation & db constraints which were added to openedx-core. On a code level, we simply add a `unicode=True/False` parameter to the `code_field()`. It's set to True for components and containers, and left as False for collections for now, since we no of no way in which a user could make non-ascii collection code (other than by editing a ZIP archive). There are some nuances around the migrations. See the comments for details. [1] openedx/openedx-platform#38402 (comment) [2] openedx/openedx-platform#38413 Bumps version to 0.44.0

…es (#558) While upgrading platform from core 0.39.2 to core 0.43.0, we realized that we've quietly supported non-ascii component and container codes in Content Libraries V2 since their release. We do not advertise our support for non-ascii codes within Libraries (and we don't want to, yet [2]), but in order to avoid breaking sites that stumbled upon non-ascii codes, we need to loosen the validation & db constraints which were added to openedx-core. On a code level, we simply add a `unicode=True/False` parameter to the `code_field()`. It's set to True for components and containers, and left as False for collections for now, since we no of no way in which a user could make non-ascii collection code (other than by editing a ZIP archive). There are some nuances around the migrations. See the comments for details. [1] openedx/openedx-platform#38402 (comment) [2] openedx/openedx-platform#38413 Bumps version to 0.44.0

kdmccormick · 2026-04-23T20:38:51Z

Thank you for all the reviews @bradenmacdonald !

Renames the openedx_django_lib.fields import in EntityLinkBase from the removed key_field helper to ref_field. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…0.43.0 Updates callers of get_learning_package_by_key (renamed to get_learning_package_by_ref), create_learning_package, and update_learning_package to use the new package_ref kwarg, and switches .key attribute reads to .package_ref. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

… 0.43.0 Switches callers of get_publishable_entity_by_key (now _by_ref) to the new name, and renames .key attribute reads on PublishableEntity, Component, and Container (via PublishableEntityMixin) to .entity_ref. Query filters using key__in/entity__key become entity_ref__in/ entity__entity_ref. The set_library_item_collections param entity_key is renamed to entity_ref for consistency. Collection.key reads are intentionally left for the collection_code rename in a later commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…re 0.43.0 Updates callers of get_component_by_key/component_exists_by_key (now _by_code) and switches the local_key kwarg on create_component, create_component_and_version, and related queries to component_code. Also renames component.local_key reads to component.component_code and adjusts a modulestore_migrator container query that filtered on publishable_entity__key (now entity_ref). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…edx-core 0.43.0 Switches get_container_by_key to get_container_by_code and renames the container creation kwarg from key to container_code for both create_container_and_version and its platform wrapper. Updates ComponentVersionMedia accesses: the model field .key becomes .path, and create_component_version_media's key kwarg becomes path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

…-core 0.43.0 Renames all Collection APIs that took key/collection_key to use collection_code: create_collection, update_collection, delete_collection, restore_collection, add_to_collection, etc. Switches Collection.key attribute reads to .collection_code across tests, signal handlers, search indexers, and modulestore_migrator. Filters like target_collection__key become target_collection__collection_code. Also updates the library restore serializer to track the renamed lp_restored_data fields: archive_org_key -> archive_org_code, archive_slug -> archive_package_code, key -> package_ref, archive_lp_key -> archive_package_ref. The archive_org_code and archive_package_code fields now allow None, since openedx-core no longer raises ValueError when the archive_package_ref cannot be parsed as {prefix}:{org_code}:{package_code}. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

fix: build locator with container_code fix: pylint and mypy fix: queries for search index fix: some missed cvm.key -> cvm.path fix: undo breaking library changes fix: openedx-core no longer raises integrityerror on conflict fix: misses in modulestore_migrator fix: search tests docs: Improve collection_code/key TODO comment

This is necessary because we are no longer presuming that package_ref follows the same format as a Content Library. In the future, we may want a more graceful way of handling this.

kdmccormick force-pushed the kdmccormick/keys branch 2 times, most recently from 5b949fa to 5dae386 Compare April 22, 2026 00:43

kdmccormick commented Apr 22, 2026

View reviewed changes

kdmccormick added the create-sandbox open-craft-grove should create a sandbox environment from this PR label Apr 22, 2026

kdmccormick force-pushed the kdmccormick/keys branch from f25cc96 to 9ec34fd Compare April 22, 2026 02:28

kdmccormick commented Apr 22, 2026

View reviewed changes

kdmccormick force-pushed the kdmccormick/keys branch from 8963e78 to 54c0e1c Compare April 22, 2026 16:05

kdmccormick marked this pull request as ready for review April 22, 2026 16:05

kdmccormick requested review from bradenmacdonald and ormsbee April 22, 2026 16:21

ormsbee mentioned this pull request Apr 22, 2026

feat: support added to export content libraries to git #38026

Open

ChrisChV mentioned this pull request Apr 22, 2026

feat: History log in content library components and containers [FC-0123] openedx/frontend-app-authoring#2948

Open

7 tasks

This was referenced Apr 22, 2026

Support Unicode block codes #38413

Open

feat: Reinstate support for Unicode component_codes and container_codes openedx/openedx-core#558

Merged

bradenmacdonald approved these changes Apr 23, 2026

View reviewed changes

kdmccormick force-pushed the kdmccormick/keys branch from 54c0e1c to bb5cc86 Compare April 23, 2026 16:58

kdmccormick changed the title ~~refactor: Upgrade to openedx-core 0.43.0 (for OEP-68)~~ refactor: Upgrade to openedx-core 0.44.0 (for OEP-68) Apr 23, 2026

This was referenced Apr 23, 2026

Resolve Opaque Key coherency issues openedx/openedx-core#322

Open

refactor: collection_id -> collection_code #38214

Closed

kdmccormick force-pushed the kdmccormick/keys branch from bb5cc86 to cd39b2d Compare April 23, 2026 18:36

kdmccormick mentioned this pull request Apr 23, 2026

fix: Allow --reset or --init on reindex_studio as no-ops #38434

Merged

kdmccormick force-pushed the kdmccormick/keys branch 3 times, most recently from cf8ba1f to bf0ac23 Compare April 23, 2026 20:38

kdmccormick enabled auto-merge (squash) April 23, 2026 20:45

kdmccormick removed the create-sandbox open-craft-grove should create a sandbox environment from this PR label Apr 23, 2026

kdmccormick and others added 9 commits April 23, 2026 17:18

build: Upgrade openedx-core pin, 0.39.2 -> 0.44.0

452c13c

refactor: Rename key_field to ref_field for openedx-core 0.43.0

b8acc81

Renames the openedx_django_lib.fields import in EntityLinkBase from the removed key_field helper to ref_field. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

fix: Return 'unknown/unknown' for un-parseable org/lib archive slugs

85fbbdd

This is necessary because we are no longer presuming that package_ref follows the same format as a Content Library. In the future, we may want a more graceful way of handling this.

kdmccormick force-pushed the kdmccormick/keys branch from bf0ac23 to 85fbbdd Compare April 23, 2026 21:18

kdmccormick merged commit 160e7e6 into master Apr 23, 2026
45 checks passed

kdmccormick deleted the kdmccormick/keys branch April 23, 2026 21:44

Conversation

kdmccormick commented Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Status

Testing

AI Usage

Uh oh!

kdmccormick Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kdmccormick left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bradenmacdonald commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kdmccormick commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bradenmacdonald commented Apr 22, 2026

Uh oh!

bradenmacdonald commented Apr 22, 2026

Uh oh!

bradenmacdonald left a comment

Choose a reason for hiding this comment

Uh oh!

kdmccormick commented Apr 23, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

kdmccormick commented Apr 21, 2026 •

edited

Loading

kdmccormick Apr 22, 2026 •

edited

Loading

bradenmacdonald commented Apr 22, 2026 •

edited

Loading

kdmccormick commented Apr 22, 2026 •

edited

Loading