feat!: Package and Entity keys are now opaque refs

The `{LearningPackage,PublishableEntity}.key` fields were always
meant to be externally-supplied refs whose format does not matter.
However, they were effectively being used as parseable psuedo-keys
for identifying containers and components, with more assumptions to their
structure being baked in over time, both within and outside openedx-core.

This commit renames them to `{package,entity}_ref`, and removes all
parsing of them from the API and from the internals (except for where
required for backcompat with Ulmo ZIP backups). Instead, the pairings
of (component_type,component_code) and (container_type,container_code)
are used to identify components and containers; their respective
entity_refs are derived from those *by conventional default* but also
are customizable by callers.

BREAKING CHANGE: Renamed key_field(...) -> ref_field(...) in openedx_django_lib.

BREAKING CHANGE: Renamed PublishableEntity.key -> PublishableEntityKey.entity_ref
BREAKING CHANGE: Renamed PublishableEntityMixin.key -> PublishableEntityMixin.entity_ref
BREAKING CHANGE: In create_publishable_entity(...), renamed param key -> entity_ref
BREAKING CHANGE: Renamed get_publishable_entity_by_key(...) ->
  get_publishable_entity_by_ref(...), and renamed param key -> entity_ref
BREAKING CHANGE: In get_entity_collections(...), renamed param entity_key -> entity_ref
BREAKING CHANGE: Removed function get_or_create_component_type_by_entity_key().
BREAKING CHANGE: Rename get_container_children_entities_keys(...) ->
  get_container_children_entity_refs(...)
BREAKING CHANGE: Renamed get_container_by_key(...) -> get_container_by_ref(..)
  and renamed param key -> entity_ref.
BREAKING CHANGE: Renamed get_container_children_entities_keys(...) ->
  get_container_children_entity_refs(...)

BRAEKING CHANGE: Renamed LearningPackage.key -> LearningPackage.package_ref.
BREAKING CHANGE: In create_learning_package(...), renamed param key -> package_ref
BREAKING CHANGE: In update_learning_package(...), renamed param key -> package_ref
BREAKING CHANGE: Renamed get_learning_package_by_key(...) ->
  get_learning_package_by_ref(...), and renamed param key -> package_ref
BREAKING CHANGE: In learning_package_exists(...), renamed param key -> package_ref

BREAKING CHANGE: In look_up_component_version_media(...):
  * Renamed param learning_package_key -> learning_package_ref
  * Renamed param component_key -> entity_ref

BREAKING CHANGE: In add_assets_to_component management command:
  * Renamed argument learning_package_key -> learning_package_ref
  * Renamed argument component_key -> entity_ref

BREAKING CHANGE In load_learning_package:
  * Renamed  param key -> package_ref
  * In return dict, within sub-dict ["lp_restored_data"]:
    * Renamed ["key"] -> ["package_ref"]
    * Renamed ["archive_lp_key"] -> ["archive_package_ref"]
    * Renamed ["archive_org_key"] -> ["archive_org_code"]
    * Renamed ["archive_slug"] -> ["archive_package_code"]
  * If archive_package_ref cannot be parsed into "{_prefix}:{org_code}:{package_code}", then
    archvie_org_code and archive_package_code will both be None. Callers should handle this case.
    (Previously, a ValueError would be raised by backup-restore code.)

Backup/Restore format changes (non-breaking):
* In [learning_package], key field is renamed -> package_ref.
* In [entity], key field is renamed -> entity_ref.
* For compatibility with Ulmo instances, both key and {package,entity}_ref
  will be written in all new archives.
* For compatibility with Ulmo archives, both key and {package,entity}_ref
  will be read, with priority given to the latter.

Part of: #322

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Author: kdmccormick

olx_importer/management/commands/load_components.py

@@ -61,33 +61,33 @@ def init_known_types(self):
 
     def add_arguments(self, parser):
         parser.add_argument("course_data_path", type=pathlib.Path)
-        parser.add_argument("learning_package_key", type=str)
+        parser.add_argument("learning_package_ref", type=str)
 
-    def handle(self, course_data_path, learning_package_key, **options):
+    def handle(self, course_data_path, learning_package_ref, **options):
         self.course_data_path = course_data_path
-        self.learning_package_key = learning_package_key
-        self.load_course_data(learning_package_key)
+        self.learning_package_ref = learning_package_ref
+        self.load_course_data(learning_package_ref)
 
     def get_course_title(self):
         course_type_dir = self.course_data_path / "course"
         course_xml_file = next(course_type_dir.glob("*.xml"))
         course_root = ET.parse(course_xml_file).getroot()
         return course_root.attrib.get("display_name", "Unknown Course")
 
-    def load_course_data(self, learning_package_key):
+    def load_course_data(self, learning_package_ref):
         print(f"Importing course from: {self.course_data_path}")
         now = datetime.now(timezone.utc)
         title = self.get_course_title()
 
-        if content_api.learning_package_exists(learning_package_key):
+        if content_api.learning_package_exists(learning_package_ref):
             raise CommandError(
-                f"{learning_package_key} already exists. "
+                f"{learning_package_ref} already exists. "
                 "This command currently only supports initial import."
             )
 
         with transaction.atomic():
             self.learning_package = content_api.create_learning_package(
-                learning_package_key, title, created=now,
+                learning_package_ref, title, created=now,
             )
             for block_type in SUPPORTED_TYPES:
                 self.import_block_type(block_type, now) #, publish_log_entry)

src/openedx_content/applets/backup_restore/api.py

@@ -5,26 +5,28 @@
 
 from django.contrib.auth.models import User as UserType  # pylint: disable=imported-auth-user
 
-from ..publishing.api import get_learning_package_by_key
+from ..publishing.api import get_learning_package_by_ref
 from .zipper import LearningPackageUnzipper, LearningPackageZipper
 
 
-def create_zip_file(lp_key: str, path: str, user: UserType | None = None, origin_server: str | None = None) -> None:
+def create_zip_file(
+        package_ref: str, path: str, user: UserType | None = None, origin_server: str | None = None
+) -> None:
     """
     Creates a dump zip file for the given learning package key at the given path.
     The zip file contains a TOML representation of the learning package and its contents.
 
-    Can throw a NotFoundError at get_learning_package_by_key
+    Can throw a NotFoundError at get_learning_package_by_ref
     """
-    learning_package = get_learning_package_by_key(lp_key)
+    learning_package = get_learning_package_by_ref(package_ref)
     LearningPackageZipper(learning_package, user, origin_server).create_zip(path)
 
 
-def load_learning_package(path: str, key: str | None = None, user: UserType | None = None) -> dict:
+def load_learning_package(path: str, package_ref: str | None = None, user: UserType | None = None) -> dict:
     """
     Loads a learning package from a zip file at the given path.
     Restores the learning package and its contents to the database.
     Returns a dictionary with the status of the operation and any errors encountered.
     """
     with zipfile.ZipFile(path, "r") as zipf:
-        return LearningPackageUnzipper(zipf, key, user).load()
+        return LearningPackageUnzipper(zipf, package_ref, user).load()

src/openedx_content/applets/backup_restore/serializers.py

@@ -14,11 +14,14 @@ class LearningPackageSerializer(serializers.Serializer):  # pylint: disable=abst
     Serializer for learning packages.
 
     Note:
-        The `key` field is serialized, but it is generally not trustworthy for restoration.
-        During restore, a new key may be generated or overridden.
+        The ref/key field is serialized but is generally not trustworthy for
+        restoration. During restore, a new ref may be generated or overridden.
     """
+
     title = serializers.CharField(required=True)
-    key = serializers.CharField(required=True)
+    # The model field is now LearningPackage.package_ref, but the archive format
+    # still uses "key".  A future v2 format may align the name.
+    key = serializers.CharField(required=True, source="package_ref")
     description = serializers.CharField(required=True, allow_blank=True)
     created = serializers.DateTimeField(required=True, default_timezone=timezone.utc)
 
@@ -42,8 +45,11 @@ class EntitySerializer(serializers.Serializer):  # pylint: disable=abstract-meth
     """
     Serializer for publishable entities.
     """
+
     can_stand_alone = serializers.BooleanField(required=True)
-    key = serializers.CharField(required=True)
+    # The model field is now PublishableEntity.entity_ref, but the archive format
+    # still uses "key".  A future v2 format may align the name.
+    key = serializers.CharField(required=True, source="entity_ref")
     created = serializers.DateTimeField(required=True, default_timezone=timezone.utc)
 
 
@@ -52,7 +58,7 @@ class EntityVersionSerializer(serializers.Serializer):  # pylint: disable=abstra
     Serializer for publishable entity versions.
     """
     title = serializers.CharField(required=True)
-    entity_key = serializers.CharField(required=True)
+    entity_ref = serializers.CharField(required=True)
     created = serializers.DateTimeField(required=True, default_timezone=timezone.utc)
     version_num = serializers.IntegerField(required=True)
 

src/openedx_content/applets/backup_restore/toml.py

@@ -43,7 +43,9 @@ def toml_learning_package(
     # Learning package main info
     section = tomlkit.table()
     section.add("title", learning_package.title)
-    section.add("key", learning_package.key)
+    # The model field is now LearningPackage.package_ref, but the archive format
+    # still uses "key".  A future v2 format may align the name.
+    section.add("key", learning_package.package_ref)
     section.add("description", learning_package.description)
     section.add("created", learning_package.created)
     section.add("updated", learning_package.updated)
@@ -89,8 +91,9 @@ def _get_toml_publishable_entity_table(
     """
     entity_table = tomlkit.table()
     entity_table.add("can_stand_alone", entity.can_stand_alone)
-    # Add key since the toml filename doesn't show the real key
-    entity_table.add("key", entity.key)
+    # The model field is now PublishableEntity.entity_ref, but the archive format
+    # still uses "key".  A future v2 format may align the name.
+    entity_table.add("key", entity.entity_ref)
     entity_table.add("created", entity.created)
 
     if not include_versions:
@@ -191,13 +194,13 @@ def toml_publishable_entity_version(version: PublishableEntityVersion) -> tomlki
     if hasattr(version, 'containerversion'):
         # If the version has a container version, add its children
         container_table = tomlkit.table()
-        children = containers_api.get_container_children_entities_keys(version.containerversion)
+        children = containers_api.get_container_children_entity_refs(version.containerversion)
         container_table.add("children", children)
         version_table.add("container", container_table)
     return version_table
 
 
-def toml_collection(collection: Collection, entity_keys: list[str]) -> str:
+def toml_collection(collection: Collection, entity_refs: list[str]) -> str:
     """
     Create a TOML representation of a collection.
 
@@ -215,7 +218,7 @@ def toml_collection(collection: Collection, entity_keys: list[str]) -> str:
     doc = tomlkit.document()
 
     entities_array = tomlkit.array()
-    entities_array.extend(entity_keys)
+    entities_array.extend(entity_refs)
     entities_array.multiline(True)
 
     collection_table = tomlkit.table()