feat: support added to export content libraries to git by marslanabdulrauf · Pull Request #38026 · openedx/openedx-platform

marslanabdulrauf · 2026-02-19T12:51:33Z

Ticket

https://github.com/mitodl/hq/issues/10083

Description

This pull request enhances the export_to_git workflow to support exporting both course and library content, including v2 libraries, to a git repository. It introduces a new export mechanism for v2 libraries using a zip-based backup API, refactors the main export function for broader content support, and improves logging and commit messages for clarity.

Support for v2 library export and generalization of export logic:

Added a new export_library_v2_to_zip function to export v2 libraries using the backup API, creating a zip backup and extracting it to the target directory.
Refactored export_to_git to accept a general content_key (course or library), automatically detect the content type, and select the appropriate export function for courses, v1 libraries, or v2 libraries. [1] [2]

Improvements to commit messages and logging:

Updated the git commit message to include the content type (course or library) and timestamp for better traceability.
Enhanced logging to provide clearer information on successful exports, including the content type and key.

Dependency and import updates:

Added imports for modules required by the new export and backup logic, including shutil, zipfile, datetime, Path, mkdtemp, slugify, and LibraryLocatorV2.

Supporting information

Link to other information about the change, such as Jira issues, GitHub issues, or Discourse discussions.
Be sure to check they are publicly readable, or if not, repeat the information here.

Testing instructions

Call this export_to_git function with different content_keys to test if they are exported properly to Github.
Make sure you have proper access rights for the repo

I have been testing it with ol-openedx-git-auto-export -- This Library export feature is not released yet

openedx-webhooks · 2026-02-19T12:51:38Z

Thanks for the pull request, @marslanabdulrauf!

This repository is currently maintained by @openedx/wg-maintenance-openedx-platform.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
- This process (including the steps you'll need to take) is documented here.
If it doesn't, simply proceed with the next step.

🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

Dependencies

This PR must be merged before / after / at the same time as ...
Blockers

This PR is waiting for OEP-1234 to be accepted.
Timeline information

This PR must be merged by XX date because ...
Partner information

This is for a course on edx.org.
Supporting documentation
Relevant Open edX discussion forum threads

🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

Details

Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

The size and impact of the changes that it introduces
The need for product review
Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

pdpinch · 2026-03-11T15:30:46Z

It introduces a new export mechanism for v2 libraries using a zip-based backup API

I was surprised this didn't exist already in libraries v2? What's the export strategy for libraries v2 from Studio?

asadali145

This works, but I have a few questions on the code.

marslanabdulrauf · 2026-03-12T16:41:13Z

It introduces a new export mechanism for v2 libraries using a zip-based backup API

I was surprised this didn't exist already in libraries v2? What's the export strategy for libraries v2 from Studio?

Its named as "Back up to local archive" which downloads a ZIP file containing Library V2 content.

asadali145

Looks better, I have a couple of suggestions and also, we need to add tests.

asadali145 · 2026-03-13T14:52:39Z

 from cms.djangoapps.contentstore.storage import course_import_export_storage

 from . import api
+from .api import create_library_v2_zip


We can remove this import and update the usage like api.create_library_v2_zip()

asadali145 · 2026-03-13T14:55:23Z


-def export_to_git(course_id, repo, user='', rdir=None):
-    """Export a course to git."""
+def export_library_v2_to_zip(library_key, root_dir, library_dir, user=None):


How about we move this to api.backup.py as well, like create_library_v2_zip?

yes it make sense 👍

asadali145

Can we add tests for the new and updated utils?

asadali145 · 2026-03-18T08:58:34Z

+    return root_dir, file_path
+
+
+def export_library_v2_to_zip(library_key, root_dir, library_dir, user=None):


I think the name is a bit misleading, this does not exports to zip but it extracts the zip created by create_library_v2_zip.

asadali145 · 2026-03-18T08:59:47Z

+    # Get user object for backup API
+    user_obj = get_user_model().objects.filter(username=user).first()
+    temp_dir, zip_path = create_library_v2_zip(library_key, user_obj)
+
+    try:
+        # Target directory for extraction
+        target_dir = os.path.join(root_dir, library_dir)
+
+        # Create target directory if it doesn't exist
+        os.makedirs(target_dir, exist_ok=True)
+
+        # Extract zip contents (will overwrite existing files)
+        with zipfile.ZipFile(zip_path, 'r') as zip_ref:
+            zip_ref.extractall(target_dir)
+
+        log.info('Extracted library v2 backup to %s', target_dir)
+
+    finally:
+        # Cleanup temporary files
+        if temp_dir.exists():


Some of the code comments are not needed here as the code is simple and readable that we don't really need. Also, there are some unnecessary blank lines as well.

asadali145 · 2026-03-18T09:00:46Z

+    is_library_v2 = isinstance(content_key, LibraryLocatorV2)
+    if is_library_v2:
+        # V2 libraries use backup API with zip extraction
+        export_xml_func = export_library_v2_to_zip
+        content_type_label = "library"
+    elif isinstance(content_key, LibraryLocator):
+        export_xml_func = export_library_to_xml
+        content_type_label = "library"
+    else:
+        export_xml_func = export_course_to_xml
+        content_type_label = "course"


nit: we could use default dict here.

asadali145

Works well for me. As discussed, we will add tests after an approval from core contributors.

asadali145 · 2026-03-25T11:57:21Z

+    return root_dir, file_path
+
+
+def export_library_v2_to_dir(library_key, root_dir, library_dir, user=None):


nit:

Suggested change

def export_library_v2_to_dir(library_key, root_dir, library_dir, user=None):

def extract_library_v2_zip_to_dir(library_key, root_dir, library_dir, user=None):

pdpinch · 2026-04-04T20:32:09Z

@marslanabdulrauf can you rebase this PR?

pdpinch · 2026-04-04T20:38:32Z

@mphilbrick211 can you recommend someone from the Libraries v2 team who might do a second review on this PR?

pdpinch · 2026-04-10T12:15:20Z

@marslanabdulrauf this one needs a rebase too.

ormsbee · 2026-04-17T17:44:16Z

@pdpinch: I can review this.

ormsbee · 2026-04-22T16:53:37Z

Reviewing this today. There's a fairly major PR in flight to account for openedx-core refactoring, but I think this PR is unaffected. Before I begin the review, can you please let me know if there were AI tools used to assist in the creation of this PR? If so, that's fine, as long as it adheres to our policy. I just want to know what was used.

Thank you.

ormsbee

Some minor requests for changes, and some optional nits.

@pdpinch, @marslanabdulrauf, @asadali145: At some point after the release, I'd like to have a discussion as to what we would need to do in order to gracefully move the git export functionality out to a plugin.

Thank folks.

ormsbee · 2026-04-22T20:14:00Z


-def export_to_git(course_id, repo, user='', rdir=None):
-    """Export a course to git."""
+def export_to_git(content_key, repo, user='', rdir=None):


Suggested change

def export_to_git(content_key, repo, user='', rdir=None):

def export_to_git(context_key, repo, user='', rdir=None):

Our terminology for a key that can be a Library or Course is a LearningContextKey, so the var should be named context_key. But I'm a bit confused how this is working, since my understanding is that you'd have to modify the git_export.py command parsing to go from CourseKey.from_string() to LearningContextKey.from_string(), and that doesn't seem to be a part of this PR? For unfortunate historical reasons, LibraryLocator is a kind of CourseKey, but the new v2 library keys aren't.

ormsbee · 2026-04-22T20:22:08Z

+    Export a course or library to git.
+
+    Args:
+        content_key: CourseKey or LibraryLocator for the content to export


But also LibraryLocatorV2 now, right?

ormsbee · 2026-04-22T20:24:27Z

+    is_library_v2 = isinstance(content_key, LibraryLocatorV2)
+    if is_library_v2:
+        # V2 libraries use backup API with zip extraction
+        content_export_func = extract_library_v2_zip_to_dir
+    elif isinstance(content_key, LibraryLocator):
+        content_export_func = export_library_to_xml
+    else:
+        content_export_func = export_course_to_xml
+        content_type_label = "course"


Nit (optional): You this setup work up here, but don't seem to use it until 65 lines later. You could do a simpler check to make sure it's a recognized key type up here (to error out early if necessary), and defer the rest of the logic until line 161. Then you don't need the indirection with the content_export_func.

So the first part could be like:

if not isinstance(context_key, (LibraryLocatorV2, LibraryLocator, CourseLocator)): raise TypeError( f"{context_key!r} for git export must be LibraryLocatorV2, LibraryLocator, " f"or CourseLocator, not {type(context_key}" )

And then your logic later on could look something like:

# We must check in this order, because LibraryLocator is a subclass of CourseLocator, # so isinstance(context_key, CourseLocator) on a LibraryLocator would return True. if isinstance(context_key, (LibraryLocatorV2, LibraryLocator)): content_type_label = "library" else: content_type_label = "course" if isinstance(context_key, LibraryLocatorV2): # V2 Library export export_library_to_xml(context_key, root_dir, content_dir, user) else: # V1 Libraries and Courses both use the same modulestore-based export export_course_to_xml(modulestore(), contentstore(), context_key, root_dir, content_dir)

That way, each block stands on its own better, and you don't have to scan up or down the function to understand what's going on. Also, the content_type_label gets written once, rather than written with a default and then overwritten afterwards, making it just a little more obvious.

(Disclaimer: This is off the cuff and I haven't tried actually running this code, so there might be something obviously wrong with it. Please just take it as a broad suggestion. )

It just occurred to me that we explicitly made the locators involved here implement is_course to make this simpler to deal with. So the "library" vs. "course" logic can be:

content_type_label = "course" if context_key.is_course else "library"

ormsbee · 2026-04-22T22:12:53Z

+    root_dir = Path(mkdtemp())
+    sanitized_lib_key = str(library_key).replace(":", "-")
+    sanitized_lib_key = slugify(sanitized_lib_key, allow_unicode=True)
+    timestamp = datetime.now().strftime("%Y-%m-%d-%H%M%S")
+    filename = f'{sanitized_lib_key}-{timestamp}.zip'
+    file_path = os.path.join(root_dir, filename)
+    origin_server = getattr(settings, 'CMS_BASE', None)


Please try to refactor this so that it shares the same code path as the celery task for backup uses, instead of being a copy that might drift as conventions change.

ormsbee · 2026-04-22T22:14:57Z

+        Exception: If backup creation or extraction fails
+    """
+    # Get user object for backup API
+    user_obj = get_user_model().objects.filter(username=user).first()


If someone doesn't want to attach user information, that's one thing. But if someone is explicitly passing a user and that user is not found, it should be an error, and not simply silently fall back to None.

ormsbee · 2026-04-22T22:20:42Z

+    return root_dir, file_path
+
+
+def extract_library_v2_zip_to_dir(library_key, root_dir, library_dir, user=None):


Suggested change

def extract_library_v2_zip_to_dir(library_key, root_dir, library_dir, user=None):

def extract_library_v2_zip_to_dir(library_key, root_dir, library_dir, username=None):

Otherwise, this could be mistaken for a User obj, when it's being used as a username here.

openedx-webhooks added the open-source-contribution PR author is not from Axim or 2U label Feb 19, 2026

openedx-webhooks added this to Contributions Feb 19, 2026

github-project-automation Bot moved this to Needs Triage in Contributions Feb 19, 2026

marslanabdulrauf mentioned this pull request Feb 19, 2026

feat: library export support being added mitodl/open-edx-plugins#737

Open

mphilbrick211 moved this from Needs Triage to Ready for Review in Contributions Feb 24, 2026

mphilbrick211 added the needs reviewer assigned PR needs to be (re-)assigned a new reviewer label Feb 24, 2026

marslanabdulrauf force-pushed the marslan/10083-export-content-libraries-git branch from 63668a5 to 24001ee Compare March 11, 2026 12:20

asadali145 suggested changes Mar 11, 2026

View reviewed changes

Comment thread cms/djangoapps/contentstore/git_export_utils.py Outdated

Comment thread cms/djangoapps/contentstore/git_export_utils.py Outdated

marslanabdulrauf force-pushed the marslan/10083-export-content-libraries-git branch from 24001ee to 9b02669 Compare March 13, 2026 10:41

marslanabdulrauf requested a review from asadali145 March 13, 2026 10:42

asadali145 suggested changes Mar 13, 2026

View reviewed changes

marslanabdulrauf force-pushed the marslan/10083-export-content-libraries-git branch from b36e709 to 30baaab Compare March 16, 2026 11:25

marslanabdulrauf requested a review from asadali145 March 16, 2026 11:25

marslanabdulrauf force-pushed the marslan/10083-export-content-libraries-git branch 4 times, most recently from 4862309 to a4af66f Compare March 17, 2026 10:50

asadali145 reviewed Mar 18, 2026

View reviewed changes

marslanabdulrauf force-pushed the marslan/10083-export-content-libraries-git branch from a4af66f to bad0d90 Compare March 19, 2026 10:08

asadali145 approved these changes Mar 25, 2026

View reviewed changes

marslanabdulrauf force-pushed the marslan/10083-export-content-libraries-git branch from d1aa766 to 9c6d943 Compare March 26, 2026 12:24

marslanabdulrauf force-pushed the marslan/10083-export-content-libraries-git branch 2 times, most recently from a7f35b6 to 6e3b9bd Compare April 6, 2026 11:47

mphilbrick211 removed the needs reviewer assigned PR needs to be (re-)assigned a new reviewer label Apr 6, 2026

mphilbrick211 moved this from Ready for Review to In Eng Review in Contributions Apr 6, 2026

marslanabdulrauf added 7 commits April 14, 2026 15:49

feat: support added to export content libraries to git

35e36a3

fix: separate common part for v2 zip and use that in git export

28b039d

fix: move common function to a new file

acdac16

fix: move export_library_v2_to_zip to backup file as well

4cc8a51

fix: lint-import fix

02e70c5

refactor: export function renamed

1e7a97e

test: re-run tests

0651d60

marslanabdulrauf force-pushed the marslan/10083-export-content-libraries-git branch 3 times, most recently from 27398c6 to 86a8815 Compare April 14, 2026 11:02

test: content library export tests added

6bfad43

marslanabdulrauf force-pushed the marslan/10083-export-content-libraries-git branch from 86a8815 to 6bfad43 Compare April 14, 2026 11:06

ormsbee self-requested a review April 17, 2026 17:44

Merge branch 'master' into marslan/10083-export-content-libraries-git

c0d1ad4

ormsbee requested changes Apr 22, 2026

View reviewed changes

		return root_dir, file_path


		def export_library_v2_to_zip(library_key, root_dir, library_dir, user=None):

		return root_dir, file_path


		def export_library_v2_to_dir(library_key, root_dir, library_dir, user=None):

	def export_library_v2_to_dir(library_key, root_dir, library_dir, user=None):
	def extract_library_v2_zip_to_dir(library_key, root_dir, library_dir, user=None):

	def export_to_git(content_key, repo, user='', rdir=None):
	def export_to_git(context_key, repo, user='', rdir=None):

Conversation

marslanabdulrauf commented Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Ticket

Description

Supporting information

Testing instructions

Uh oh!

openedx-webhooks commented Feb 19, 2026

Uh oh!

pdpinch commented Mar 11, 2026

Uh oh!

asadali145 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

marslanabdulrauf commented Mar 12, 2026

Uh oh!

asadali145 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

asadali145 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

asadali145 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pdpinch commented Apr 4, 2026

Uh oh!

pdpinch commented Apr 4, 2026

Uh oh!

pdpinch commented Apr 10, 2026

Uh oh!

ormsbee commented Apr 17, 2026

Uh oh!

ormsbee commented Apr 22, 2026

Uh oh!

ormsbee left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

marslanabdulrauf commented Feb 19, 2026 •

edited

Loading