feat: export archived records and back-fill file content in team sync#3795
Conversation
Team sync copies a file row's storage path but not its bytes, relying on the operator to move blobs into the target storage separately. When a blob is still missing, importing the file now fetches it from the source's file content API and writes it before saving the row, rather than failing. A 404 means the source is missing the blob too: the file's metadata is imported without content (so rows that reference it still resolve) and the file is listed in the sync report. Other fetch errors abort the sync. Force-delete now warns that it also clears the team's files from backend storage, so they must be re-imported. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Versioned models filter is_archived=False on their default manager, so archived rows (and their file bytes) were silently left out of the team export. Use get_all() to bypass that filter: - team_scoped_queryset() serves archived rows for every versioned model. - Drop the collection_files is_archived filter; it only existed because archived collections weren't exported, which is no longer true. - get_team_files_queryset() includes archived files' bytes in the zip so imported archived rows don't point at missing blobs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
![codescene-delta-analysis[bot]](https://avatars.githubusercontent.com/in/53921?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
delete_collection_task runs after Collection.archive() has set is_archived=True, but it loaded the collection via Collection.objects, whose manager hides archived rows -- so the task was a silent no-op and left CollectionFile rows orphaned. Load the row with get_all() instead. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (12)
📝 WalkthroughWalkthroughThis PR adds a file-content backfill mechanism to the team export/import pipeline: a new client method and exception fetch missing source file blobs, the importer recovers missing blobs during import (recording unresolvable filenames), and sync_team reports missing files and updates the force-delete confirmation. Separately, manifest export querysets, the team files export task, and collection deletion now use unfiltered managers ( Estimated code review effort: 4 (Complex) | ~60 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
The importer's existence check used the default manager, which hides archived and soft-deleted rows. On re-import those rows weren't found and got recreated as duplicates -- and for channels that violates the unique external_id. Look up through _base_manager so hidden target rows are matched and updated instead. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Use _base_manager in team_scoped_queryset so ExperimentChannel's deleted=False filter is bypassed alongside the versioning is_archived filter, giving a faithful snapshot that includes soft-deleted channels. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
(1 files improve in Code Health)
Gates Failed
(1 file with Lines of Code in a Single File)
Our agent can fix these. Install it.
Gates Passed
Reason for failure
| Enforce advisory code health rules | Violations | Code Health Impact | |
|---|---|---|---|
| test_importer.py | 1 advisory rule | 9.39 → 8.82 | Suppress |
View Improvements
| File | Code Health Impact | Categories Improved |
|---|---|---|
| importer.py | 9.39 → 10.00 | Overall Code Complexity |
Quality Gate Profile: Clean Code Collective
Install CodeScene MCP: safeguard and uplift AI-generated code. Catch issues early with our IDE extension and CLI tool.
| from datetime import UTC, datetime | ||
|
|
||
| import pytest | ||
| import requests |
There was a problem hiding this comment.
❌ New issue: Lines of Code in a Single File
This module has 611 lines of code, improve code health by reducing it to 600
2 participants
Product Description
Makes team-to-team sync a faithful, complete snapshot. Two gaps are closed:
Technical Description
Export archived records (
feat: export archived records in the team export API)is_archived=Falseon their default manager.team_scoped_queryset()now callsmodel.objects.get_all()(theVersionsObjectManagerMixinescape hatch) when available to bypass that filter, so archived rows are served for every versioned model.get_team_files_queryset()(the file-blob zip) also usesFile.objects.get_all(), so archived files' bytes ship too — otherwise imported archivedFilerows would reference blobs that were never uploaded. (Archiving a file never removes its storage blob, so the bytes are available.)Back-fill missing file content (
feat: back-fill missing file content when syncing a team)404from the source means the source is missing the blob too: the row is imported without content (so references still resolve) and the file is listed in the sync report. Other fetch errors abort the sync.Fix archived-collection deletion (
fix: delete files of archived collections in delete_collection_task)delete_collection_taskruns afterCollection.archive()has setis_archived=True, but it loaded the row viaCollection.objects, whose manager hides archived rows — so the task was a silent no-op that leftCollectionFilerows orphaned. It now loads the row withget_all().Migrations
No migrations in this PR.
Demo
N/A
Docs and Changelog
🤖 Generated with Claude Code