feat: expand team-export surface to remaining team models#3712
Conversation
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add 38 models to the export manifest (custom actions, collections, files, embeddings, channels, events, notifications, chat messages, traces, evaluations, human annotations, analysis) with their team-scoping, secret, exclude, and extra-filter registries; serialize file/vector/pydantic fields; and import them via the existing FK/m2m machinery plus an annotation-queue-aggregate matcher. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds tags, comments, scores, pricing rules and usage records to the export surface. Generic-FK content types are serialized as app_label.model labels and resolved through the translation store on import; every possible target model is itself synced (pinned by test_generic_fk_target_models_are_synced). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The post_save signal sets last_activity_at to timezone.now() (import time) rather than the original message created_at. After importing all chat messages for a batch, backfill each session's last_activity_at with the max HUMAN message created_at via a subquery scoped to the target team. Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
📝 WalkthroughWalkthroughThis PR extends the team data sync export/import system with generic foreign key support end-to-end. The export serializer gains custom field classes for file paths, pgvector values, and pydantic schemas, plus a factory that emits content-type labels ( Estimated code review effort🎯 5 (Critical) | ⏱️ ~120 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
apps/utils/factories/evaluations.py (1)
199-199: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDrop the redundant
computed_atfactory value.
EvaluationRunAggregate.computed_atisauto_now_add=True, sofactory.LazyFunction(timezone.now)is overwritten on create and only adds noise. If a deterministic timestamp is needed for tests, set it after creation withupdate()or a post-generation hook.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/utils/factories/evaluations.py` at line 199, Remove the redundant computed_at assignment from the evaluation factory so EvaluationRunAggregate relies on its auto_now_add behavior instead of factory.LazyFunction(timezone.now). Update the factory definition in the evaluations factory class that builds EvaluationRunAggregate instances, and if tests need a fixed timestamp, apply it after creation with an update call or a post-generation hook rather than setting it during factory construction.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/teams/export/importer.py`:
- Around line 277-281: The generic-FK handling in importer logic is silently
accepting missing translations by assigning None from
FKTranslationStore.get_target(), which can leave unresolved relations behind.
Update the gfk_pairs loop in the importer method that builds field_values so it
fails fast the same way regular FK resolution does: detect when get_target()
returns no translated target for the type_label and fk field, and raise the
unresolved-FK error instead of storing None. Keep the fix localized to the
generic relation translation path so ContentType lookup and valid target
assignment still behave the same.
In `@apps/teams/export/manifest.py`:
- Around line 81-82: The export manifest ordering in Manifest needs to be
adjusted so evaluation_datasets is processed before evaluation_messages. Update
the ManifestEntry sequence in manifest.py to place the evaluation_datasets entry
ahead of the evaluation_messages entry, since EvaluationMessage depends on
evaluationdataset__team and must not be imported before its related dataset
exists.
---
Nitpick comments:
In `@apps/utils/factories/evaluations.py`:
- Line 199: Remove the redundant computed_at assignment from the evaluation
factory so EvaluationRunAggregate relies on its auto_now_add behavior instead of
factory.LazyFunction(timezone.now). Update the factory definition in the
evaluations factory class that builds EvaluationRunAggregate instances, and if
tests need a fixed timestamp, apply it after creation with an update call or a
post-generation hook rather than setting it during factory construction.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 4b433f6d-f188-4e0c-b93b-e66fa8a50eed
📒 Files selected for processing (19)
api-schemas/export.ymlapps/api/export/serializers.pyapps/api/export/tests/test_serializers.pyapps/api/export/tests/test_views.pyapps/teams/export/importer.pyapps/teams/export/manifest.pyapps/teams/export/tests/test_importer.pyapps/teams/export/tests/test_manifest.pyapps/teams/management/commands/sync_team.pyapps/utils/factories/analysis.pyapps/utils/factories/annotations.pyapps/utils/factories/custom_actions.pyapps/utils/factories/documents.pyapps/utils/factories/evaluations.pyapps/utils/factories/experiment.pyapps/utils/factories/files.pyapps/utils/factories/human_annotations.pyapps/utils/factories/notifications.pyopen-chat-studio.cs-export_apis
Mirror regular-FK resolution: raise UnresolvedForeignKey when a generic FK's target was never imported instead of silently nulling the relation. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A mode-160000 gitlink named after the repo's own working directory was committed by mistake (there is no .gitmodules), which breaks fresh clones and any git submodule operation. Drop it from the index. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Fail loudly (UnresolvedForeignKey) when an m2m member of a synced model has no translation, instead of silently dropping the link -- matching scalar FK resolution. Members of deliberately-unsynced models are still left out. - Mute model signals for the whole import (mute_signals) so the ChatMessage post_save handler no longer stamps each session's first/last_activity_at to the import time; the source values are preserved instead. Removes the now-unneeded activity backfill. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_VectorField subclassed serializers.Field with no schema hint, so drf-spectacular rendered embedding columns as bare strings. Annotate it as a list of floats so the generated schema matches the JSON actually served. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Picks up the pgvector array types and the 400 (previously a stale 409) response on the new secret-resource list endpoints. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Gates Failed
Enforce advisory code health rules
(1 file with Complex Method, Complex Conditional)
Our agent can fix these. Install it.
Gates Passed
3 Quality Gates Passed
Reason for failure
| Enforce advisory code health rules | Violations | Code Health Impact | |
|---|---|---|---|
| test_importer.py | 2 advisory rules | 10.00 → 9.39 | Suppress |
Quality Gate Profile: Clean Code Collective
Install CodeScene MCP: safeguard and uplift AI-generated code. Catch issues early with our IDE extension and CLI tool.
| return value.model_dump(mode="json") if isinstance(value, pydantic.BaseModel) else value | ||
|
|
||
|
|
||
| class _RelativeFileField(serializers.FileField): |
There was a problem hiding this comment.
Does this work when using S3 storage?
Code reviewNo issues found. Checked for bugs and CLAUDE.md compliance. |
Product Description
no change — internal team-export tooling.
Technical Description
Follow-up to the export_apis branch: expands the team-export manifest from a handful of core models to the remaining team-owned surface (chats/messages, traces, evaluations, human annotations, analyses, notifications, cost tracking, custom actions, documents/files, events, generic-tagging).
Points worth a reviewer's attention:
customtaggeditem,usercomment,score) are placed last because theircontent_objectcan point at many of the preceding models;usage_recordsfollowpricing_rules.app_label.modellabel (stable across servers) instead of the sourceContentTypepk, and the importer resolves the type by natural key and translates the object id through the FK store. Both sides read the GFK column names off the field itself (generic_fk_fields) rather than hardcoding. Coverage is guarded bytest_generic_fk_target_models_are_synced.TEAM_PATH_REGISTRYvalues may now be a list of ORM paths (OR'd together,distinct()applied) for models reachable via more than one relation — e.g.EventActionhas no team FK and is scoped through either of its two trigger one-to-ones.EXTRA_FILTERSadds post-team-scoping row exclusions; used to drop assistant-attachedCustomActionOperationrows (assistants aren't synced) so the model's check constraint holds on import.FileField/ImageField→ stored relative path (not/media/URL, which Django's safe-join rejects on assign), anddjango_pydantic_fieldschema fields → plain JSON.Collection.files/DocumentSource.filesare excluded from serialization (theirCollectionFilethrough rows carry the link) since.set()is rejected for explicit through models.annotationqueueaggregatematches an existing target row (recreated as a side effect of importing annotations) rather than creating a second, which the queue one-to-one forbids.export.yml(the generated OpenAPI schema) is regenerated to match.Migrations
N/A — no migrations.
Docs and Changelog