Skip to content

Preserve created_at / updated_at on annotations imported via storage sync #9797

Description

@mjpesavento-ASPEN

Summary

When annotations are imported into a Label Studio project via a storage sync (S3, GCS, Azure, local file — anything that hits ImportStorage.add_task), the created_at and updated_at values on the resulting Annotation rows are always set to the moment of import, overwriting any timestamps present in the source JSON.

For deployments importing historical or migrated annotation data, this silently destroys the real annotation time — the audit trail, time-based analytics, and any downstream reporting keyed on when the annotation was actually created.

I'd like to propose an opt-in env var — LABEL_STUDIO_PRESERVE_IMPORT_TIMESTAMPS, default false — that changes this behavior only when explicitly requested. I'm happy to open the PR.

Current behavior

Annotation is defined with:

created_at = models.DateTimeField(_('created at'), auto_now_add=True, ...)
updated_at = models.DateTimeField(_('updated at'), auto_now=True, ...)

ImportStorage.add_task (in label_studio/io_storages/base_models.py) passes each annotation dict through AnnotationSerializer(...).save(). The serializer treats those two fields as read-only because of the auto_now/auto_now_add flags, so any created_at/updated_at present in the source dict is silently dropped, and Django's model layer sets both to now() on save.

Result: importing an annotation whose source JSON says "created_at": "2024-06-01T12:00:00Z" produces a row with created_at = <now>.

Reproduction

  1. Enable an S3 (or local) source storage on a project.
  2. Sync a task whose JSON looks like:
    {
      "data": { "image": "s3://bucket/img.png" },
      "annotations": [{
        "result": [...],
        "created_at": "2024-06-01T12:00:00Z",
        "updated_at": "2024-06-01T12:05:00Z",
        "completed_by": 3
      }]
    }
  3. Trigger the sync.
  4. Query the resulting annotation: created_at and updated_at are the sync time, not the values from the JSON.

Proposal

Add an opt-in Django setting sourced from an env var:

  • Name: LABEL_STUDIO_PRESERVE_IMPORT_TIMESTAMPS
  • Default: false — behavior identical to today.
  • When true: on the annotation-creation branch of ImportStorage.add_task, use a subclass of AnnotationSerializer that makes created_at/updated_at writable, and wrap the save in a suppress_autotime context manager that temporarily disables the model's auto_now/auto_now_add flags. If the source JSON doesn't include one of the fields, fall back to now() so we never write a NULL timestamp.

Note: a suppress_autotime helper already exists in the codebase at label_studio/core/old_ls_migration.py, used by the legacy sqlite→postgres migration path. The PR would extract that helper into a proper shared utility module (e.g., core/utils/db.py) before reusing it here — new code shouldn't depend on a module named old_ls_migration, and this extract is a clean two-file refactor. The utility itself doesn't change.

add_task is the correct target for this change: it's the shared final step for the current _scan_and_create_links (v1) scanner and, per the design comments in the codebase, is intended to remain the endpoint for the planned _scan_and_create_links_v2 pipeline once that's implemented. Patching there catches both flows.

Key properties of the proposal:

  • Behind a flag. Existing users see no behavior change. Zero risk to current deployments.
  • Preserves the current serializer flow. No bypass of AnnotationSerializerAnnotationDuplicateError handling, FSM state init, and feature-flag validation gates all still fire.
  • Small diff. Two commits — one small refactor to move suppress_autotime into a proper utils module, one focused feature commit. Together ~40 lines net plus tests.

Why not just require users to write the timestamp via API after import?

Annotation.updated_at is auto_now=True, which fires on every save. Writing the timestamp after the fact requires the same suppress_autotime trick — you can't just PATCH it. The fix belongs at the point of import, not as a post-hoc correction.

Impact if merged

  • Deployments migrating from other annotation platforms (Prodigy, doccano, an older LS instance, in-house tooling) can preserve real annotation timestamps on import.
  • Deployments backing up and restoring via JSON export/import round-trip preserve the timeline.
  • The default is unchanged, so no user is surprised.

Willingness to implement

Yes — I'll open a PR against develop with the change plus a test case. Just wanted an issue on file to confirm the direction is welcome before submitting.

Environment (for context, not necessarily reproduction-critical)

  • Label Studio Community edition
  • Currently patched locally against a snapshot from develop at 1.16.0.dev0 (commit 123a32f28, Jan 2025), preparing to rebase on nightly.
  • Postgres 13 backing store.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions