Skip to content

How to force a full re-sync of a project to self-hosted cloud after an in-place project merge/rename? #595

Description

@Erika1408

Summary

After reorganizing my local projects (merging/renaming via POST /projects/migrate) and deleting the now-obsolete project names from a self-hosted cloud, the reorganized projects will no longer re-push to the cloud. Local SQLite has the full, correct data, but the cloud keeps the stale/empty state, and every supported command reports "already exported / nothing to do". I couldn't find a supported way to force a full re-mirror of a project without touching last_acked_seq or deleting sync_mutations (which the docs warn against).

Environment

  • engram 1.19.0 (client)
  • Self-hosted cloud: engram 1.19.0 in Docker (cloud serve) + Postgres, ENGRAM_CLOUD_ALLOWED_PROJECTS=*

What I did (repro)

  1. Ran a local engram serve and used POST /projects/migrate with {"old_project": "project-old", "new_project": "project-new"} to:
    • merge a duplicate project (project-old, ~270 memories) into project-canonical (now ~300 memories locally)
    • move several one-off projects into a single catch-all project
  2. On the cloud Postgres, deleted the rows for the now-obsolete project names from cloud_chunks, cloud_project_sessions, cloud_project_controls, cloud_mutations.

The problem

The reorganized projects diverge and won't reconcile. For a merged project: local ~300 memories, cloud ~34.

engram sync --cloud --project project-canonicalNothing new to sync — all memories already exported.

The merged memories are flagged exported/acked locally, but they were deleted from the cloud under their old project name, so they never re-upload under the new name.

What I tried (all failed)

  1. Official upgrade flow:

    engram cloud upgrade doctor    --project <p>   # status: ready
    engram cloud upgrade repair    --project <p> --dry-run   # upgrade_repair_noop
    engram cloud upgrade bootstrap --project <p>   # stage: bootstrap_verified, noop:false
    

    Bootstrap "verifies" at the stale count and never pushes the missing memories.

  2. tools/repair-missing-session-directory.sh --fix-exported <p>"No local observations in project export scope with missing required fields." (Fields are not empty; not the empty-payload case.)

  3. engram sync --cloud --project <p> --force → still Nothing new to sync.

  4. A manual re-queue (acked_at = NULL for the project's sync_mutations + clearing its cloud:<project> sync_state/sync_chunks, then engram sync --cloud) only partially re-pushed and did not converge — so I reverted it. I did not edit last_acked_seq or delete mutations.

Questions

  1. What is the supported way to force a full re-push / re-mirror of a single project's local memories to a self-hosted cloud after an in-place /projects/migrate merge/rename — i.e. "reset this project's cloud sync state and re-upload everything from the local observations"?
  2. When merging/renaming projects that were already synced to cloud, what is the correct cloud-side step to keep the cloud consistent? Should /projects/migrate be run against the cloud server too (it 404s on cloud serve), instead of deleting the old project rows?
  3. Is there a way to make bootstrap treat a project as "not yet mirrored" so it re-pushes the full local state?

Happy to provide doctor --json, schema, or logs. Thanks!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions