Skip to content

CBG-5443: do not fail migration with unknown prefixes in bucket#8359

Open
gregns1 wants to merge 3 commits into
mainfrom
CBG-5443
Open

CBG-5443: do not fail migration with unknown prefixes in bucket#8359
gregns1 wants to merge 3 commits into
mainfrom
CBG-5443

Conversation

@gregns1

@gregns1 gregns1 commented Jun 12, 2026

Copy link
Copy Markdown
Contributor

CBG-5443

Simple change to not fail migration job when unknown prefixes are in the bucket during migration. Logging remains in place for these prefixes and will log extra warning at end of job if we have keys remaining. Loop in place for errors during the migration run.

Pre-review checklist

  • Removed debug logging (fmt.Print, log.Print, ...)
  • Logging sensitive data? Make sure it's tagged (e.g. base.UD(docID), base.MD(dbName))
  • Updated relevant information in the API specifications (such as endpoint descriptions, schemas, ...) in docs/api

Dependencies (if applicable)

  • Link upstream PRs
  • Update Go module dependencies when merged

Integration Tests

Copilot AI review requested due to automatic review settings June 12, 2026 16:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates metadata migration behavior so that _sync:-prefixed documents with unrecognised prefixes no longer cause the migration job to fail or retry until max passes. Instead, migration completion is gated only on per-document move/delete errors, while unknown-prefix keys are left in place and surfaced via logging and stats for observability.

Changes:

  • Stop treating unknown-prefix fallback keys as a completion blocker; complete migration when stats.Errors == 0.
  • Add an end-of-run warning when completing with unknown-prefix keys still present on the fallback.
  • Update unit test expectations and Prometheus/stat descriptions to match the new policy.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
db/metadata_migration.go Updates MigrateMetadata function comment to reflect new semantics for the returned “remaining” count (unknown-prefix only).
db/background_mgr_metadata_migration.go Changes completion gating to ignore unknown-prefix count; adds warning on completion with leftovers; adjusts give-up error message.
db/background_mgr_metadata_migration_test.go Updates test to assert migration completes successfully even when an unknown-prefix _sync: doc remains on fallback.
base/stats_descriptions.go Updates stat description to clarify unknown-prefix keys do not block completion.

Comment thread db/metadata_migration.go Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

@adamcfraser adamcfraser left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Posted a question offline on the timing of this change - let's discuss.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants