Skip to content

fix: reconcile auto-increment state for copy and clone - #26197

Merged
XuPeng-SH merged 8 commits into
matrixorigin:mainfrom
VioletQwQ-0:codex/issue-23143-copy-clone-auto-increment-reconcile
Jul 28, 2026
Merged

fix: reconcile auto-increment state for copy and clone#26197
XuPeng-SH merged 8 commits into
matrixorigin:mainfrom
VioletQwQ-0:codex/issue-23143-copy-clone-auto-increment-reconcile

Conversation

@VioletQwQ-0

@VioletQwQ-0 VioletQwQ-0 commented Jul 27, 2026

Copy link
Copy Markdown
Collaborator

issue #23143

What type of PR is this?

  • bug
  • feature
  • improvement
  • documentation

What this PR does / why we need it:

This is the next serial prerequisite in the #25883 split. It makes COPY/CLONE allocator reconciliation independently correct before the user-facing ALTER TABLE ... AUTO_INCREMENT=N entry is enabled.

Invariant:

  • a visible destination allocator is reset to max(requested offset, copied-row maximum, source allocator offset);
  • fresh CLONE maps allocator state by cloned column name because planner IDs are in a new coordinate space;
  • ALTER COPY maps retained columns by stable ColId, preserving rename/reorder and preventing a dropped-and-readded same-name column from inheriting old state;
  • hidden fake-PK allocators remain independent from visible AUTO_INCREMENT requests;
  • failed/cancelled ALTER COPY statements discard transaction-private resets, and all internal SQL results are closed.

Changes

  • collect source allocator offsets and visible row maxima at the selected snapshot/account;
  • translate source catalog indexes to destination identities before execution;
  • replace fake value insertion with transaction-aware SetOffset;
  • reconcile temporary ALTER COPY tables after copied rows become visible in the DDL transaction;
  • validate destination signed/unsigned integer bounds before resetting;
  • add regression coverage for fresh ID collisions, rename/reorder, drop/add name reuse, hidden allocators, three-way bounds, overflow, cancellation, rollback cleanup, quoted identifiers, and error-result cleanup;
  • add an issue-linked distributed case that snapshots the real source allocator offset and proves COPY ALTER preserves its reserved range.

Non-goals

  • no ALTER TABLE ... AUTO_INCREMENT=N SQL entry;
  • no mixed-version enablement gate;
  • no change to the epoch attached to allocated writes;
  • no generic transaction/recovery changes.

Testing

  • go test ./pkg/incrservice ./pkg/sql/colexec/table_clone ./pkg/sql/compile -count=1
  • focused -race runs for the new incrservice, table_clone, and compile tests
  • go test ./pkg/... -run '^$'
  • dedicated issue_23143_copy_auto_increment distributed regression added; fresh CI execution is pending for the current head
  • git diff --check
  • mo-self-review: PASS
  • exact-head mo-pr-preflight-review: PASS review=PASS validation=PENDING for 6b1bb91fa0c292ab056adab3a8ba34935125e9fc

Diff size

  • production code: 5 files, +416/-99
  • tests: 7 files, +689/-4
  • generated code: none

Dependency

Depends on the merged epoch fence (#26027) and transactional allocator reset (#26085). The next split item will start from main only after this PR merges.

@matrix-meow matrix-meow added the size/XL Denotes a PR that changes [1000, 1999] lines label Jul 27, 2026
@mergify mergify Bot added the kind/bug Something isn't working label Jul 27, 2026
@VioletQwQ-0
VioletQwQ-0 marked this pull request as ready for review July 27, 2026 05:53
@qodo-code-review

Copy link
Copy Markdown

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

@VioletQwQ-0

Copy link
Copy Markdown
Collaborator Author

Resolved the current merge conflict in 202a547 by normally merging upstream/main at a55820b.

The only content conflict was in pkg/sql/compile/alter_test.go, where this PR added the AUTO_INCREMENT reset-cleanup test and upstream added the IVF INCLUDE-column affected-index test at the same location. The resolution keeps both complete tests unchanged.

Validation passed: affected package tests, focused race tests for both conflict-adjacent paths, git diff checks, index-plugin domain guards, full self-review, and exact-head preflight. The PR remains Ready; fresh CI is running.

@VioletQwQ-0

Copy link
Copy Markdown
Collaborator Author

The Ubuntu UT failure on 202a5473e was a 60-minute timeout during the heavy-race phase, not a test failure: the raw job log contains no failed package, panic, assertion, or generated ut-report. The previous PR head completed the same UT in about 25 minutes, and recent upstream PRs on the same runner label completed it in about 29–32 minutes. I classified this as runner/infrastructure performance rather than a code defect.

The branch was four commits behind main, so I rebased it onto c6d7a1168. The only conflict was two adjacent independent tests in pkg/sql/compile/alter_test.go; both complete test functions and their assertions were retained.

Validation on new head 6b1bb91fa:

  • go test ./pkg/incrservice ./pkg/sql/colexec/table_clone ./pkg/sql/compile -count=1
  • focused -race tests for the ALTER COPY cleanup, upstream IVF INCLUDE behavior, stable-identity reconciliation, and clone allocator bounds
  • git diff --check upstream/main...HEAD
  • complete mo-self-review: PASS
  • exact-head preflight: PASS review=PASS validation=PENDING

Fresh CI is now running on the rebased head.

@XuPeng-SH XuPeng-SH left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deep-reviewed exact head 6b1bb91fa0.

Verified visible allocator reconciliation (max(requested, copied max, source offset)), stable ColId/name mapping, hidden fake-PK isolation, fulltext/partition hidden-index allocator mapping, rollback cleanup, result lifecycle, integer bounds, snapshot/account handling, and DDL-only performance impact. The previously reported fulltext hidden-table gap is fixed on this head.

Local affected-package tests pass, and exact-head CI/BVT/SCA/coverage are green. No blocking production issue found.

@mergify

mergify Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Queued — the merge queue status continues in this comment ↓.

@mergify

mergify Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

queue

⚠️ Invalid commit message

Details

section not found

@mergify

mergify Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Merge Queue Status

🛑 Queue command has been cancelled

@mergify

mergify Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Merge Queue Status

  • Entered queue2026-07-28 06:28 UTC · Rule: main · triggered by a @mergifyio queue command
  • 🟠 Checks running · in-place
  • 🚫 Left the queue2026-07-28 08:23 UTC · at c61c432cc5fb0fe94f0acdc4fcbf1e7258fcc053

This pull request spent 1 hour 55 minutes 42 seconds in the queue, with no time running CI.

Waiting for
  • any of: [🛡 GitHub branch protection]
    • check-neutral = Matrixone Standlone CI / e2e BVT Test on Linux/x64(LAUNCH, PESSIMISTIC)
    • check-skipped = Matrixone Standlone CI / e2e BVT Test on Linux/x64(LAUNCH, PESSIMISTIC)
    • check-success = Matrixone Standlone CI / e2e BVT Test on Linux/x64(LAUNCH, PESSIMISTIC)
  • any of: [🛡 GitHub branch protection]
    • check-neutral = Matrixone CI / UT Test on Ubuntu/x86
    • check-skipped = Matrixone CI / UT Test on Ubuntu/x86
    • check-success = Matrixone CI / UT Test on Ubuntu/x86
  • any of: [🛡 GitHub branch protection]
    • check-neutral = Matrixone CI / SCA Test on Linux/arm64
    • check-skipped = Matrixone CI / SCA Test on Linux/arm64
    • check-success = Matrixone CI / SCA Test on Linux/arm64
  • any of: [🛡 GitHub branch protection]
    • check-neutral = Matrixone Compose CI / multi cn e2e bvt test docker compose(PROXY)
    • check-skipped = Matrixone Compose CI / multi cn e2e bvt test docker compose(PROXY)
    • check-success = Matrixone Compose CI / multi cn e2e bvt test docker compose(PROXY)
  • any of: [🛡 GitHub branch protection]
    • check-neutral = Matrixone Utils CI / Coverage
    • check-skipped = Matrixone Utils CI / Coverage
    • check-success = Matrixone Utils CI / Coverage
  • any of: [🛡 GitHub branch protection]
    • check-neutral = Matrixone UT Coverage / UT Coverage on Ubuntu/x86
    • check-skipped = Matrixone UT Coverage / UT Coverage on Ubuntu/x86
    • check-success = Matrixone UT Coverage / UT Coverage on Ubuntu/x86
All conditions
  • any of [🛡 GitHub branch protection]:
    • check-neutral = Matrixone Standlone CI / e2e BVT Test on Linux/x64(LAUNCH, PESSIMISTIC)
    • check-skipped = Matrixone Standlone CI / e2e BVT Test on Linux/x64(LAUNCH, PESSIMISTIC)
    • check-success = Matrixone Standlone CI / e2e BVT Test on Linux/x64(LAUNCH, PESSIMISTIC)
  • any of [🛡 GitHub branch protection]:
    • check-neutral = Matrixone CI / UT Test on Ubuntu/x86
    • check-skipped = Matrixone CI / UT Test on Ubuntu/x86
    • check-success = Matrixone CI / UT Test on Ubuntu/x86
  • any of [🛡 GitHub branch protection]:
    • check-neutral = Matrixone CI / SCA Test on Linux/arm64
    • check-skipped = Matrixone CI / SCA Test on Linux/arm64
    • check-success = Matrixone CI / SCA Test on Linux/arm64
  • any of [🛡 GitHub branch protection]:
    • check-neutral = Matrixone Compose CI / multi cn e2e bvt test docker compose(PROXY)
    • check-skipped = Matrixone Compose CI / multi cn e2e bvt test docker compose(PROXY)
    • check-success = Matrixone Compose CI / multi cn e2e bvt test docker compose(PROXY)
  • any of [🛡 GitHub branch protection]:
    • check-neutral = Matrixone Utils CI / Coverage
    • check-skipped = Matrixone Utils CI / Coverage
    • check-success = Matrixone Utils CI / Coverage
  • any of [🛡 GitHub branch protection]:
    • check-neutral = Matrixone UT Coverage / UT Coverage on Ubuntu/x86
    • check-skipped = Matrixone UT Coverage / UT Coverage on Ubuntu/x86
    • check-success = Matrixone UT Coverage / UT Coverage on Ubuntu/x86
  • #review-threads-unresolved = 0 [🛡 GitHub branch protection]
  • github-review-approved [🛡 GitHub branch protection] (documentation)
  • github-review-decision = APPROVED [🛡 GitHub branch protection] (documentation)

Reason

Pull request #26197 has been dequeued

Pull request from fork cannot be queued. This pull request comes from a fork, and Mergify needs the author's permission to update its branch.

The author needs to enable "Allow edits from maintainers" on this pull request.

Failing checks:

Hint

You should look at the reason for the failure and decide if the pull request needs to be fixed or if you want to requeue it.
If you do update this pull request, it will automatically be requeued once the queue conditions match again.
If you think this was a flaky issue instead, you can requeue the pull request, without updating it, by posting a @mergifyio queue comment.

Tick the box to put this pull request back in the merge queue (same as @mergifyio queue).

  • Requeue this pull request

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dequeued kind/bug Something isn't working size/XL Denotes a PR that changes [1000, 1999] lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants