Migration to make two sequences BIGINT. by ggainey · Pull Request #2235 · pulp/pulp_container

ggainey · 2026-03-03T19:54:18Z

This addresses the failure noted in https://issues.redhat.com/browse/PULP-890 .

gerrod3 · 2026-03-03T21:22:28Z

Is this related to this issue: #2080. I see they share the same JIRA.

gerrod3 · 2026-03-03T21:23:34Z

+# This is an IDEMPOTENT change - it can be backported, because applying it
+# multiple times is harmless.


What branches are you thinking of backporting to? I thought the problem with backporting migrations was the divergence between the sequences.

There are several possible issues with backporting migrations, and "not idempotent" is the biggest one. The comment is a reminder that this change doesn't have that problem.

Backporting this will definitely require a hand-backport, since the "previous migration" in its dependency-list won't be in older branches. It'll still work though. If, say, in container/2.20 we mark this as 0040_bigint_sequences.py, dependent on 0039, and an installation then updates to 2.28 - 0040_add_remote_repo_filter.py will apply, and then and eventually 0047_bigint_sequences will REapply, do nothing, and be fine.

I'm almost certain this will still run into a Django error about inconsistent migration history down the road.
What if we execute that SQL (if necessary) as part of a post_migration hook?

Do we not need to change the Django class of the field too? (Or is that impossible, since we don't have the model of the through table?)

We can change the default that Django chooses for AutoFields - see https://docs.djangoproject.com/en/4.2/ref/settings/#default-auto-field. But that only helps for future-models.

It's not that we don't have a Model - we do, it's (for this particular problem) BlobManifest. The problem is that that Model doesn't come with a "primary key" - just a multi-field uniqueness-constraint. In Django5.2, we can define multi-key PKs (see https://docs.djangoproject.com/en/6.0/topics/composite-primary-key/), which will make the point moot. Some day.

I can definitely investigate a post-migration hook - would make backporting easier/safer. Drafting this PR while I experiment.

gerrod3 · 2026-03-03T21:26:17Z

+    ]
+
+    operations = [
+        migrations.RunSQL(update_sequences_to_bigint),


Should there be a reverse_sql specified? What about elidable?

It's not elidable (unless/until we take one of the paths mentioned in the hackmd I've been writing up).

Reversing the change is only possible if done before the sequence hits 2G - if its bigger than that, then trying to get from BIGINT back to INTEGER is going to (at best) cause problems. Generally speaking, this should never be reversed.

ggainey · 2026-03-03T21:38:48Z

Is this related to this issue: #2080. I see they share the same JIRA.

It is - somehow didn't notice #2080 when looking for open issues :(

dralley · 2026-03-03T22:19:04Z

@ggainey @gerrod3 going forwards, how do we ensure that non-BaseModel-derived models use the correct sequence type?

We can update these specific fields to BigAutoField, but we should probably also ensure that new such fields are set to BigAutoField by default via:
https://docs.djangoproject.com/en/6.0/ref/settings/#std-setting-DEFAULT_AUTO_FIELD
https://docs.djangoproject.com/en/6.0/topics/db/models/#automatic-primary-key-fields

And/or migrate to UUID pks via BaseModel

ggainey · 2026-03-03T22:21:35Z

@ggainey @gerrod3 going forwards, how do we ensure that non-BaseModel-derived models use the correct sequence type?

Yes - see some thoughts here

dralley · 2026-03-03T22:22:30Z

+# multiple times is harmless.
+update_sequences_to_bigint = """
+ALTER SEQUENCE container_blobmanifest_id_seq AS BIGINT;
+ALTER SEQUENCE container_manifestlistmanifest_id_seq AS BIGINT;


Does Django still recognize this as AutoField, or does it see it as BigAutoField? I suppose even if the former is true it still works?

dralley · 2026-03-03T22:46:33Z

For what this is, as long as it doesn't confuse Django either now or for a future BigAutoField migration, LGTM

fixes pulp#2080.

patchback · 2026-03-09T13:46:28Z

Backport to 2.22: 💚 backport PR created

✅ Backport PR branch: patchback/backports/2.22/76528efe32e82c4f53f9cb6102e85d82b2528eac/pr-2235

Backported as #2247

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

patchback · 2026-03-09T13:46:28Z

Backport to 2.26: 💚 backport PR created

✅ Backport PR branch: patchback/backports/2.26/76528efe32e82c4f53f9cb6102e85d82b2528eac/pr-2235

Backported as #2248

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

patchback · 2026-03-09T13:46:28Z

Backport to 2.20: 💔 cherry-picking failed — conflicts found

❌ Failed to cleanly apply 76528ef on top of patchback/backports/2.20/76528efe32e82c4f53f9cb6102e85d82b2528eac/pr-2235

Backporting merged PR #2235 into main

Ensure you have a local repo clone of your fork. Unless you cloned it
from the upstream, this would be your origin remote.
Make sure you have an upstream repo added as a remote too. In these
instructions you'll refer to it by the name upstream. If you don't
have it, here's how you can add it:
```
$ git remote add upstream https://github.com/pulp/pulp_container.git
```

Ensure you have the latest copy of upstream and prepare a branch
that will hold the backported code:

$ git fetch upstream
$ git checkout -b patchback/backports/2.20/76528efe32e82c4f53f9cb6102e85d82b2528eac/pr-2235 upstream/2.20

Now, cherry-pick PR Migration to make two sequences BIGINT. #2235 contents into that branch:
```
$ git cherry-pick -x 76528efe32e82c4f53f9cb6102e85d82b2528eac
```
If it'll yell at you with something like fatal: Commit 76528efe32e82c4f53f9cb6102e85d82b2528eac is a merge but no -m option was given., add -m 1 as follows instead:
```
$ git cherry-pick -m1 -x 76528efe32e82c4f53f9cb6102e85d82b2528eac
```
At this point, you'll probably encounter some merge conflicts. You must
resolve them in to preserve the patch from PR Migration to make two sequences BIGINT. #2235 as close to the
original as possible.

Push this branch to your fork on GitHub:

$ git push origin patchback/backports/2.20/76528efe32e82c4f53f9cb6102e85d82b2528eac/pr-2235

Create a PR, ensure that the CI is green. If it's not — update it so that
the tests and any other checks pass. This is it!
Now relax and wait for the maintainers to process your pull request
when they have some cycles to do reviews. Don't worry — they'll tell you if
any improvements are necessary when the time comes!

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

patchback · 2026-03-09T13:46:28Z

Backport to 2.27: 💚 backport PR created

✅ Backport PR branch: patchback/backports/2.27/76528efe32e82c4f53f9cb6102e85d82b2528eac/pr-2235

Backported as #2250

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

patchback · 2026-03-09T13:46:29Z

Backport to 2.24: 💚 backport PR created

✅ Backport PR branch: patchback/backports/2.24/76528efe32e82c4f53f9cb6102e85d82b2528eac/pr-2235

Backported as #2249

🤖 @patchback
I'm built with octomachinery and
my source is open — https://github.com/sanitizers/patchback-github-app.

github-actions Bot added the no-issue label Mar 3, 2026

gerrod3 reviewed Mar 3, 2026

View reviewed changes

ggainey force-pushed the pulp-890 branch from 73c5fe0 to c6bbe22 Compare March 3, 2026 21:45

github-actions Bot removed the no-issue label Mar 3, 2026

dralley reviewed Mar 3, 2026

View reviewed changes

ggainey marked this pull request as draft March 4, 2026 13:29

ggainey force-pushed the pulp-890 branch 2 times, most recently from 5b1da84 to 8271b39 Compare March 5, 2026 14:28

ggainey marked this pull request as ready for review March 5, 2026 14:53

ggainey marked this pull request as draft March 5, 2026 15:13

Update several auto-created id fields/sequences to be BIGINT.

59054f3

fixes pulp#2080.

ggainey marked this pull request as ready for review March 5, 2026 19:32

ggainey force-pushed the pulp-890 branch from 8271b39 to 59054f3 Compare March 5, 2026 19:33

ggainey requested a review from mdellweg March 5, 2026 21:28

mdellweg approved these changes Mar 9, 2026

View reviewed changes

ggainey mentioned this pull request Mar 9, 2026

Replace sequence-update post_migrate hook with a migration #2246

Open

ggainey merged commit 76528ef into pulp:main Mar 9, 2026
13 checks passed

ggainey added backport-2.20 backport-2.26 backport-2.22 backport-2.24 backport-2.27 labels Mar 9, 2026

		# This is an IDEMPOTENT change - it can be backported, because applying it
		# multiple times is harmless.

Conversation

ggainey commented Mar 3, 2026

Uh oh!

gerrod3 commented Mar 3, 2026

Uh oh!

gerrod3 Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

ggainey Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

mdellweg Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

ggainey Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

gerrod3 Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

ggainey Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

ggainey commented Mar 3, 2026

Uh oh!

dralley commented Mar 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ggainey commented Mar 3, 2026

Uh oh!

dralley Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

dralley commented Mar 3, 2026

Uh oh!

Uh oh!

patchback Bot commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Backport to 2.22: 💚 backport PR created

Uh oh!

patchback Bot commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Backport to 2.26: 💚 backport PR created

Uh oh!

patchback Bot commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Backport to 2.20: 💔 cherry-picking failed — conflicts found

Backporting merged PR #2235 into main

Uh oh!

patchback Bot commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Backport to 2.27: 💚 backport PR created

Uh oh!

patchback Bot commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Backport to 2.24: 💚 backport PR created

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

dralley commented Mar 3, 2026 •

edited

Loading

patchback Bot commented Mar 9, 2026 •

edited

Loading

patchback Bot commented Mar 9, 2026 •

edited

Loading

patchback Bot commented Mar 9, 2026 •

edited

Loading

patchback Bot commented Mar 9, 2026 •

edited

Loading

patchback Bot commented Mar 9, 2026 •

edited

Loading