fix(brainztableinator): widen MusicBrainz Discogs ID columns to BIGINT#404
Closed
SimplicityGuy wants to merge 1 commit into
Closed
fix(brainztableinator): widen MusicBrainz Discogs ID columns to BIGINT#404SimplicityGuy wants to merge 1 commit into
SimplicityGuy wants to merge 1 commit into
Conversation
Discogs IDs are emitted by the extractor as i64 and have crossed the
int4 limit (2,147,483,647), causing "integer out of range" failures on
insert in brainztableinator (releases especially). With requeue=True the
failing message loops forever.
- CREATE TABLE columns discogs_{artist,label,release,master}_id → BIGINT
(fixes fresh deployments)
- Add _MUSICBRAINZ_MIGRATIONS: ALTER COLUMN ... TYPE BIGINT for each,
run after tables and before indexes. CREATE TABLE IF NOT EXISTS never
alters a pre-existing table, so existing deployments need the ALTER.
ALTER TYPE is a no-op (no rewrite) when already BIGINT, so it is safe
to run on every startup.
- Regression tests + idempotency-check exemption for ALTER ... TYPE
- Update brainztableinator/README.md schema docs to BIGINT
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DipaE4GgAEomY77fw9tWDk
Owner
Author
|
Closing as superseded — this fix was already merged to main via #376 (identical BIGINT columns + ALTER COLUMN migrations + README update). No functional difference remains. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Brainztableinator fails to insert MusicBrainz records with
integer out of range— most visibly on releases. Withrequeue=True, the failing message loops forever.Root cause: The extractor extracts Discogs IDs as
i64(extractor/src/jsonl_parser.rs:33), and Discogs IDs have crossed theint4ceiling (2,147,483,647). But themusicbrainzschema declared the four Discogs-ID columns asINTEGER:musicbrainz.artists.discogs_artist_idmusicbrainz.labels.discogs_label_idmusicbrainz.releases.discogs_release_idmusicbrainz.release_groups.discogs_master_idThese columns are written directly by
brainztableinator.py(lines 523, 568, 602, 639).Fix
Both halves are required:
CREATE TABLEcolumns →BIGINT— fixes fresh deployments._MUSICBRAINZ_MIGRATIONS(ALTER COLUMN … TYPE BIGINT) — fixes existing deployments.CREATE TABLE IF NOT EXISTSnever alters a pre-existing table, so already-deployed databases keep theirint4columns without an explicitALTER.ALTER … TYPE BIGINTperforms no table rewrite when the column is alreadyBIGINT, so it is safe to run on every startup. Ordered after table creation and before the indexes that reference these columns.Tests
TestMusicBrainzDiscogsIdColumns: assertsCREATE TABLEusesBIGINT, a widening migration exists per column, one migration per column, and correct execution ordering (tables → migrations → indexes).ALTER … TYPE(inherently idempotent, noIF NOT EXISTSform exists).brainztableinator/README.mdschema docs updated toBIGINT.All schema-init tests pass;
uv run mypy .clean; pre-commit hooks pass.🤖 Generated with Claude Code
https://claude.ai/code/session_01DipaE4GgAEomY77fw9tWDk