Skip to content

fix(brainztableinator): widen MusicBrainz Discogs ID columns to BIGINT#404

Closed
SimplicityGuy wants to merge 1 commit into
mainfrom
fix/musicbrainz-discogs-id-bigint
Closed

fix(brainztableinator): widen MusicBrainz Discogs ID columns to BIGINT#404
SimplicityGuy wants to merge 1 commit into
mainfrom
fix/musicbrainz-discogs-id-bigint

Conversation

@SimplicityGuy

Copy link
Copy Markdown
Owner

Problem

Brainztableinator fails to insert MusicBrainz records with integer out of range — most visibly on releases. With requeue=True, the failing message loops forever.

Root cause: The extractor extracts Discogs IDs as i64 (extractor/src/jsonl_parser.rs:33), and Discogs IDs have crossed the int4 ceiling (2,147,483,647). But the musicbrainz schema declared the four Discogs-ID columns as INTEGER:

  • musicbrainz.artists.discogs_artist_id
  • musicbrainz.labels.discogs_label_id
  • musicbrainz.releases.discogs_release_id
  • musicbrainz.release_groups.discogs_master_id

These columns are written directly by brainztableinator.py (lines 523, 568, 602, 639).

Fix

Both halves are required:

  1. CREATE TABLE columns → BIGINT — fixes fresh deployments.
  2. _MUSICBRAINZ_MIGRATIONS (ALTER COLUMN … TYPE BIGINT) — fixes existing deployments. CREATE TABLE IF NOT EXISTS never alters a pre-existing table, so already-deployed databases keep their int4 columns without an explicit ALTER. ALTER … TYPE BIGINT performs no table rewrite when the column is already BIGINT, so it is safe to run on every startup. Ordered after table creation and before the indexes that reference these columns.

Tests

  • New TestMusicBrainzDiscogsIdColumns: asserts CREATE TABLE uses BIGINT, a widening migration exists per column, one migration per column, and correct execution ordering (tables → migrations → indexes).
  • Idempotency check updated to exempt ALTER … TYPE (inherently idempotent, no IF NOT EXISTS form exists).
  • Statement-count assertions updated for the added migrations.
  • brainztableinator/README.md schema docs updated to BIGINT.

All schema-init tests pass; uv run mypy . clean; pre-commit hooks pass.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DipaE4GgAEomY77fw9tWDk

Discogs IDs are emitted by the extractor as i64 and have crossed the
int4 limit (2,147,483,647), causing "integer out of range" failures on
insert in brainztableinator (releases especially). With requeue=True the
failing message loops forever.

- CREATE TABLE columns discogs_{artist,label,release,master}_id → BIGINT
  (fixes fresh deployments)
- Add _MUSICBRAINZ_MIGRATIONS: ALTER COLUMN ... TYPE BIGINT for each,
  run after tables and before indexes. CREATE TABLE IF NOT EXISTS never
  alters a pre-existing table, so existing deployments need the ALTER.
  ALTER TYPE is a no-op (no rewrite) when already BIGINT, so it is safe
  to run on every startup.
- Regression tests + idempotency-check exemption for ALTER ... TYPE
- Update brainztableinator/README.md schema docs to BIGINT

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DipaE4GgAEomY77fw9tWDk
@SimplicityGuy

Copy link
Copy Markdown
Owner Author

Closing as superseded — this fix was already merged to main via #376 (identical BIGINT columns + ALTER COLUMN migrations + README update). No functional difference remains.

@SimplicityGuy SimplicityGuy deleted the fix/musicbrainz-discogs-id-bigint branch July 2, 2026 03:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant