Skip to content

feat: return unsafe integers inside JSON columns as exact strings with supportBigNumbers#4388

Merged
sidorares merged 2 commits into
sidorares:masterfrom
vlasky:feat/json-bigint-exact
Jul 11, 2026
Merged

feat: return unsafe integers inside JSON columns as exact strings with supportBigNumbers#4388
sidorares merged 2 commits into
sidorares:masterfrom
vlasky:feat/json-bigint-exact

Conversation

@vlasky

@vlasky vlasky commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Problem

JSON column values are parsed with plain JSON.parse, which silently rounds integers beyond Number.MAX_SAFE_INTEGER to the nearest double, with no error and no way to detect it after the fact:

// column value in the database: {"id": 9007199254740993}
const [rows] = await conn.query('SELECT doc FROM t');
rows[0].doc.id; // 9007199254740992  (wrong, silently)

This affects MySQL JSON columns and, since #4373, MariaDB JSON columns identified via extended metadata. BIGINT/DECIMAL columns already have an opt-out from this class of corruption (supportBigNumbers), but values inside JSON documents had none short of jsonStrings: true (which gives up parsing entirely).

Change

When the existing supportBigNumbers option is enabled, integers inside parsed JSON documents that cannot be accurately represented as JavaScript Numbers are now returned as exact String objects - mirroring the option's long-documented behaviour for BIGINT/DECIMAL columns:

const conn = mysql.createConnection({ ..., supportBigNumbers: true });
rows[0].doc.id; // '9007199254740993'  (exact)
  • Safe integers, floats (including unsafe-magnitude decimals and exponent notation), numeric strings and nulls are unaffected.
  • Default behaviour without the option is unchanged (existing lossy JSON.parse).
  • jsonStrings: true continues to return the raw (always exact) document text and takes precedence.

Implementation

  • A single packet.parseJson(encoding, supportBigNumbers) method now backs all eight JSON parse sites across the four row parsers (text/binary codegen + static variants), alongside the existing parseVector()/parseGeometryValue() helpers.
  • Exactness uses JSON.parse source access (proposal-json-parse-with-source, Node.js 22+ / V8 12.2), feature-detected once at module load. Older runtimes keep plain JSON.parse behaviour, so nothing changes for them and engines: node >= 8.0 is unaffected.
  • Hot-path cost is one regex test: documents without a run of 16+ digits (the shortest numeral that can exceed Number.MAX_SAFE_INTEGER) take the plain JSON.parse path with no reviver.
  • No parser-cache changes needed: supportBigNumbers is already part of the cache key.
  • Docs updated in ConnectionOptions and QueryOptions typings.

Tests

test/unit/parsers/support-big-numbers-json.test.mts covers all four parsers x both JSON flavours (MySQL Types.JSON and MariaDB extendedFormat === 'json'): unsafe positive/negative integers, safe integers, floats, unsafe-magnitude decimals, exponent notation, numerals inside strings, JSON null, bare root-level scalars, the option-off default, and jsonStrings precedence. The expectations self-adjust via the same feature detection, so CI lanes on Node 18/20 assert the (unchanged) legacy behaviour while 22/24 assert exactness.

Full suite passes locally (214 files) on Node 22 against MySQL 8.4 and MariaDB 11.8; also verified end-to-end over both text and binary protocols on both servers.

…h supportBigNumbers

JSON column values (MySQL JSON, and MariaDB JSON identified via extended
metadata) are parsed with plain JSON.parse, which silently rounds
integers beyond Number.MAX_SAFE_INTEGER to the nearest double: a stored
{"id": 9007199254740993} comes back as 9007199254740992 with no error.

When the existing supportBigNumbers option is enabled, such integers are
now returned as exact String objects, mirroring the option's documented
behaviour for BIGINT and DECIMAL columns; safe integers, floats and
numeric strings are unaffected, and the default behaviour without the
option is unchanged. jsonStrings continues to return the raw (always
exact) document text.

Implemented as a packet.parseJson() method shared by all four row
parsers. Exactness uses JSON.parse source access (proposal-json-parse-
with-source, Node.js 22+); older runtimes fall back to plain JSON.parse.
A 16-digit pre-scan keeps the reviver off the hot path for documents
that cannot contain unsafe integers. supportBigNumbers is already part
of the parser cache key, so no cache changes are needed.
@vlasky vlasky marked this pull request as ready for review July 10, 2026 10:41
Comment thread lib/packets/packet.js Outdated
Comment on lines +55 to +57
// The shortest numeral able to exceed Number.MAX_SAFE_INTEGER has 16
// digits; documents without such a digit run parse exactly with plain
// JSON.parse, keeping the reviver off the hot path

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When comments are exactly what the logic already does next, I think they become unnecessary. The same goes for other comments that explain the implementation. What do you think about using comments only for things that the logic alone doesn't explain, or for the reasoning behind the decisions? 🙋🏻‍♂️

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to prune the comments.

@wellwelwel wellwelwel Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! It looks like GitHub Actions is down (unrelated to the PR). I'm going to restart the workflow (Done).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pruned in f3294cd - dropped everything that narrated the control flow, kept three short rationale notes (the TC39 proposal name/availability behind the feature detection, why the regex is 16 digits, and that parseJson reuses supportBigNumbers' existing BIGINT out-of-range contract rather than adding a new option).

@codecov

codecov Bot commented Jul 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.52%. Comparing base (5034e57) to head (f3294cd).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4388      +/-   ##
==========================================
+ Coverage   91.49%   91.52%   +0.02%     
==========================================
  Files          91       91              
  Lines       14726    14768      +42     
  Branches     1934     1948      +14     
==========================================
+ Hits        13474    13516      +42     
  Misses       1252     1252              
Flag Coverage Δ
compression-0 90.80% <100.00%> (+0.02%) ⬆️
compression-1 91.50% <100.00%> (+0.02%) ⬆️
static-parser-0 90.38% <100.00%> (+0.14%) ⬆️
static-parser-1 90.62% <100.00%> (+0.13%) ⬆️
tls-0 91.05% <100.00%> (+0.02%) ⬆️
tls-1 91.30% <100.00%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@sidorares sidorares merged commit a26ff14 into sidorares:master Jul 11, 2026
104 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants