Skip to content

Fix integer overflow in the number parser digit counter - #1134

Merged
byroot merged 1 commit into
ruby:masterfrom
fuzzyrichie:fix-parse-digits-int-wrap
Nov 10, 2026
Merged

byroot merged 1 commit into
ruby:masterfrom
fuzzyrichie:fix-parse-digits-int-wrap

Conversation

@fuzzyrichie

Copy link
Copy Markdown
Contributor

Background

json_parse_digits returns the number of digits it consumed as (int)(state->cursor - start) (ext/json/ext/parser/parser.c). The counter is then used for all the fast-path routing of numbers:

int mantissa_digits = json_parse_digits(state, &mantissa);      // line 1480
...
if (RB_LIKELY(mantissa_digits <= MAX_FAST_INTEGER_SIZE))         // 19, json_decode_integer

A JSON number whose digit run reaches 2^31 digits (a ≥ 2GiB document) wraps the counter negative, so mantissa_digits <= 19 is true on a uint64 accumulator that long since wrapped mod 2^64 — and the parser silently returns a small garbage Integer instead of the number's true value:

$ ruby -rjson -e 'p JSON.parse(File.read("huge.json"))[0].bit_length'
63        # a 2_147_483_664-digit number cannot be 63 bits

The same wrap also defeats the exponent_digits >= 20 guard (parser.c:1519), letting a wrapped uint64 exponent accumulator through the INT64_MAX check.

Fix

Widen the digit counters to rb_len_t — the same direction as #1133:

  • json_parse_digits returns rb_len_t (both return sites)
  • mantissa_digits, fractional_digits, exponent_digits, decimal_point_pos in the number entry
  • json_decode_integer / json_decode_float take rb_len_t mantissa_digits

The routing comparisons (<= 19, == MAX_FAST_UINT64_SIZE, > 18, >= 20, + exponent < -307/-324) then see the true count and route huge numbers to the existing large-number paths (json_decode_large_integer / json_decode_large_float), which already derive lengths from the raw text as rb_len_t — no new limits, no behavior change for any input that fits the old counters.

Validation

  • Full test suite on the patched build: 612 tests, 4,040 assertions, 0 failures, 0 errors (parser, ext parser, minefield, resumable, float fallback, encoding, fixtures, coder, generator, common interface)
  • Empirical, at the wrap scale (stock ruby 3.4.12, json gem from this tree): a 2,147,483,664-digit document [ + 1×2³¹⁺¹⁶ + ]:
    • before: returns a 63-bit Integer in 0.5s (wrong value)
    • after: correctly enters the large-number path (sustained conversion; the correct path routes through rb_cstr2inum)
  • Feasible-size differential: every numeric boundary probe is byte-identical before/after — UINT64_MAX and +1, INT64_MAX/MIN, 19/20/21 digits, 1e308/1e309, 5e-324/4.9e-324, 100k-digit bignum (EXACT vs Kernel::Integer both sides)

Notes

  • No regression test included, deliberately: the wrap needs a ≥ 2GiB digit run, and the corrected path converts it via rb_cstr2inum, which is quadratic in the digit count — neither side of the test is feasible in CI. Happy to add one if you want a unit-level C harness instead.
  • Java parser: java/src/json/ext/Parser.java keeps the same int counters (parseDigits → int mantissa_digits / int exponent_digits, lines ~633/~659); I could not build JRuby here, so I left it out of this change rather than ship an untested patch. Happy to follow up.
  • Found by a hostile-input audit of the number parser (source analysis + boundary batteries); the ≥ 2GiB single-number document is admittedly an extreme input — I'd call this a correctness fix, not a security issue.

Follows #1133 (rb_len_t migration).

@byroot

byroot commented Nov 10, 2026

Copy link
Copy Markdown
Member

Thanks for the patch, but rb_len_t isn't the right type to use here. rb_len_t is for the length of Ruby objects (e.g. Array, Hash), just use size_t or ssize_t here.

json_parse_digits returns the number of digits consumed as an
(int)(cursor - start).  A JSON number whose digit run is 2^31 or
longer (a >= 2GiB document) wraps the counter negative, and the
mantissa_digits <= MAX_FAST_INTEGER_SIZE fast-path check on a long
wrapped uint64 accumulator then returns a small garbage Integer
instead of the number's true value:

    $ ruby -rjson -e 'p JSON.parse(File.read("huge.json"))[0].bit_length'
    63        # a 2_147_483_664 digit number cannot be 63 bits

The same wrap also defeats the exponent_digits >= 20 guard, letting
a wrapped exponent accumulator reach the exponent.

Widen the digit counters from int to ssize_t, so the routing
comparisons (<= 19, == 20, > 18, >= 20) see the true count and
route huge numbers to the existing large-number paths, which
already derive their lengths from the raw text.  ssize_t keeps the
signed semantics the int counters relied on -- decimal_point_pos
starts at -1, and json_decode_float does signed arithmetic in
mantissa_digits + exponent < -307 with a negative int64 exponent --
which an unsigned size_t would break, while still being wide enough
to hold the true digit count.

Not feasible to regression-test from Ruby: the wrap needs a
>= 2GiB digit run, and the corrected path converts it through
rb_cstr2inum which is quadratic in the digit count.  Validated
empirically against a 2_147_483_664 digit document: before this
change it returns a 63 bit Integer in 0.5s; after it correctly
enters the large-number path.  All numeric boundary probes
(uint64 max, 19/20/21 digits, subnormals, 100k digit bignum) are
byte-identical before and after, and the full test suite passes.

Note: java/src/json/ext/Parser.java keeps the same int counters;
I could not build JRuby here so I left it out of this change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@fuzzyrichie
fuzzyrichie force-pushed the fix-parse-digits-int-wrap branch from 0cb79ca to e4dfd77 Compare November 10, 2026 18:32
@fuzzyrichie

Copy link
Copy Markdown
Contributor Author

Thanks, makes sense — switched the digit counters from rb_len_t to ssize_t (force-pushed).

I went with ssize_t rather than size_t because these counters need to stay signed: decimal_point_pos starts at -1, and json_decode_float does signed arithmetic in mantissa_digits + exponent < -307 / < -324 where exponent is a negative int64_t — an unsigned size_t would flip those comparisons via the usual-arithmetic-conversions and break the float-underflow guard. ssize_t keeps the exact semantics the old int counters had while being wide enough for the true digit count.

The pre-existing rb_len_t len params on json_decode_large_integer / json_decode_large_float are Ruby-object-length-shaped, so I left those as-is.

@byroot
byroot merged commit 3f91317 into ruby:master Nov 10, 2026
42 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants