From 55af8ef72bfb6c152adf8ef8a10acb01422405de Mon Sep 17 00:00:00 2001 From: Richard Desmarais Date: Tue, 10 Nov 2026 12:42:56 -0500 Subject: [PATCH] Shrink parsed strings that contained escape sequences json_string_unescape allocates its output buffer with the size of the raw (escaped) input, then decodes into it: \uXXXX escapes compress 6 raw bytes to at most 4 output bytes (12 for a surrogate pair), \n and friends compress 2 to 1. rb_str_set_len only sets the length, so every parsed string containing escapes retains the raw size as buffer capacity for as long as the parsed document is alive: JSON.parse('["' + 'm\u00e9lange\u00e0 ' * 3000 + '"]') # 200 such 50 KB strings: 4.8 MB of retained capacity hold # 7.8 MB of content (61% slack) Use rb_str_resize instead, which on shrink both embeds strings that now fit inline (freeing their heap buffer) and reallocates still-larger buffers to the exact length, following MRI's own shrink policy. Measured on ruby 3.4.12 with a 12.5 MB document of 50 KB escaped strings: retained string capacity drops from 12.52 MB to 7.76 MB (content 7.75 MB, slack 61% -> 0%), with parse time unchanged (0.129 -> 0.131 ms per 130 KB document, within noise) and no new allocations. Documents without escapes are unaffected (fast path never enters unescape). --- ext/json/ext/parser/parser.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/ext/json/ext/parser/parser.c b/ext/json/ext/parser/parser.c index b456a4e23..47541a4cb 100644 --- a/ext/json/ext/parser/parser.c +++ b/ext/json/ext/parser/parser.c @@ -1075,7 +1075,14 @@ NOINLINE(static) VALUE json_string_unescape(JSON_ParserState *state, JSON_Parser MEMCPY(buffer, p, char, stringEnd - p); buffer += stringEnd - p; } - rb_str_set_len(result, buffer - bufferStart); + // The buffer was sized for the raw (escaped) input, but the unescaped + // content can be much shorter (e.g. \uXXXX escapes: 6 raw bytes per at + // most 4 output bytes, 12 for a surrogate pair). rb_str_set_len would + // leave the raw size as capacity for the string's whole lifetime; + // rb_str_resize shrinks oversized buffers (embedding short strings and + // reallocating large ones to the exact length), so parsed strings only + // retain the bytes they actually hold. + rb_str_resize(result, (rb_len_t)(buffer - bufferStart)); if (symbolize) { result = rb_str_intern(result);