Repository navigation
vm: (1..).to_set never came back — refuse an endless range - #853
Conversation
```console $ timeout 6 rbgo -e '(1..).to_set'; echo $? 124 ``` On `main` today. `Enumerable#to_set` enumerated the endless range forever, allocating as it went. **One expression**, from any untrusted Ruby an embedding host evaluates — and unlike the repr recursion fixed in #852 there is nothing to recover from, because nothing faults: no stack overflows, no panic is raised, the process simply never returns. ## How it hid: every other member of the family was already guarded | | on `main` | |---|---| | `(1..).to_a` | `RangeError: cannot convert endless range to an array` | | `(1..).last` | `RangeError: cannot get the last element of endless range` | | `(1..).max` | `RangeError: cannot get the maximum of endless range` | | `(1..).size` | `Infinity` | | **`(1..).to_set`** | **hangs** | `Range#to_set` is **new in Ruby 4.0**, so there was no 3.4-era guard to copy — and its spec sits behind `ruby_version_is "4.0"`, which the gate cannot reach while the shim resolves at 3.4.1 (#799). The version sweep recorded `core/range/to_set_spec.rb` as **"failed to load"**; it had not failed to load, the per-file `TIMEOUT` had killed it. I only found that out by running the file by hand and watching it not finish. ## MRI's own, and the message is asserted character for character range.c at tag v4.0.9: ```c static VALUE range_to_set(int argc, VALUE *argv, VALUE range) { if (NIL_P(RANGE_END(range))) { rb_raise(rb_eRangeError, "cannot convert endless range to a set"); } return rb_call_super(argc, argv); } ``` `core/range/to_set_spec.rb:22` asserts that wording exactly. The check is on `end`, not on either endpoint, so a **beginless** range still fails from the iteration as before (`TypeError: can't iterate from NilClass`) — that is a test. **Unconditional, not version-gated.** No spec asserts the old behaviour — nothing can assert a hang — so the only thing a version gate would preserve is the way to wedge the interpreter. ## Measured, with the control `core/range/to_set_spec.rb` on the pinned corpus: | shim `CUR_VERSION` | binary | result | |---|---|---| | 3.4.1 | this branch | `pass=3 fail=1` — **`BASELINE` records 3**, unchanged | | 4.0.9 | this branch | **`pass=4 fail=1`**, exit 0 | | 4.0.9 | `main` | **exit 124 — hung**, no `RBGO_RESULT` at all | The third row is the control, and it is what makes the second mean something. The `fail=1` in both of my rows is **pre-existing and unrelated**: it is `Range#to_set given positional arguments | instantiates an object of provided as the first argument set class` — rbgo returns a plain `Set` where the spec wants the given subclass. It is why `BASELINE` says 3 for a file with 4 active examples at 3.4.1, and this change neither fixes nor touches it. ## Also found, and deliberately NOT changed here `min`/`max` with a custom comparison block diverge from MRI by one word: ``` rbgo : cannot get the minimum of unbounded range with custom comparison method MRI : cannot get the minimum of endless range with custom comparison method (range.c:1649) MRI : cannot get the maximum of beginless range with custom comparison method (range.c:1762) ``` rbgo uses **one** format string with a `%s` for min/max (builtins.go:11903) and the word `unbounded` for both, where MRI uses **opposite** words — `endless` for min, `beginless` for max. So it is not a one-word fix but an asymmetry that is easy to invert, and **no spec asserts either message**, so nothing would catch getting it backwards. Filed rather than folded into a crash fix. Tests in `crash_hardening_test.go`, beside the recursion guards, since a hang belongs to the same class. `gofmt` clean, frozen prelude regenerated (`prelude_frozen_test.go` is what would catch forgetting that). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Green and ready — held back on purpose, not stalled25 of 25 checks green, It is not merged because ⛔ I did not get that right the first time tonight. On #851 I wrote So: whoever picks this up, it merges as-is once the lease frees. The measurement
— and the 🤖 Generated with Claude Code |
…g the answer (#863) * conformance: cmd/oracle — ask the reference interpreter without losing the answer Every parity fix in this repository starts with *"what does MRI actually do here?"*, and the obvious ways to ask it throw part of the answer away: - `ruby probe.rb > out 2>&1` discards the **exit status**, which for half these probes IS the answer — a SyntaxError is what they are looking for; - a pipe into `grep` or `tail` discards whichever part did not match the pattern the asker already had in mind, which is the part worth reading; - running the reference and rbgo as two separate commands invites comparing a fresh measurement against a remembered one. So this tool PARSES NOTHING. Both streams are copied through verbatim with their byte counts, the status is printed as a number, and the only judgement it makes is byte equality between engines. ```sh go run ./scripts/conformance/rubyspec/cmd/oracle \ -script probe.rb /usr/bin/ruby ./rbgo-main ./rbgo-branch ``` Three engines is the shape the measurement rules want: reference, control (`main`), treatment (the branch). A difference between the last two that the reference does not share is the defect, or the fix. ## ⛔ The row that justifies comparing BYTES and not lengths Measured while writing #856, on the warning probe: ```text === the engines DISAGREED === /usr/bin/ruby vs ./rbgo-main: stderr 1008 vs 407 bytes /usr/bin/ruby vs ./rbgo-branch: stderr 1008 vs 1008 bytes ``` The second row is **the same length and different content**: the branch had gained the warning but still differed from MRI in the line number and the quote style. A comparison by length — which is what reading two `wc -c` outputs amounts to, and what I had been doing by eye — would have called that parity. `-selftest` plants exactly that case, `'$x'` versus `'$y'`, equal length and one byte apart, and refuses to pass unless the comparator sees it. It runs in the BINARY as well as under `go test`, like `cmd/phantom`'s controls, because a comparator stuck on "agreed" would make every measurement taken with this tool read as parity — the most reassuring possible way for it to be broken. ## Exit status, and the distinction that is tested `0` agreed (or one engine named), `1` disagreed, `2` refused — no script, an unreadable script, no engine, or an engine that would not start. That last one is deliberate and has its own test: *"could not ask"* and *"answered differently"* call for opposite responses — fix the invocation, or go read the divergence — so a missing interpreter must never arrive as a disagreement. Same split, and same exit codes, as `cmd/ratchet`. A probe that never returns is killed at `-timeout` (default 1m) and REPORTED as `KILLED` rather than hanging the measurement. That is not hypothetical: #853 was an endless range that never came back, and it was first recorded as "failed to load" because a timeout had killed it silently. ## Tests The fake engines are `/bin/sh` scripts, not Ruby, on purpose: the tool knows nothing about Ruby, and a test needing an interpreter installed would measure the machine instead of the tool. They cover the exit status surviving, the two streams not being swapped (a single buffer could not tell you), the three refusals, the kill path, and — as the control for the disagreement cases — two engines that agree. ⛔ One of those tests failed first time and the tool was right: I had written `stderr 0 vs 26 bytes` where `warning: something\n` is 19. The byte counts being printed at all is what made that a one-line correction instead of a theory. README: a section under `cmd/guards`, with the disagreement above as the worked example. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * conformance: keep the oracle probes, with the answers written down Four probes under `cmd/oracle/testdata/probes`, one question each, and a README holding what the reference answered and which upstream function owns the rule. The shapes take a minute to write and the reference run takes a second. What costs time is working out **which** function in MRI decides — `parse.y` does not decide constant-ness, `rb_sym_constant_char_p` does; `rb_gvar_get` does not warn, the undef getter it installs does — and which rows separate the plausible-but-wrong implementation from the right one. A probe thrown away is that work thrown away. ⛔ Three of these were written, lost to a reboot that emptied the scratchpad, and written again. They belong in the repository, which is where the rule about durable directories has been pointing all along. | probe | the row that earns its keep | |---|---| | `globals.rb` | silence after `$x = nil` — the subject is "never assigned", not "reads as nil" | | `constant_names.rb` | the **titlecase** digraphs `Dž`/`Lj` are constants and the **caseless** letters `日`/`א`/`ب` are not, so "not lowercase" is wrong in both directions | | `dynamic_constant.rb` | `Class.new { X = 1 }` inside a `def` is refused while `define_method(:m) { X = 1 }` at class level is not — the rule is syntactic, not about what will run as a method | | `basicobject_conversion.rb` | a BasicObject has **no** `#respond_to?`, which is not the same as having an overridden one | Replayed against `/usr/bin/ruby` before committing: 6 constant names of 12 characters, 16 `dynamic constant assignment` refusals (the seventeenth refusal is `class definition in method body`, a different message from a different check), 4 clean conversions, and the globals probe's six blocks. The README also records the two differences against the installed 2.6.10 that are **expected** and are not rbgo defects — the quote style (`` `$x' `` before 3.4, `'$x'` after, which is why ruby/spec matches on `` [`'] ``) and 2.6's one-line lag in a warning's position — and says plainly what that interpreter cannot answer at all: it predates numbered parameters, core `Set` and everything else 3.x–4.0 added, so a probe about those gets silence, which reads exactly like a negative. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…binary (#869) * conformance: cmd/specdelta — hold the corpus still and vary only the binary `cmd/ratchet` judges one sweep against `BASELINE`. That is the right gate for *"may this land"* and the wrong instrument for *"what does this change do"*: a baseline comparison folds in every difference between the recording machine and this one, and the version move in #799 needs the other reading — hold the corpus and the machine fixed, vary only the interpreter, and name every file that moved. ```console $ specdelta -root ~/ruby-spec-corpus -dir core/set -quiet ./rbgo-main ./rbgo-409 file control treat delta core/set/flatten_merge_spec.rb 3 0 -3 core/set/to_s_spec.rb 2 4 +2 core/set/divide_spec.rb 9 8 -1 core/set/equal_value_spec.rb 4 3 -1 core/set/exclusion_spec.rb 3 2 -1 core/set/flatten_spec.rb 10 9 -1 core/set/join_spec.rb 5 4 -1 core/set/proper_superset_spec.rb 3 2 -1 core/set/superset_spec.rb 3 2 -1 57 file(s): control 175 passing, treatment 167 passing, -8 9 file(s) moved, 8 regressed ``` That table took the `Set` half of the version move from "eight files to work through" to **two defects**, which is written up on #799. ## What it refuses, and why each refusal is not a 1 Exit **2**, never 1, for: an engine that would not start, output with an unreadable `RBGO_RESULT` line, two result lines in one run, and an **empty population**. The last one matters most — a mistyped `-dir` would otherwise print `0 files, no regressions` and exit 0, which is the most convincing possible way for this tool to tell someone their change is safe. `parse` refuses a missing or non-numeric field rather than defaulting it to zero, for `cmd/ratchet`'s reason: a zero meaning "unreadable" and a zero meaning "nothing passed" lead to opposite conclusions. A file that produced no result line at all is *named* — "DID NOT RUN on the treatment" — rather than counted as zero, which is the distinction that made an endless range look like a load failure in #853. ## ⛔ The second channel, and the invariant I got wrong first The `RBGO_RESULT pass=…` line comes out of the **subject's own stdout** (#843), so a subject that miscounts can report any total it likes. The `RBGO_DETAIL` lines are an independent channel, and reading them rather than the tally is what caught a figure misattributed between two branches earlier today. My first version asserted `details == fail + error` and fired on its first real run, against `core/set/set_spec.rb` — `pass=0 fail=0 error=0 skip=1` with one `skip` detail, entirely well-formed. Reading spec_helper.rb instead of assuming its shape gives this: | kind | counter | detail line | | |---|---|---|---| | `fail` | `$RB_FAIL` | yes | :983, :987 | | `error` | `$RB_ERROR` | yes | :995, :1029, :1102 | | `skip` | `$RB_SKIP` | yes | :990 | | `skip` | `$RB_SKIP` | **no** | :970 — a version/platform guard | | `hookerror` | **none** | yes | :965, :1020 — before/after `:all` | So no equality is derivable from the four printed numbers. What holds is `details >= fail + error`, because those two kinds each push exactly one line, and a shortfall means detail lines went missing. That is what the tool checks, and the test names every row above — including the two that made equality wrong in opposite directions. The last row of that table is a defect in the harness rather than in this tool: a `before(:all)` that raises is reported in the detail lines and counted in no tally at all. Filed separately. ## Tests `parse` first and hardest, since that is where a silent wrong answer would live: seven malformed result lines that must each be an error, the no-result case that must not be, the detail-shortfall table, the empty-population refusal, the four usage refusals, and an end-to-end pass over fake `/bin/sh` engines in both directions — an improvement must exit 0 and the same pair reversed must exit 1. The engines are shell scripts, not Ruby, because the tool knows nothing about Ruby and a test needing an interpreter installed would measure the machine. ⛔ One test failed first time and the tool was right: I had written a "good" result line as `fail=3 error=1` with three detail lines, which is four expected and three given. The message it printed said "check the arithmetic", and the arithmetic to check was mine. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * specdelta: one run per engine cannot tell a flake from a difference The tool's first real use reported ``` core/random/bytes_spec.rb 9 8 -1 ``` between two binaries that differ by **one file and three constants** — a version string. Three runs of each said why: | | run 1 | run 2 | run 3 | |---|---|---|---| | control | 9 | 9 | **8** | | treatment | **8** | 9 | 9 | A coin flip, attributed to the change. `core/random/bytes_spec.rb` carries no version guard at all, which is what made the row suspicious enough to re-run — but a row that *did* sit behind a guard would have looked explained and gone into a report. ## `-repeat N` Above 1, each file runs N times per engine and the tool keeps the **lowest** pass count — not the mean and not the last, because the question a reader asks of this table is "did anything get worse" and the answer must not rest on a lucky run. An engine that disagrees with itself marks the file `UNSTABLE`, and its delta is **excluded from the verdict**, so the exit status stops depending on which way the coin landed: ```console $ specdelta -dir core/random -repeat 3 ./rbgo-main ./rbgo-409 core/random/bytes_spec.rb 8 8 +0 ⛔ UNSTABLE: an engine disagreed with itself, delta excluded ... 10 file(s): control 65 passing, treatment 65 passing, +0 0 file(s) moved, 0 regressed ``` The default stays 1, because three passes over `core/` is three times the twenty-odd minutes — and the flag's help says plainly that the regression verdict is only trustworthy above 1. `-repeat 0` is a usage refusal rather than a silent no-op. ## Tests The instability test asserts **both** modes, so the difference between them is visible rather than claimed: with `-repeat 1` the flake reads as a regression and exits 1; with `-repeat 3` the file is named `UNSTABLE`, counted as `0 regressed`, and exits 0. Its fake engine scores 8 the first time it is asked and 9 after, which is the direction that looks like a regression on the first run. Two controls beside it: a stable engine must not be called unstable, and a stable pair must stay at exit 0. Also pinned, because `abs` measured 0% and the ordering is documented behaviour: the loudest movement prints first. It needs two files — with one, the sort comparator is never called — and they are named so that alphabetical order and movement order disagree. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> * conformance: README — what cmd/specdelta is for A section beside `cmd/oracle`'s, with the `core/set` table as the worked example and the three refusals spelled out: the flake detection, the two-channel check and the empty-population exit 2. The `Set` row is the point of the tool in one line — it took that half of the 4.0 version move from "eight files to work through" to **two defects**, because six of the eight are examples 4.0 stopped asking for and a delta cannot say so. Only the `ruby_version_is` guards can, which is why the section says to read them before reading the numbers. Co-authored-by: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
vm:
(1..).to_setnever came back — refuse an endless rangeOn
maintoday.Enumerable#to_setenumerated the endless range forever,allocating as it went. One expression, from any untrusted Ruby an embedding
host evaluates — and unlike the repr recursion fixed in #852 there is nothing to
recover from, because nothing faults: no stack overflows, no panic is raised,
the process simply never returns.
How it hid: every other member of the family was already guarded
main(1..).to_aRangeError: cannot convert endless range to an array(1..).lastRangeError: cannot get the last element of endless range(1..).maxRangeError: cannot get the maximum of endless range(1..).sizeInfinity(1..).to_setRange#to_setis new in Ruby 4.0, so there was no 3.4-era guard to copy —and its spec sits behind
ruby_version_is "4.0", which the gate cannot reachwhile the shim resolves at 3.4.1 (#799). The version sweep recorded
core/range/to_set_spec.rbas "failed to load"; it had not failed to load,the per-file
TIMEOUThad killed it. I only found that out by running the fileby hand and watching it not finish.
MRI's own, and the message is asserted character for character
range.c at tag v4.0.9:
core/range/to_set_spec.rb:22asserts that wording exactly. The check is onend, not on either endpoint, so a beginless range still fails from theiteration as before (
TypeError: can't iterate from NilClass) — that is a test.Unconditional, not version-gated. No spec asserts the old behaviour —
nothing can assert a hang — so the only thing a version gate would preserve is
the way to wedge the interpreter.
Measured, with the control
core/range/to_set_spec.rbon the pinned corpus:CUR_VERSIONpass=3 fail=1—BASELINErecords 3, unchangedpass=4 fail=1, exit 0mainRBGO_RESULTat allThe third row is the control, and it is what makes the second mean something.
The
fail=1in both of my rows is pre-existing and unrelated: it isRange#to_set given positional arguments | instantiates an object of provided as the first argument set class— rbgo returns a plainSetwhere the spec wantsthe given subclass. It is why
BASELINEsays 3 for a file with 4 activeexamples at 3.4.1, and this change neither fixes nor touches it.
Also found, and deliberately NOT changed here
min/maxwith a custom comparison block diverge from MRI by one word:rbgo uses one format string with a
%sfor min/max (builtins.go:11903) andthe word
unboundedfor both, where MRI uses opposite words —endlessformin,
beginlessfor max. So it is not a one-word fix but an asymmetry that iseasy to invert, and no spec asserts either message, so nothing would catch
getting it backwards. Filed rather than folded into a crash fix.
Tests in
crash_hardening_test.go, beside the recursion guards, since a hangbelongs to the same class.
gofmtclean, frozen prelude regenerated(
prelude_frozen_test.gois what would catch forgetting that).Co-Authored-By: Claude Opus 5 noreply@anthropic.com
🤖 Generated with Claude Code