Skip to content

vm: (1..).to_set never came back — refuse an endless range - #853

Merged
tannevaled merged 1 commit into
mainfrom
range-to-set-endless
Nov 1, 2026
Merged

tannevaled merged 1 commit into
mainfrom
range-to-set-endless

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

vm: (1..).to_set never came back — refuse an endless range

$ timeout 6 rbgo -e '(1..).to_set'; echo $?
124

On main today. Enumerable#to_set enumerated the endless range forever,
allocating as it went. One expression, from any untrusted Ruby an embedding
host evaluates — and unlike the repr recursion fixed in #852 there is nothing to
recover from, because nothing faults: no stack overflows, no panic is raised,
the process simply never returns.

How it hid: every other member of the family was already guarded

on main
(1..).to_a RangeError: cannot convert endless range to an array
(1..).last RangeError: cannot get the last element of endless range
(1..).max RangeError: cannot get the maximum of endless range
(1..).size Infinity
(1..).to_set hangs

Range#to_set is new in Ruby 4.0, so there was no 3.4-era guard to copy —
and its spec sits behind ruby_version_is "4.0", which the gate cannot reach
while the shim resolves at 3.4.1 (#799). The version sweep recorded
core/range/to_set_spec.rb as "failed to load"; it had not failed to load,
the per-file TIMEOUT had killed it. I only found that out by running the file
by hand and watching it not finish.

MRI's own, and the message is asserted character for character

range.c at tag v4.0.9:

static VALUE
range_to_set(int argc, VALUE *argv, VALUE range)
{
    if (NIL_P(RANGE_END(range))) {
        rb_raise(rb_eRangeError, "cannot convert endless range to a set");
    }
    return rb_call_super(argc, argv);
}

core/range/to_set_spec.rb:22 asserts that wording exactly. The check is on
end, not on either endpoint, so a beginless range still fails from the
iteration as before (TypeError: can't iterate from NilClass) — that is a test.

Unconditional, not version-gated. No spec asserts the old behaviour —
nothing can assert a hang — so the only thing a version gate would preserve is
the way to wedge the interpreter.

Measured, with the control

core/range/to_set_spec.rb on the pinned corpus:

shim CUR_VERSION binary result
3.4.1 this branch pass=3 fail=1 — BASELINE records 3, unchanged
4.0.9 this branch pass=4 fail=1, exit 0
4.0.9 main exit 124 — hung, no RBGO_RESULT at all

The third row is the control, and it is what makes the second mean something.

The fail=1 in both of my rows is pre-existing and unrelated: it is
Range#to_set given positional arguments | instantiates an object of provided as the first argument set class — rbgo returns a plain Set where the spec wants
the given subclass. It is why BASELINE says 3 for a file with 4 active
examples at 3.4.1, and this change neither fixes nor touches it.

Also found, and deliberately NOT changed here

min/max with a custom comparison block diverge from MRI by one word:

rbgo  : cannot get the minimum of unbounded range with custom comparison method
MRI   : cannot get the minimum of endless  range with custom comparison method   (range.c:1649)
MRI   : cannot get the maximum of beginless range with custom comparison method  (range.c:1762)

rbgo uses one format string with a %s for min/max (builtins.go:11903) and
the word unbounded for both, where MRI uses opposite words — endless for
min, beginless for max. So it is not a one-word fix but an asymmetry that is
easy to invert, and no spec asserts either message, so nothing would catch
getting it backwards. Filed rather than folded into a crash fix.

Tests in crash_hardening_test.go, beside the recursion guards, since a hang
belongs to the same class. gofmt clean, frozen prelude regenerated
(prelude_frozen_test.go is what would catch forgetting that).

Co-Authored-By: Claude Opus 5 noreply@anthropic.com

🤖 Generated with Claude Code

```console
$ timeout 6 rbgo -e '(1..).to_set'; echo $?
124
```

On `main` today. `Enumerable#to_set` enumerated the endless range forever,
allocating as it went. **One expression**, from any untrusted Ruby an embedding
host evaluates — and unlike the repr recursion fixed in #852 there is nothing to
recover from, because nothing faults: no stack overflows, no panic is raised,
the process simply never returns.

## How it hid: every other member of the family was already guarded

| | on `main` |
|---|---|
| `(1..).to_a` | `RangeError: cannot convert endless range to an array` |
| `(1..).last` | `RangeError: cannot get the last element of endless range` |
| `(1..).max` | `RangeError: cannot get the maximum of endless range` |
| `(1..).size` | `Infinity` |
| **`(1..).to_set`** | **hangs** |

`Range#to_set` is **new in Ruby 4.0**, so there was no 3.4-era guard to copy —
and its spec sits behind `ruby_version_is "4.0"`, which the gate cannot reach
while the shim resolves at 3.4.1 (#799). The version sweep recorded
`core/range/to_set_spec.rb` as **"failed to load"**; it had not failed to load,
the per-file `TIMEOUT` had killed it. I only found that out by running the file
by hand and watching it not finish.

## MRI's own, and the message is asserted character for character

range.c at tag v4.0.9:

```c
static VALUE
range_to_set(int argc, VALUE *argv, VALUE range)
{
    if (NIL_P(RANGE_END(range))) {
        rb_raise(rb_eRangeError, "cannot convert endless range to a set");
    }
    return rb_call_super(argc, argv);
}
```

`core/range/to_set_spec.rb:22` asserts that wording exactly. The check is on
`end`, not on either endpoint, so a **beginless** range still fails from the
iteration as before (`TypeError: can't iterate from NilClass`) — that is a test.

**Unconditional, not version-gated.** No spec asserts the old behaviour —
nothing can assert a hang — so the only thing a version gate would preserve is
the way to wedge the interpreter.

## Measured, with the control

`core/range/to_set_spec.rb` on the pinned corpus:

| shim `CUR_VERSION` | binary | result |
|---|---|---|
| 3.4.1 | this branch | `pass=3 fail=1` — **`BASELINE` records 3**, unchanged |
| 4.0.9 | this branch | **`pass=4 fail=1`**, exit 0 |
| 4.0.9 | `main` | **exit 124 — hung**, no `RBGO_RESULT` at all |

The third row is the control, and it is what makes the second mean something.

The `fail=1` in both of my rows is **pre-existing and unrelated**: it is
`Range#to_set given positional arguments | instantiates an object of provided as
the first argument set class` — rbgo returns a plain `Set` where the spec wants
the given subclass. It is why `BASELINE` says 3 for a file with 4 active
examples at 3.4.1, and this change neither fixes nor touches it.

## Also found, and deliberately NOT changed here

`min`/`max` with a custom comparison block diverge from MRI by one word:

```
rbgo  : cannot get the minimum of unbounded range with custom comparison method
MRI   : cannot get the minimum of endless  range with custom comparison method   (range.c:1649)
MRI   : cannot get the maximum of beginless range with custom comparison method  (range.c:1762)
```

rbgo uses **one** format string with a `%s` for min/max (builtins.go:11903) and
the word `unbounded` for both, where MRI uses **opposite** words — `endless` for
min, `beginless` for max. So it is not a one-word fix but an asymmetry that is
easy to invert, and **no spec asserts either message**, so nothing would catch
getting it backwards. Filed rather than folded into a crash fix.

Tests in `crash_hardening_test.go`, beside the recursion guards, since a hang
belongs to the same class. `gofmt` clean, frozen prelude regenerated
(`prelude_frozen_test.go` is what would catch forgetting that).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@tannevaled

Copy link
Copy Markdown
Contributor Author

Green and ready — held back on purpose, not stalled

25 of 25 checks green, CLEAN, bound to 9deb20fe (the head of this
branch). Nothing here needs re-running.

It is not merged because go-embedded-ruby/ruby is leased by another session
working the plan9 port (#855), and that session keeps renewing: I watched the
lease move 05:27 → 06:10 → 06:47 while a gated agentsync claim -wait sat on
it. agentsync claim … && ghmerge … refused and the merge did not run, which is
the protocol doing its job.

⛔ I did not get that right the first time tonight. On #851 I wrote
claim ; ghmerge instead of claim && ghmerge, the claim printed REFUSED with
the holder's name, and the merge ran anyway — into a repository someone else
held. I told the holder immediately with the list of commits so they could
rebase rather than discover it. The semicolon is the whole of that mistake, and
it is why this one is waiting.

So: whoever picks this up, it merges as-is once the lease frees. The measurement
in the description stands —

shim CUR_VERSION binary result
3.4.1 this branch pass=3 fail=1, BASELINE records 3
4.0.9 this branch pass=4 fail=1, exit 0
4.0.9 main exit 124 — hung

— and the fail=1 in both of my rows is the pre-existing
to_set(SetSubclass) divergence, unrelated and already counted in the baseline.

🤖 Generated with Claude Code

@tannevaled
tannevaled merged commit 0535b98 into main Nov 1, 2026
25 checks passed
@tannevaled
tannevaled deleted the range-to-set-endless branch November 1, 2026 07:50
tannevaled added a commit that referenced this pull request Nov 2, 2026
…g the answer (#863)

* conformance: cmd/oracle — ask the reference interpreter without losing the answer

Every parity fix in this repository starts with *"what does MRI actually do
here?"*, and the obvious ways to ask it throw part of the answer away:

- `ruby probe.rb > out 2>&1` discards the **exit status**, which for half these
  probes IS the answer — a SyntaxError is what they are looking for;
- a pipe into `grep` or `tail` discards whichever part did not match the pattern
  the asker already had in mind, which is the part worth reading;
- running the reference and rbgo as two separate commands invites comparing a
  fresh measurement against a remembered one.

So this tool PARSES NOTHING. Both streams are copied through verbatim with their
byte counts, the status is printed as a number, and the only judgement it makes
is byte equality between engines.

```sh
go run ./scripts/conformance/rubyspec/cmd/oracle \
  -script probe.rb /usr/bin/ruby ./rbgo-main ./rbgo-branch
```

Three engines is the shape the measurement rules want: reference, control
(`main`), treatment (the branch). A difference between the last two that the
reference does not share is the defect, or the fix.

## ⛔ The row that justifies comparing BYTES and not lengths

Measured while writing #856, on the warning probe:

```text
=== the engines DISAGREED ===
  /usr/bin/ruby vs ./rbgo-main:   stderr 1008 vs 407 bytes
  /usr/bin/ruby vs ./rbgo-branch: stderr 1008 vs 1008 bytes
```

The second row is **the same length and different content**: the branch had
gained the warning but still differed from MRI in the line number and the quote
style. A comparison by length — which is what reading two `wc -c` outputs
amounts to, and what I had been doing by eye — would have called that parity.

`-selftest` plants exactly that case, `'$x'` versus `'$y'`, equal length and one
byte apart, and refuses to pass unless the comparator sees it. It runs in the
BINARY as well as under `go test`, like `cmd/phantom`'s controls, because a
comparator stuck on "agreed" would make every measurement taken with this tool
read as parity — the most reassuring possible way for it to be broken.

## Exit status, and the distinction that is tested

`0` agreed (or one engine named), `1` disagreed, `2` refused — no script, an
unreadable script, no engine, or an engine that would not start.

That last one is deliberate and has its own test: *"could not ask"* and
*"answered differently"* call for opposite responses — fix the invocation, or go
read the divergence — so a missing interpreter must never arrive as a
disagreement. Same split, and same exit codes, as `cmd/ratchet`.

A probe that never returns is killed at `-timeout` (default 1m) and REPORTED as
`KILLED` rather than hanging the measurement. That is not hypothetical: #853 was
an endless range that never came back, and it was first recorded as "failed to
load" because a timeout had killed it silently.

## Tests

The fake engines are `/bin/sh` scripts, not Ruby, on purpose: the tool knows
nothing about Ruby, and a test needing an interpreter installed would measure the
machine instead of the tool. They cover the exit status surviving, the two
streams not being swapped (a single buffer could not tell you), the three
refusals, the kill path, and — as the control for the disagreement cases — two
engines that agree.

⛔ One of those tests failed first time and the tool was right: I had written
`stderr 0 vs 26 bytes` where `warning: something\n` is 19. The byte counts being
printed at all is what made that a one-line correction instead of a theory.

README: a section under `cmd/guards`, with the disagreement above as the worked
example.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

* conformance: keep the oracle probes, with the answers written down

Four probes under `cmd/oracle/testdata/probes`, one question each, and a README
holding what the reference answered and which upstream function owns the rule.

The shapes take a minute to write and the reference run takes a second. What
costs time is working out **which** function in MRI decides — `parse.y` does not
decide constant-ness, `rb_sym_constant_char_p` does; `rb_gvar_get` does not
warn, the undef getter it installs does — and which rows separate the
plausible-but-wrong implementation from the right one. A probe thrown away is
that work thrown away.

⛔ Three of these were written, lost to a reboot that emptied the scratchpad, and
written again. They belong in the repository, which is where the rule about
durable directories has been pointing all along.

| probe | the row that earns its keep |
|---|---|
| `globals.rb` | silence after `$x = nil` — the subject is "never assigned", not "reads as nil" |
| `constant_names.rb` | the **titlecase** digraphs `Dž`/`Lj` are constants and the **caseless** letters `日`/`א`/`ب` are not, so "not lowercase" is wrong in both directions |
| `dynamic_constant.rb` | `Class.new { X = 1 }` inside a `def` is refused while `define_method(:m) { X = 1 }` at class level is not — the rule is syntactic, not about what will run as a method |
| `basicobject_conversion.rb` | a BasicObject has **no** `#respond_to?`, which is not the same as having an overridden one |

Replayed against `/usr/bin/ruby` before committing: 6 constant names of 12
characters, 16 `dynamic constant assignment` refusals (the seventeenth refusal
is `class definition in method body`, a different message from a different
check), 4 clean conversions, and the globals probe's six blocks.

The README also records the two differences against the installed 2.6.10 that
are **expected** and are not rbgo defects — the quote style (`` `$x' `` before
3.4, `'$x'` after, which is why ruby/spec matches on `` [`'] ``) and 2.6's
one-line lag in a warning's position — and says plainly what that interpreter
cannot answer at all: it predates numbered parameters, core `Set` and everything
else 3.x–4.0 added, so a probe about those gets silence, which reads exactly
like a negative.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
tannevaled added a commit that referenced this pull request Nov 2, 2026
…binary (#869)

* conformance: cmd/specdelta — hold the corpus still and vary only the binary

`cmd/ratchet` judges one sweep against `BASELINE`. That is the right gate for
*"may this land"* and the wrong instrument for *"what does this change do"*: a
baseline comparison folds in every difference between the recording machine and
this one, and the version move in #799 needs the other reading — hold the corpus
and the machine fixed, vary only the interpreter, and name every file that moved.

```console
$ specdelta -root ~/ruby-spec-corpus -dir core/set -quiet ./rbgo-main ./rbgo-409
file                                        control   treat  delta
core/set/flatten_merge_spec.rb                    3       0     -3
core/set/to_s_spec.rb                             2       4     +2
core/set/divide_spec.rb                           9       8     -1
core/set/equal_value_spec.rb                      4       3     -1
core/set/exclusion_spec.rb                        3       2     -1
core/set/flatten_spec.rb                         10       9     -1
core/set/join_spec.rb                             5       4     -1
core/set/proper_superset_spec.rb                  3       2     -1
core/set/superset_spec.rb                         3       2     -1

57 file(s): control 175 passing, treatment 167 passing, -8
9 file(s) moved, 8 regressed
```

That table took the `Set` half of the version move from "eight files to work
through" to **two defects**, which is written up on #799.

## What it refuses, and why each refusal is not a 1

Exit **2**, never 1, for: an engine that would not start, output with an
unreadable `RBGO_RESULT` line, two result lines in one run, and an **empty
population**. The last one matters most — a mistyped `-dir` would otherwise
print `0 files, no regressions` and exit 0, which is the most convincing
possible way for this tool to tell someone their change is safe.

`parse` refuses a missing or non-numeric field rather than defaulting it to
zero, for `cmd/ratchet`'s reason: a zero meaning "unreadable" and a zero meaning
"nothing passed" lead to opposite conclusions. A file that produced no result
line at all is *named* — "DID NOT RUN on the treatment" — rather than counted as
zero, which is the distinction that made an endless range look like a load
failure in #853.

## ⛔ The second channel, and the invariant I got wrong first

The `RBGO_RESULT pass=…` line comes out of the **subject's own stdout** (#843),
so a subject that miscounts can report any total it likes. The `RBGO_DETAIL`
lines are an independent channel, and reading them rather than the tally is what
caught a figure misattributed between two branches earlier today.

My first version asserted `details == fail + error` and fired on its first real
run, against `core/set/set_spec.rb` — `pass=0 fail=0 error=0 skip=1` with one
`skip` detail, entirely well-formed. Reading spec_helper.rb instead of assuming
its shape gives this:

| kind | counter | detail line | |
|---|---|---|---|
| `fail` | `$RB_FAIL` | yes | :983, :987 |
| `error` | `$RB_ERROR` | yes | :995, :1029, :1102 |
| `skip` | `$RB_SKIP` | yes | :990 |
| `skip` | `$RB_SKIP` | **no** | :970 — a version/platform guard |
| `hookerror` | **none** | yes | :965, :1020 — before/after `:all` |

So no equality is derivable from the four printed numbers. What holds is
`details >= fail + error`, because those two kinds each push exactly one line,
and a shortfall means detail lines went missing. That is what the tool checks,
and the test names every row above — including the two that made equality wrong
in opposite directions.

The last row of that table is a defect in the harness rather than in this tool:
a `before(:all)` that raises is reported in the detail lines and counted in no
tally at all. Filed separately.

## Tests

`parse` first and hardest, since that is where a silent wrong answer would live:
seven malformed result lines that must each be an error, the no-result case that
must not be, the detail-shortfall table, the empty-population refusal, the four
usage refusals, and an end-to-end pass over fake `/bin/sh` engines in both
directions — an improvement must exit 0 and the same pair reversed must exit 1.
The engines are shell scripts, not Ruby, because the tool knows nothing about
Ruby and a test needing an interpreter installed would measure the machine.

⛔ One test failed first time and the tool was right: I had written a "good"
result line as `fail=3 error=1` with three detail lines, which is four expected
and three given. The message it printed said "check the arithmetic", and the
arithmetic to check was mine.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

* specdelta: one run per engine cannot tell a flake from a difference

The tool's first real use reported

```
core/random/bytes_spec.rb      9       8     -1
```

between two binaries that differ by **one file and three constants** — a version
string. Three runs of each said why:

| | run 1 | run 2 | run 3 |
|---|---|---|---|
| control | 9 | 9 | **8** |
| treatment | **8** | 9 | 9 |

A coin flip, attributed to the change. `core/random/bytes_spec.rb` carries no
version guard at all, which is what made the row suspicious enough to re-run —
but a row that *did* sit behind a guard would have looked explained and gone
into a report.

## `-repeat N`

Above 1, each file runs N times per engine and the tool keeps the **lowest**
pass count — not the mean and not the last, because the question a reader asks
of this table is "did anything get worse" and the answer must not rest on a
lucky run. An engine that disagrees with itself marks the file `UNSTABLE`, and
its delta is **excluded from the verdict**, so the exit status stops depending
on which way the coin landed:

```console
$ specdelta -dir core/random -repeat 3 ./rbgo-main ./rbgo-409
core/random/bytes_spec.rb   8   8   +0  ⛔ UNSTABLE: an engine disagreed with itself, delta excluded
...
10 file(s): control 65 passing, treatment 65 passing, +0
0 file(s) moved, 0 regressed
```

The default stays 1, because three passes over `core/` is three times the
twenty-odd minutes — and the flag's help says plainly that the regression
verdict is only trustworthy above 1.

`-repeat 0` is a usage refusal rather than a silent no-op.

## Tests

The instability test asserts **both** modes, so the difference between them is
visible rather than claimed: with `-repeat 1` the flake reads as a regression
and exits 1; with `-repeat 3` the file is named `UNSTABLE`, counted as
`0 regressed`, and exits 0. Its fake engine scores 8 the first time it is asked
and 9 after, which is the direction that looks like a regression on the first
run. Two controls beside it: a stable engine must not be called unstable, and a
stable pair must stay at exit 0.

Also pinned, because `abs` measured 0% and the ordering is documented
behaviour: the loudest movement prints first. It needs two files — with one, the
sort comparator is never called — and they are named so that alphabetical order
and movement order disagree.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

* conformance: README — what cmd/specdelta is for

A section beside `cmd/oracle`'s, with the `core/set` table as the worked example
and the three refusals spelled out: the flake detection, the two-channel check
and the empty-population exit 2.

The `Set` row is the point of the tool in one line — it took that half of the
4.0 version move from "eight files to work through" to **two defects**, because
six of the eight are examples 4.0 stopped asking for and a delta cannot say so.
Only the `ruby_version_is` guards can, which is why the section says to read
them before reading the numbers.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant