Skip to content

Ruby 4.0.9: the figure, the version, and a limitation that is now resolved - #44

Merged
tannevaled merged 2 commits into
mainfrom
ruby-40-figures
Nov 4, 2026
Merged

tannevaled merged 2 commits into
mainfrom
ruby-40-figures

Conversation

@tannevaled

Copy link
Copy Markdown
Contributor

rbgo reports RUBY_VERSION 4.0.9 since ruby#882, and BASELINE was re-recorded at that version on the pinned runner that enforces it: 23,511 → 23,673.

⛔ My grep found one of four; the tool's arithmetic found the rest

$ git grep -n "23,511\|23511" -- content
content/en/_index.md:13:...        # ONE hit

One hit reads exactly like "one place to fix", and that is what I concluded. Then tools/conformancefigure said:

4 total claim(s) and 2 version claim(s) checked against the gate's 23511
content/en/_index.md:13: claims 23673, the gate records 23511

Four checked, one disagreeing — so three claims still agreed with the old figure, in a directory I had just declared clean. They are written with a space, 23 511, which neither of my patterns could match:

content/en/conformance.md:209
content/en/conformance.md:262
content/en/roadmap.md:22

What caught it was not re-reading the page. It was a count that refused to add up. A tool that prints only its findings would have agreed with me; one that prints its denominator makes a too-narrow search self-refuting. Worth keeping in mind for any future checker here.

The known-limitation row is resolved, not deleted

The RUBY_VERSION row asked for "a CI sweep that moves CUR_VERSION without regenerating the baseline, so the ratchet names every file that moves in both directions." That sweep was run — both engines built from one commit minus the version string, over core and language together. The row now carries its result, including the two cautions the numbers need:

  • +144 is a net, and 13 regressed is not 13 defects: six of the thirteen lost only examples the corpus stops asking for, each with a CLOSED guard in the census;
  • core/objectspace/_id2ref_spec.rb reading 10 → 0 is eleven examples below 4.0 against one above it, not ten regressions.

The 27 that went red are the move's real remaining work, listed per item on ruby#799.

Also corrected

CUR_VERSION = RUBY_VERSION, so the shim cannot disagree with the interpreter — the "two copies, agreeing" wording predates ruby#845. And the baseline is recorded on the runner that enforces it, not on a laptop: one laptop reads +9 against CI for no change in the code.

Verified: go run ./tools/conformancefigure --root content/en now prints 4 total claim(s) and 2 version claim(s) checked against the gate's 23673 at Ruby 4.0.9.

🤖 Generated with Claude Code

tannevaled and others added 2 commits November 3, 2026 15:21
…olved

rbgo reports RUBY_VERSION 4.0.9 (ruby#882) and BASELINE was re-recorded at that
version, on the pinned runner that enforces it: 23,511 -> 23,673.

## ⛔ My grep found one of four, and the tool's ARITHMETIC found the rest

Searching content for `23,511` and `23511` returned a single hit, and that
reads exactly like "one place to fix". `tools/conformancefigure` then said:

    4 total claim(s) and 2 version claim(s) checked ...
    content/en/_index.md:13: claims 23673, the gate records 23511

Four claims checked, ONE disagreeing -- so three claims still agreed with the
old figure, in a file I had just declared clean. They are written with a SPACE,
`23 511`, which neither of my patterns could match:

    content/en/conformance.md:209
    content/en/conformance.md:262
    content/en/roadmap.md:22

What caught it was not re-reading the page. It was a count that refused to add
up: 4 checked against 1 flagged. A tool that reports its DENOMINATOR makes a
narrow search self-refuting; one that printed only its findings would have
agreed with me.

## The known-limitation row is resolved, not deleted

The `RUBY_VERSION` row asked for "a CI sweep that moves `CUR_VERSION` without
regenerating the baseline, so the ratchet names every file that moves in both
directions". That sweep was run. The row now carries its result rather than
vanishing, including the two cautions the numbers need:

  - `+144` is a NET, and `13 regressed` is not 13 defects: six of the thirteen
    lost only examples the corpus stops asking for, each with a CLOSED guard in
    the census;
  - `core/objectspace/_id2ref_spec.rb` reads 10 -> 0 and that is eleven
    examples below 4.0 against ONE above it, not ten regressions.

The 27 examples that WENT RED are the move's real remaining work, and no pass
total shows them.

## Also stated

`CUR_VERSION = RUBY_VERSION`, so the shim cannot disagree with the interpreter
-- the "two copies, agreeing" wording was true before ruby#845 and is no longer
the situation. And the baseline was re-recorded on the pinned runner that
enforces it, not on a laptop: one laptop reads +9 against CI for no change in
the code.

⛔ This cannot land before ruby#882. Verified, not assumed: run against main
today the checker prints all six disagreements and exits 1.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…iming it

core/random/bytes_spec.rb draws 8 or 9 passing examples from run to run. The
2026-11-03 re-record banked the 9 and the next ordinary CI run went red, so
ruby#885 pinned that row to its floor: the gate's total is 23,672, not 23,673.

Refreshing the figure exposed a second defect in conformancefigure itself. It
reported "4 total claim(s) checked" on a tree that holds five, and "5" on the
landing tree that holds six. The two it skipped were:

    **The gate's current total is **23 673****          (conformance.md:157)
    The sum of those rows is <b>23,673</b>              (landing list.html:73)

The first uses "total is", which the alternation did not list. The second puts
two words between "sum" and "is", and the alternation spelled out only the
three determiners met so far — "the sum is", "its sum is", "whose sum is".
Neither line was ever a candidate, so both runs looked complete while leaving a
stale figure on a published page.

The verdicts the tool did reach were all correct. What made the gap visible was
the denominator it prints next to them: 4 against a grep that found 5. A tool
that printed only its findings would have agreed with the omission.

claimBefore now accepts "(sum|total) is" and "(sum|total) of <words> is", and
both lines are vocabulary cases, so a future narrowing fails before it ships.
Nothing speculative was added: every alternative is a phrasing one of the two
sites uses today. The landing page's ratio line ("93.3% pass (22,488 of
24,116)") is already a must-miss case and still misses.

Checked locally against both trees: docs 5/5 checked and clean, landing 6/6
checked and clean, against the gate's 23672 at Ruby 4.0.9.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@tannevaled
tannevaled merged commit 48a9bd6 into main Nov 4, 2026
2 checks passed
@tannevaled
tannevaled deleted the ruby-40-figures branch November 4, 2026 11:55
tannevaled added a commit to go-embedded-ruby/go-embedded-ruby.github.io that referenced this pull request Nov 4, 2026
v0.6.0's vocabulary spelled out "the sum is", "its sum is" and "whose sum is",
so this page's "The sum of those rows is <b>23,673</b>" was never a candidate.
The lane printed "5 total claim(s) checked" and passed, on a page holding six,
one of them stale. go-embedded-ruby/docs#44 widened the pattern; v0.7.0 carries
it.

Run with the exact published command this lane runs, fetched through the proxy
against this tree:

    $ go run github.com/go-embedded-ruby/docs/tools/conformancefigure@v0.7.0 --root .
    go: downloading github.com/go-embedded-ruby/docs v0.7.0
    6 total claim(s) and 1 version claim(s) checked against the gate's 23672 at Ruby 4.0.9 (derived from RUBY_VERSION)

Six, not five, and no disagreement.

The other number that moved was already predicted in the comment this replaces:
"1 version claim … derived from RUBY_VERSION" is go-embedded-ruby/ruby#845
collapsing the two copies of the version into one, and nothing on the page had
to change for it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
tannevaled added a commit to go-embedded-ruby/go-embedded-ruby.github.io that referenced this pull request Nov 4, 2026
* Ruby 4.0.9: the figure, and the version it is counted at

rbgo reports RUBY_VERSION 4.0.9 (ruby#882) and BASELINE was re-recorded at that
version on the pinned runner that enforces it: 23,511 -> 23,673.

Six claims on this page move, and the lane's own checker named every one of
them rather than my reading the page:

    hugo.toml:74                 claims 23673, the gate records 23511
    layouts/_default/list.html:76   "
    layouts/index.html:27           "
    layouts/index.html:62           "
    layouts/index.html:71           "
    layouts/index.html:62: says the gate measures at 4.0.9, it measures at 3.4.1

That last line is the v0.6.0 version claim, and it is the one that mattered
most here. The page said:

    Those guards resolve at Ruby 3.4.1 -- the version this runtime reports --
    so every example behind a `ruby_version_is "4.0"` guard is removed from the
    suite before anything is counted: 61 block guards and 5 inline sites sit at
    that boundary, while the latest MRI release is 4.0.9.

Every word of that is now false, and none of it is a number, so only the
version check could catch it. It now says the opposite, and states what the
move cost: +144 passing and 27 examples that WENT RED -- a figure no pass total
shows, found by comparing the two runs' failure descriptions.

`layouts/_default/list.html:73` is a DATED claim and therefore exempt from the
check; it is updated anyway, because exempt from the gate is not the same as
allowed to be wrong.

⛔ This cannot land before ruby#882. The checker reads the interpreter
repository's main, so until the bump merges it refuses this page -- verified,
not assumed: run against main today it prints the six disagreements above and
exits 1.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* The figure is 23,672, and the page now says why it went down by one

core/random/bytes_spec.rb draws 8 or 9 passing examples from run to run. The
2026-11-03 re-record banked the 9, the next ordinary CI run went red, and
ruby#885 pinned that row to its floor. The gate's total is 23,672.

All six claims on this site move: hugo.toml's blurb, the hero trust line, the
conformance card, the stat tile, and the two baseline bullets in list.html.
The blurb and the first bullet also say what happened, because "23,673 →
23,672" with no explanation reads as a regression and it is the opposite: the
number went down so that the gate stops passing on a coin toss.

conformancefigure could not see one of those six. "The sum of those rows is
<b>23,673</b>" puts two words between "sum" and "is", and its vocabulary
spelled out only "the sum is", "its sum is", "whose sum is" — so the line was
never a candidate and the run reported "5 checked" on a page holding six. That
is fixed in go-embedded-ruby/docs; this PR will bump the pinned version in
deploy-pages.yml once the fix is released, so the gate here can refuse the line
it just missed.

Verified locally with the fixed tool: 6 claims checked, 0 disagreements,
against the gate's 23672 at Ruby 4.0.9.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Pin the figure gate to v0.7.0, which can see one more line of this page

v0.6.0's vocabulary spelled out "the sum is", "its sum is" and "whose sum is",
so this page's "The sum of those rows is <b>23,673</b>" was never a candidate.
The lane printed "5 total claim(s) checked" and passed, on a page holding six,
one of them stale. go-embedded-ruby/docs#44 widened the pattern; v0.7.0 carries
it.

Run with the exact published command this lane runs, fetched through the proxy
against this tree:

    $ go run github.com/go-embedded-ruby/docs/tools/conformancefigure@v0.7.0 --root .
    go: downloading github.com/go-embedded-ruby/docs v0.7.0
    6 total claim(s) and 1 version claim(s) checked against the gate's 23672 at Ruby 4.0.9 (derived from RUBY_VERSION)

Six, not five, and no disagreement.

The other number that moved was already predicted in the comment this replaces:
"1 version claim … derived from RUBY_VERSION" is go-embedded-ruby/ruby#845
collapsing the two copies of the version into one, and nothing on the page had
to change for it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant