feat(oracle-gaps): every streamed line starts with `oracle-gaps: <kind>` by KyleAMathews · Pull Request #46 · TanStack/oracles

KyleAMathews · 2026-11-12T19:17:09Z

Summary

Agents act on a gap check's results as they stream. With this PR, every line that the oracle-gaps CLI writes for a check, on stdout and stderr, starts with oracle-gaps: and then its kind. One grep '^oracle-gaps: ' finds all of them, and the word after the prefix filters one kind. This is the maintainer's request of 2026-11-12, relayed by the coordinator. The coordinator also decided:

the same prefixes on stdout and stderr
a done line for a run that a signal stops
no --max-gaps: an agent stops a runaway run itself with SIGINT or SIGTERM, which stops cleanly since fix(oracle-gaps): a per-mutant budget, a heartbeat, signal-safe workers, and no copies beside the tests #40. A fixed cap would cut off a first run with 70 to 130 real gaps.
after the review (2026-11-12): every line of either stream is a line of the stream, with only the two exception blocks below

The line formats:

Kind	Line
`config`	`oracle-gaps: config vitest.config.ts`, or `oracle-gaps: config none, Vitest's default discovery`
`baseline`	`oracle-gaps: baseline pass tests/a.test.ts: 3 tests`, `oracle-gaps: baseline fail tests/b.test.ts: 1 of 2 tests failed`
`baseline` (decisive)	`oracle-gaps: baseline all 4 oracle files pass`, or `oracle-gaps: baseline 1 of 4 oracle files failed (provisional)`
`blocked`	`oracle-gaps: blocked <file> fails with no mutant active at <test>, so the mutants it owns wait for it: …`
`gap`	`oracle-gaps: gap <kind> L<line> <id> <change>`
`progress`	`oracle-gaps: progress judged 25 of 120 mutants: 7 gaps so far, 18 not gaps`, and the `unmutated:` and `mutant phase:` timings
`busy`	`oracle-gaps: busy <mutant id or baseline> in <test file> for <n>s`
`summary`	`oracle-gaps: summary <run id>: 120 mutants, 98 killed, 12 gaps, 10 unowned (warnings). …`
`legend`	`oracle-gaps: legend <class>: <what it means and what to do>`, once for each class of gap the run found
`function`	`oracle-gaps: function fn <name> (<source>) (owners: <files>)`, the heading over that function's gap lines
`note`	`oracle-gaps: note …`: the `--hang-limit` note, Vite's own messages (`note vite: …`), ordinary files left out, the provisional count, unowned counts, the results' path, and where the tests' console output went (`note test output in <path>`)
`done` (last)	`oracle-gaps: done completed, 12 gaps, report <path>/summary.md`, with `(provisional)` after the count when an oracle file failed. Also `oracle-gaps: done refused[, report <path>]`, `oracle-gaps: done error, log <path>`, and `oracle-gaps: done stopped (SIGINT)`

Two kinds of line have no prefix: the lines indented under a failing baseline file (failing:, law:, counterexample:, replay:), and a refusal's refused: and fix: block on stderr. No line is blank.
The decisive baseline line comes before any mutant runs.
The done line is the last line of both stdout and stderr, once each, for every ending, and its status matches the exit code: 0 completed, 2 refused, 3 error, and 130 or 143 stopped.
The stdout carries the summary as lines of the stream (summary, legend, function, gap, note). I chose kinds over leaving the summary out of stdout, so that the terminal still shows each gap class's directive and each function's owners. summary.md holds the report for reading.
A test's own writes to the process's stdout and stderr go to test-output.log in the output directory, and one note names that log.
A check through the daemon prints the same lines, when its check ends. Streaming them while the check runs is F8 of the daemon lane, a later PR.
--help is not a check, so its usage text has no prefix.
Exit codes do not change.

Fixes from the review (F1 to F11)

F1, F2: stdout had the summary's body without the prefix, and a test's raw output. Both now follow the table above.
F3: a signal sent just after the completed line gave two done lines. Once the run writes its ending, a stop signal no longer cuts it off. The supervisor writes its stopped line only when the run did not end with its own code (0, 2, or 3).
F4: a refusal through the daemon dropped the cold run's note vite: line. Vite's notes now go through the check's log, which the daemon relays.
F5: the wrapper config now reads the package's config as Vite does. It awaits a promise default and refuses a default that is not an object with Vite's wording ("config must export or return an object.").
F7: a config with a syntax error printed esbuild's own error block. The CLI now parses the config with esbuild, silent, before Vite bundles it, and refuses a syntax error with esbuild's first error.
F8: the done line ends both streams, as above.
F9 to F11: the README says which endings print no done line, fixes the heartbeat line, and says that --help is not a check.
F6, F12: F6 is a limit below. F12 needs no action.

Fixes from the second review (N1 to N3)

N1: a SIGINT while the config warmed cut off its load. The run reported the error as a refusal, wrote done refused, and died by the signal, so the supervisor wrote done stopped too. Now the run installs its own stop handler first: a stop ends it at once, with no line, unless it is writing its ending. Before it writes its ending, it says so to the supervisor over an IPC channel. The supervisor writes the stopped line only when no such message came, and decides at the run's close.
N2: the check owed the test-output note whenever file a's baseline loaded, which the law does not say. The driver now observes whether the run captured output, the note is owed exactly then, and a run whose file a test wrote must have captured it. A new dimension, consoleOutput, gives file a no writing test.
N3: the supervisor installs its stop handlers before it starts the run. A signal that comes while Node still starts the CLI, before its first line runs, still ends it with no done line, and the README says so.
The test hook: ORACLE_GAPS_TEST_WARM_PROBE lets the oracle stop a run while its config warms. When set, the first warm-up waits on a service in the run's process group that a stop ends, as it ends esbuild's service. Meanwhile a handler that exits 200 ms after the signal holds the process, as Vitest's does. The hook does nothing when unset.

P10: a run whose process dies on its own

When the run's process ended without its ending and without a stop from the CLI, the CLI exited 128 plus the signal and wrote no done line. A V8 out-of-memory abort is a SIGABRT, so it gave exit 134 and no line. The CLI now writes oracle-gaps: done error, the run's process exited with <signal or code> on both streams and exits 3 (the coordinator's decision of 2026-11-12). It passes the run's stderr on, and when that stderr says the heap ran out, the line adds the remedy: give Node more memory (NODE_OPTIONS=--max-old-space-size=<MB>) or run fewer --contexts. The CLI's test-only hooks are now ORACLE_GAPS_TEST_WARM_PROBE and ORACLE_GAPS_TEST_HOLD_EXIT_MS, as the third review asked.

Evidence

Law: law 15 of the end-to-end gap oracle, "the stream's lines are greppable":
- every line of stdout and stderr is a line of a stream kind, or of the two exception blocks, or a line that Node writes for the process
- one decisive baseline line comes before any line about a mutant
- the stdout prints each gap once, on a gap line
- exactly one done line ends each stream, the same on both, of the kind that the exit code says
- a run that a signal reaches after its completed line ends as it completed
- a refusal through the daemon prints the cold run's notes
- a completed run whose tests wrote output names that log once.
The classify oracle's law 8 and the static oracle's refusal law check the same formats from their own boundaries. Authority: the maintainer's request and the coordinator's decisions of 2026-11-12, above.
Reach:
- new states: a stop signal after the completed line, a config refusal through the daemon, refused configNull, and refused configSyntax
- a new config form: promise
- the declared states that law 15 also applies in: blocked by brokenOwner, each refused <fault>, the stop states, and law 9's daemon states.
File a of every fixture has a test that writes to the process's stdout and stderr.

RED: on 8260bda (the reviewed head), with the extended oracles (86d391e):

Oracle	Law	Difference
classify	`stdout`	"the stdout has a line that is not a line of the stream"
static	`report-shaped ending`	"the refusal's stdout is [], not its done line"
e2e, every outcome class	`greppable stream`	"the run wrote a line on stderr that is not a line of the stream: raw stderr from a test"
e2e, configThrows	`greppable stream`	"stdout has 0 done lines"
e2e, configNull	`config`	the null default was accepted
e2e, configSyntax	`greppable stream`	esbuild's own `✘ [ERROR]` line

On main (81a5451), the first version of law 15 was RED in the same three oracles.

GREEN: on this branch, merged with main at cc7bf09:
- the e2e fixed seed and the seven law-15 calibrations: 44 passed at ab18c60, and "a declaration fault: setupThrows" after its fix at 2d09b99
- the four fast gap suites: 33,817 tests
- the unit tests: 114
- tsc --noEmit, prettier, e2e-shards.mjs check, and pr-shards.mjs check
Mutant: I put each hostile mutant into this branch, and the oracle rejected each one under the intended law:
- gapLine without the prefix: classify stdout, "0 gap lines for 1 gaps".
- The refusal's done line without its prefix: static report-shaped ending.
- The stdout without its decisive baseline line: classify stdout.
- The supervisor without its stopped line (nostopline): e2e greppable stream at checkpoint stopped.
- The decisive line logged after the mutant phase (lateverdict): e2e greppable stream at cli.
- The supervisor that always writes its stopped line (stoppedalways, F3): e2e greppable stream at stopped, "stderr has 2 done lines and ends with "oracle-gaps: done stopped (SIGINT)"".
- Vite's notes on the CLI's own stderr (vitenotes, F4): e2e greppable stream at incremental, "the refusal through the daemon at start prints the notes […], and the cold run […]".
Third round:
- RED on 3d1507c (9e93944 production plus the warm-up probe): "every outcome class", law greppable stream, "the run stopped while its config warmed reported a refusal".
- GREEN on 4b5d3f5: the targeted e2e units (every outcome class with the warm-up stop, a load failure beside a hang, setupThrows, configThrows, configSyntax, and the N2 calibration) pass, 6 of 6. The fast suites pass (33,817 tests), and so do the unit tests (114).
- Mutant for N2: the note printed whenever the run ends, greppable stream, "the run names its tests' console output in 1 notes, owed 0 (captured: false)".
- New state: a stop signal while the config warms, and a completed run whose tests wrote nothing. New calibration: a run names a test-output log it never captured, bound to the history "a load failure beside a hang".
P10:
- RED at 19e185a (4b5d3f5 production, with the hook rename): "every outcome class", law greppable stream, "the run sent SIGINT, workers as threads (abort): stderr has 0 done lines". The new stop timing abort has a test of file a send its process SIGABRT, in worker threads.
- GREEN at 91e5367: "every outcome class" passes.
- Mutant: the CLI's old exit-128+sig path, greppable stream, "stderr has 0 done lines".
Calibrations:
- The e2e oracle has seven new ones:
  - a streamed line with no prefix
  - the decisive line after a mutant's line
  - a completed run that ends with the old completion line
  - a stopped run with no done line
  - a raw line on stdout
  - a run with two done lines
  - a refusal through the daemon without the cold run's notes, which is bound to the history "a declaration fault: configThrows"
- The static oracle has one: a refusal without its done line.
Model fixes during the review work:
- The check owed a test-output note in every completed run. In setupThrows, the setup file throws before any test runs, so nothing is captured, and the law owes no note. The check now owes it exactly when file a's baseline loads.
- The classify calibration "the stdout's baseline comes after its gaps" split the stdout at a blank line, which the stdout no longer has. It now moves the lines before the summary line.
Limits:
- A SIGKILL of the CLI, or a signal sent to the run's own child process rather than to the CLI, ends the run with no done line. The README says so.
- An internal error (exit 3) ends with oracle-gaps: done error, log <path>. No legal history reaches one, and its own message lines (oracle-gaps internal error: …, and a Node message's extra lines) have no prefix (F6).
- A syntax error in a module that the config imports, not in the config itself, still lets esbuild print its own block. No history generates one.
- On Windows, where the run is not supervised, a signal stop prints no done line.
- A signal that comes while Node still starts the CLI, before its first line runs, ends it with no done line (N3).
- The stop during warm-up is judged through the ORACLE_GAPS_TEST_WARM_PROBE hook, which models the esbuild service and Vitest's handler. The real race was seen once in 40 tries.
- Through the daemon, the lines arrive when the check ends.
- The session directory <package>/.oracle-gaps-<pid> stays in the package, as fix(oracle-gaps): keep the daemon's state, bundles, and default output in the OS temp directory #43 decided: its wrapper configs import the package's config and must resolve the package's modules.

Code weight

The production changes are modest:

a streamLine(kind, text) helper and a verdictLine in stream.mjs
the prefixed log calls, Vite's logger, the config's syntax check, and the wrapper's default read in check.mjs
the stdout summary as stream lines in report.mjs
the done line in ending.mjs, and the supervisor's one-done-line rule in index.mjs
the console redirect in capture-setup.mjs

The usage text, the README, and the pending changesets name the new lines.

This pull request and its description were written by Isaac.

Every line the CLI writes for a check starts with `oracle-gaps: ` and its kind, on stdout and stderr, cold and through the daemon; one decisive baseline line comes before any mutant; and a done line ends every run (completed, refused, or stopped). The e2e, classify, and static oracles judge it, with four new e2e calibrations and one static one. RED on main (81a5451): classify `stdout`, "the stdout's line after the baseline is "", not the decisive ..."; static `report-shaped ending`, "the refusal's last line is not `oracle-gaps: done refused`"; e2e `config`, "the run named its config as [], owed ["oracle-gaps: config vitest.config.ts"]". Co-authored-by: Isaac <no-reply@databricks.com>

…alibration The calibration "the stdout summary omits a gap" read a gap line by its old indented form, so on the prefixed stdout it found none and could not run. It now reads `oracle-gaps: gap …`, as law 8 does. Co-authored-by: Isaac <no-reply@databricks.com>

Agents act on a check's results as they stream, so every line the CLI writes for a check, on stdout and stderr, cold and through the daemon, now starts with `oracle-gaps: ` and its kind: config, note, baseline, blocked, progress, gap, busy, or done. One decisive baseline line comes before any mutant runs (`baseline all <n> oracle files pass`, or `baseline <k> of <n> oracle files failed (provisional)`), and every run ends with a done line: `completed, <n> gaps[ (provisional)], report <path>`, `refused[, report <path>]`, `error, log <path>`, or `stopped (<signal>)`. The done line replaces the stdout's `Full report:`, `output:`, and `completed:` lines and the refusal's `output:` line. Co-authored-by: Isaac <no-reply@databricks.com>

Co-authored-by: Isaac <no-reply@databricks.com>

runClisAtOnce ignored stdout, so law 15 read no done line for the two daemon checks that start at once and failed "a second daemon check of the budget witness" and "an edit sequence through the daemon" with "ends with undefined, not its done line". It now reads both streams, as runCli does. Co-authored-by: Isaac <no-reply@databricks.com>

…m's prefix A config that failed to load printed Vite's "failed to load config from …" on stderr without the stream's prefix, and a config with no default export also printed esbuild's bundling error for the wrapper's default import. Vite's logger now writes `oracle-gaps: note vite: …`, and the wrapper reads the config's default export at run time, so a config without one fails when it loads, as under Vitest, with no esbuild output. Law 15 found both (configThrows, configNoDefault). Co-authored-by: Isaac <no-reply@databricks.com>

coderabbitai · 2026-11-12T19:17:17Z

Important

Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

Source mode: Auto
Repository: Unavailable
Branch: Unavailable
File: No configuration file from this repository applied
Commit: Unavailable
Layers: defaults
Review profile: CHILL
Plan: Advanced
Run ID: adcd60f1-7ac2-4e61-85fa-513c1ae89d03

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

changeset-bot · 2026-11-12T19:17:17Z

🦋 Changeset detected

Latest commit: 606e525

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@tanstack/oracles	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

…nfig's forms The review of #46 (F1 to F5, F8) found what law 15 did not judge: - stdout's lines: the summary body and a test's console output were not lines of the stream. streamOf now judges stdout as it judges stderr, and file a has a test that writes to the process's stdout and stderr; a completed run names where that output went in one note; - every ending: the done line ends both streams, once each, of the kind the exit status says (0 completed, 2 refused, 3 error, else stopped); - a signal after the completed line (stopRuns `after`, with the CLI's exit held 500 ms): the run ends as it completed, with one done line; - a config refused through the daemon prints the cold run's notes; - the config's forms: a promise of a config (Vite awaits it), a null default, and a syntax error, each judged as Vitest judges it. The classify oracle's law 8 now reads the stdout summary as stream lines (summary, legend, function, note) and rejects any other line. New e2e calibrations: a raw line on stdout, two done lines, and a daemon refusal without the cold run's notes. Co-authored-by: Isaac <no-reply@databricks.com>

Co-authored-by: Isaac <no-reply@databricks.com>

…onfig semantics The review of #46 (F1 to F5, F7 to F11), against law 15's extension: - F1: the stdout summary is lines of the stream: `summary` for the totals, `legend` for each directive, `function` for each heading, and `note` for the provisional count, unowned counts, files left out, and the results' path. No line is blank; summary.md holds the report to read. - F2: a test's own writes to the process's stdout and stderr go to test-output.log in the output directory, and one note names it. - F3: once the run writes its ending, a stop signal no longer cuts it off, and the supervisor writes the stopped line only when the run did not end with its own code (0, 2, or 3): one done line per stream. - F4: Vite's notes go through the check's log, which the daemon relays, so a refusal through the daemon prints the cold run's lines. - F5: the wrapper awaits the config's default, a promise included, and refuses one that is not an object with Vite's wording. - F7: a config that does not parse is refused with esbuild's first error, parsed silently before Vite bundles it. - F8: the done line ends stdout and stderr for every ending. - F9 to F11: the README says which endings print no done line, fixes the heartbeat line, and says that --help is not a check. Co-authored-by: Isaac <no-reply@databricks.com>

The calibration "the stdout's baseline comes after its gaps" split the stdout at its first blank line, which the stdout no longer has. It now moves every line before the `oracle-gaps: summary` line after the rest, the same wrong answer in the stdout's new form. Co-authored-by: Isaac <no-reply@databricks.com>

The full fixed seed failed "a declaration fault: setupThrows" with "the run names its tests' console output in 0 notes". There the setup file throws before any test runs, so file a's test that writes to the process's streams never runs, nothing is captured, and the law (a note names the log when output was captured) owes no note. The check owed one in every completed run; it now owes one exactly when file a's baseline loads, as the model's baseline says. Co-authored-by: Isaac <no-reply@databricks.com>

… 15's The calibration "a refusal through the daemon drops the cold run's notes" searched candidates for 433 s; it is now bound to the constructed history "a declaration fault: configThrows" (#31's history binding). The other new calibrations, measured at ab18c60 (58 to 79 s each), are recorded in e2e-shards.mjs, two of them moved to e2e-5 for balance. Co-authored-by: Isaac <no-reply@databricks.com>

Co-authored-by: Isaac <no-reply@databricks.com>

pkg-pr-new · 2026-11-12T21:35:16Z

Open in StackBlitz

npm i https://pkg.pr.new/TanStack/oracles/@tanstack/eslint-plugin-oracles@46

npm i https://pkg.pr.new/TanStack/oracles/@tanstack/oracles@46

commit: 606e525

…es a stop during warm-up The re-review of #46: - N2: 2d09b99 owed the test-output note whenever file a's baseline loads, a rule the law ("a note names the log when output was captured") does not state. The driver now observes whether the run captured output, the note is owed exactly then, and a run whose file a test wrote must have captured it. A new dimension, consoleOutput, gives file a no writing test; "a load failure beside a hang" pins it off and enters "a completed run whose tests wrote nothing". A calibration names a log never captured. - N1: stop runs now have a `when`: while the witness spins, after the completed line, or while the config warms (`warm`, through the CLI's ORACLE_GAPS_WARM_PROBE hook). A stop during warm-up ends with one stopped line per stream, and is never reported as a refusal. Co-authored-by: Isaac <no-reply@databricks.com>

ORACLE_GAPS_WARM_PROBE names a marker: the first context's warm-up writes it and then waits, as a config load can, on a service in the run's process group that a stop signal ends (as it ends esbuild's), while a handler that exits 200 ms after the signal holds the process (as Vitest's does). Unset, it does nothing. It lets the oracle stop a run during warm-up on purpose. Co-authored-by: Isaac <no-reply@databricks.com>

…ists Co-authored-by: Isaac <no-reply@databricks.com>

…one line The re-review of #46: - N1: a SIGINT while the config warmed cut off the config's load; the run reported the error as a refusal, wrote `done refused`, and then died by the signal, so the supervisor wrote `done stopped` as well. The run now installs its own stop handler first: a stop ends it at once with no line, unless it is writing its ending. It tells the supervisor, over an IPC channel, that it writes its ending; the supervisor writes the stopped line only when no such message came, and decides at the run's `close`, after its messages. - N3: the supervisor installs its stop handlers before it starts the run. A signal that comes while Node still starts the CLI, before its first line runs, still ends it with no done line; the README says so. Co-authored-by: Isaac <no-reply@databricks.com>

The third review of #46: ORACLE_GAPS_WARM_PROBE is now ORACLE_GAPS_TEST_WARM_PROBE, and ORACLE_GAPS_HOLD_EXIT_MS, the other hook the CLI's oracle sets, ORACLE_GAPS_TEST_HOLD_EXIT_MS. Their comments say that they are test-only, and so does the README. Co-authored-by: Isaac <no-reply@databricks.com>

The coordinator's decision of 2026-11-12 on P10: when the run's process dies on its own, with no ending of its own and no stop from the CLI (a V8 out-of-memory abort is a SIGABRT), the CLI still writes the done line, on both streams: `oracle-gaps: done error, the run's process exited with <signal or code>`, and exits 3. Stop runs gain `when` `abort`: a test of file a aborts the process (`GAP_ABORT`, in worker threads), and the run is owed exactly that line. "every outcome class" enters the new state "a run whose process aborted". Co-authored-by: Isaac <no-reply@databricks.com>

process.abort() is unavailable in worker threads, so the test threw and the run completed; process.kill(process.pid, 'SIGABRT') ends the run's process, as a V8 out-of-memory abort does. Co-authored-by: Isaac <no-reply@databricks.com>

The coordinator's decision of 2026-11-12 on P10: when the run's process ends without its ending and without a stop from the CLI (a V8 out-of-memory abort is a SIGABRT), the CLI exited 128 plus the signal with no done line. It now writes `oracle-gaps: done error, the run's process exited with <signal or code>` on both streams and exits 3, with the remedy when the run's stderr, which the CLI now passes on, said that the heap ran out (NODE_OPTIONS=--max-old-space-size, or fewer --contexts). It kills what is left of the group at the run's exit and waits at most 2 s for the run's stderr to close. Co-authored-by: Isaac <no-reply@databricks.com>

The calibration "a refused run is reported as complete" set exit 0 and left the refused done line, so law 15 (the done line's kind matches the exit) rejected it before the law it targets, ownership. It now also gives both streams the completed done line, as a run that says it completed would print, so only the refusal law rejects it. Its owed law and law 15 are unchanged. Co-authored-by: Isaac <no-reply@databricks.com>

Keeps #46's line protocol and law 15 with the work law: the extra-work gap kind in the streamed-gap pattern, the extra-work legend line under the new oracle-gaps: legend prefix, and the README's work law before the protocol section. Co-authored-by: Isaac <no-reply@databricks.com>

Since #46 the stream has no blank line, so the observed stdout lead is the whole stdout, and its notes name the run's scratch directory (.scratch-<random>), which differs between two runs of one history. The determinism test of "every outcome class" failed at observation 0 ("cli") on main after #46. Each lead line now names the scratch directory as the rest of the trace does. RED (on the merge of main): "two runs of constructed history: every outcome class recorded different traces at observation 0 (cli)", the first difference in the results note's .scratch-<id> path. GREEN after. Co-authored-by: Isaac <no-reply@databricks.com>

…tructed failing owner, and a copy's random stream (#54) * test(oracle-gaps): record a stopped run's session directories by identity, not pid The stop law reads only whether a later run kept an earlier run's session directory, but the driver observed the directories by name, `.oracle-gaps-<pid>`, which differs on every run, so the determinism test of "every outcome class" failed. Each directory is now the nth distinct one of its history (`session#n`). RED: determinism: constructed history: every outcome class, "two runs recorded different traces at observation 1 (stopped)". GREEN after. Mutant (the CLI keeps stale session directories): "the run after a stopped one left its session directory session#0 in the package". Co-authored-by: Isaac <no-reply@databricks.com> * test(oracle-gaps): construct the failing owner's ownership, not borrow it from the base The constructed history "a failing owner under a config that --config names" owes the state "blocked by brokenOwner", which the law owes only when the failing owner, file b, owns a mutant that no passing owner kills (law 14; maintainer disposition G5, 2026-11-11). The construction owned only file a's function and relied on its generated base for something b owns, so a base with nothing b owns missed the state. It now adds a second function that only b owns. RED: ORACLE_SEED=520648412 ORACLE_PATH=0 replay of "random seed: a failing owner under a config that --config names": "did not enter blocked by brokenOwner". GREEN: the same replay and the fixed-seed unit pass. Co-authored-by: Isaac <no-reply@databricks.com> * test(oracle-gaps): a mutant's copy draws from Math.random what its original drew The capture seeds the ambient random source per test, so a test's draws repeat in every run. The witness w_draws observes a Math.random draw beside a cache write that nothing observes: its if-false mutant changes only the cache, so it is owed not-observed. RED: "src/sut.ts:w_draws:74:if-false#0: not-judged, owed not-observed". A mutant runs in a copy of its test file, named for its batch (.oracle-gaps-<pid>-<n>-<file>), and the capture seeded the source by the copy's path, while the unmutated runs run the original file: every copy draws another stream than the baseline. The klona run without --seed showed the same thing through oracleProperty's drawn campaign seed, so a random unit's histories changed under each mutant. Co-authored-by: Isaac <no-reply@databricks.com> * fix(oracle-gaps): seed a copy's ambient randomness by its original test The capture seeds Math.random, crypto.getRandomValues, and crypto.randomUUID per test, so a test sees the same values in every run. It seeded them by the test's key, whose file is the copy's path in a mutant batch (.oracle-gaps-<pid>-<n>-<file>), while the unmutated runs run the original file: every mutant drew another stream. A test that observes a draw then reads as changed under a mutant that changes nothing (not-judged where not-observed is owed), and an oracleProperty campaign with no --seed draws its seed from Math.random, so its random units ran other histories under every mutant (the klona run without --seed: 10 false not-judged gaps). The batch now names each copy's original file, and the copy seeds by the key in that file. GREEN: e2e "every outcome class" passes, w_draws not-observed. Co-authored-by: Isaac <no-reply@databricks.com> * docs(changeset): a mutant's copy draws what its original draws Co-authored-by: Isaac <no-reply@databricks.com> * test(oracle-gaps): name the scratch directory in the stdout lead Since #46 the stream has no blank line, so the observed stdout lead is the whole stdout, and its notes name the run's scratch directory (.scratch-<random>), which differs between two runs of one history. The determinism test of "every outcome class" failed at observation 0 ("cli") on main after #46. Each lead line now names the scratch directory as the rest of the trace does. RED (on the merge of main): "two runs of constructed history: every outcome class recorded different traces at observation 0 (cli)", the first difference in the results note's .scratch-<id> path. GREEN after. Co-authored-by: Isaac <no-reply@databricks.com> * test(oracle-gaps): observe a stop's session directories as counts, not ordered names Review of #54: session#n numbered the directories in readdir order, which sorts by pid, so with two directories present the numbering depended on the pids (a replica: the kept one was not session#0 in 518 of 1000 trials). The observation now holds counts, which no name or order changes: the session directories present once the run ended, and how many of them the previous stopped run had left. The law is as strict: any kept directory is a violation. GREEN: "fixed seed 20261105: every outcome class" and its determinism unit. Mutant (the CLI keeps stale session directories): "the run after a stopped one left 1 of its session directories in the package". The stdout lead holds no timing line: the progress lines with seconds (unmutated, mutant phase) go to stderr through log, and the stdout summary (report.mjs) has none. Co-authored-by: Isaac <no-reply@databricks.com> --------- Co-authored-by: Isaac <no-reply@databricks.com>

…its one done line (#46) Co-authored-by: Isaac <no-reply@databricks.com>

KyleAMathews and others added 6 commits November 12, 2026 10:13

Merge origin/main (#44) into feat/gap-cli-greppable-lines

57a74bc

Co-authored-by: Isaac <no-reply@databricks.com>

KyleAMathews and others added 4 commits November 12, 2026 12:40

Merge origin/main (#47) into feat/gap-cli-greppable-lines

07f842a

Co-authored-by: Isaac <no-reply@databricks.com>

KyleAMathews mentioned this pull request Nov 12, 2026

Self-oracle counterexample: calibration selector finds no history when history names repeat (seed 1187311827) #50

Open

KyleAMathews and others added 4 commits November 12, 2026 14:14

docs(changeset): the greppable stream's review fixes

0aa1553

Co-authored-by: Isaac <no-reply@databricks.com>

ci(oracles): record the bound daemon-notes calibration's cost

9e93944

Co-authored-by: Isaac <no-reply@databricks.com>

KyleAMathews and others added 9 commits November 12, 2026 14:39

test(oracle-gaps): add the wrote-nothing state after the reach set ex…

76ce8f9

…ists Co-authored-by: Isaac <no-reply@databricks.com>

KyleAMathews mentioned this pull request Nov 13, 2026

Self-oracle counterexample: timeout presentation not exercised (seed 569029462) #51

Open

KyleAMathews merged commit 6858b9d into main Nov 13, 2026
57 of 58 checks passed

KyleAMathews deleted the feat/gap-cli-greppable-lines branch November 13, 2026 03:06

KyleAMathews mentioned this pull request Nov 13, 2026

fix(oracle-gaps): a file a test reads through fs is an input of its daemon run #55

Open

KyleAMathews mentioned this pull request Nov 13, 2026

fix(oracle-gaps): deterministic stop sessions and stdout lead, a constructed failing owner, and a copy's random stream #54

Merged

KyleAMathews mentioned this pull request Nov 13, 2026

feat(oracle-gaps): write the stream to <out>/stream.log and name it in the first line #57

Open

KyleAMathews added a commit that referenced this pull request Nov 13, 2026

docs(oracles): read the gap check's oracle-gaps: prefixed stream and …

f98b7cd

…its one done line (#46) Co-authored-by: Isaac <no-reply@databricks.com>

KyleAMathews mentioned this pull request Nov 13, 2026

docs(oracles): ship setup, design, and review skills, with a prompt to try them #7

Open

Navigation Menu

Repository navigation

feat(oracle-gaps): every streamed line starts with `oracle-gaps: <kind>` - #46

feat(oracle-gaps): every streamed line starts with `oracle-gaps: <kind>`#46
KyleAMathews merged 23 commits into
mainfrom
feat/gap-cli-greppable-lines

KyleAMathews commented Nov 12, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Nov 12, 2026 •

edited

Loading

Uh oh!

changeset-bot Bot commented Nov 12, 2026 •

edited

Loading

Uh oh!

pkg-pr-new Bot commented Nov 12, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

KyleAMathews commented Nov 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Fixes from the review (F1 to F11)

Fixes from the second review (N1 to N3)

P10: a run whose process dies on its own

Evidence

Code weight

Uh oh!

coderabbitai Bot commented Nov 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

changeset-bot Bot commented Nov 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

pkg-pr-new Bot commented Nov 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

KyleAMathews commented Nov 12, 2026 •

edited

Loading

coderabbitai Bot commented Nov 12, 2026 •

edited

Loading

changeset-bot Bot commented Nov 12, 2026 •

edited

Loading

pkg-pr-new Bot commented Nov 12, 2026 •

edited

Loading