Repository navigation
fix(oracles): the self-oracle timeout law applies only to slow specs that load - #63
Draft
KyleAMathews wants to merge 2 commits into
Draft
KyleAMathews wants to merge 2 commits into
KyleAMathews wants to merge 2 commits into
Conversation
…ation The shrunk counterexample of #59: a slow spec whose owns list is empty fails to load, runs no driver, and the timeout law reports a missing timeout presentation. This fixed history reproduces it; the law is fixed in the next commit. Co-authored-by: Isaac <no-reply@databricks.com>
A slow spec that fails to load never runs its driver, so it cannot owe a timed-out call to present. Condition the timeout obligation on the model's load expectation, which already predicts the load failure. Fixes #59 Co-authored-by: Isaac <no-reply@databricks.com>
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #59
Decisions
expected, the model's ownloadsprediction. It adds no production import, mutant, or probe to the model.loadsexpectation). This is a law-scope decision, and it has not been ratified. It was pushed before the register was read. Hold merge until it is recorded or overruled.Law
The self-oracle's timeout law (
checkSuite,packages/oracles/tests/oracle-property-self-oracle.test.ts) says that when a plan has a slow spec and timeout presentation, the run child must exercise that presentation. A slow spec whose declaration fails to load (for exampleowns: []) never runs its driver, so it owes no timed-out call. The authority is the model's own load prediction inpredictSpec(theloads: falsereturns forownsshape, among others), and the registration law insrc/oracle-property.ts. This change only conditions the timeout obligation on that prediction. It does not change the law's text or the model.Reach
timeout presentation of a slow spec that fails to load, under timeout presentation withtiming: slowandownsMode: empty. It entersregistration-error.-1999165303reproduces it deterministically.timeout continuation presentation,timeout every receiving cut, and the timeout input units). Those still require the exercised presentation.unregistered, invalidwork.ratio, predict mismatches, and the otherloads: falsereturns) reach the same branch by the same mechanism. OnlyownsMode: emptyis pinned here. Their coverage is in the full suite run.RED
origin/main(affeb1a), Vitest 4.1.12, 2 workers, with the CI config:ORACLE_SEED=-1999165303 ORACLE_PATH=0:0:…:0 ORACLE_PROPERTY='self-oracle.framework' ORACLE_UNIT='random seed: generated' vitest run -t '^self-oracle\.framework(?: | > )random seed: generated$'. 6 of 6 runs fail withtimeout presentation was not exercised. On Vitest 5.1.0, 5 of 5 fail.-t 'timeout presentation of a slow spec that fails to load'fails withtimeout presentation was not exercised.GREEN
vitest.oracle-framework.config.mjssuite, Vitest 4.1.12, 2 workers, with the guard but before the regression entry was added: 851 of 851 pass in 37 minutes. The local Vitest 5.1.0 full run was stopped, per the maintainer's rule. CI runs both Vitest 4 and Vitest 5 framework shards on this PR.Mutant
timeout presentation was not exercised, and the pinned seed fails the same way (the earlier RED).if (false && …)on thetimeout-presentationobservation). On the loaded draft unittimeout continuation presentation, the oracle rejects it: 3timeout presentation was not exercisedfailures across 4 failed tests. The guard does not hide a real timeout bug.Limits
ownsMode: emptyload failure is pinned. The other load-failing causes are argued from the same mechanism and rely on the full suite in CI.src/) code is changed. The fix is only in the oracle's precondition.This pull request and its description were written by Isaac.