test: property-based fuzz testing for js->zig->js roundtrip by nazarhussain · Pull Request #41 · ChainSafe/zapi

nazarhussain · 2026-05-29T10:24:50Z

Adds a property-based fuzz harness for zapi Number, BigInt and Uint8Array converters (vitest + fast-check, driven through a dedicated test_fuzz_numeric.node addon). The harness surfaced several latent bugs and an API sharp edge.

Stacked on #83. The library fixes and API changes the harness found (exact-by-default toI64/toU64/toI128, the *LowBits variants, BigInt.fromWords, the getValueBigintWords sign fix and the Number doc comments) now live in #83. This PR contains only the fuzz tests, the build wiring and CI. Merge #83 first.

Changes

tests/fuzz/: round-trip fuzz targets and fast-check properties for Number, BigInt and Uint8Array, with a seeds workflow (see tests/fuzz/README.md).
tests/unit/conversions.test.ts: fixed edge-value checks of the same targets against their oracles, run by test:js.
tests/build.zig.zon / tests/build.zig: a separate Zig package for the fuzz addon, mirroring examples/ and consuming zapi via --fork=... It installs to the root zig-out/.
package.json: build:zig now runs build:zig:examples and build:zig:tests; test:js also runs tests/unit/; adds test:fuzz:round_trip and the fast-check dev dependency.
Dockerfile.fuzz, tests/fuzz/platform/run-fuzz.ts and .github/workflows/fuzz-image.yml: the fuzz image for the ChainSafe fuzz platform. Pull requests build and smoke-test it, which covers the fuzz properties in CI; publishing to GHCR is a manual run.

🤖 Generated with Claude Code

wemeetagain · 2026-07-24T14:57:43Z

      - name: Test
        run: pnpm test:js

+  test-fuzz:


seems a little strange to fuzz in CI

wemeetagain

I think the API changes and additional comments are good

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Round-trips JS BigInt → i128 → JS BigInt via toI128 and fromWords. Property test covers the i128 boundary plus out-of-range truncation via BigInt.asIntN(128, ·). Oracle test adds deterministic boundary cases for zero, I128_MIN/MAX (exact), just-out-of-range ±1, and oversized values whose low 128 bits are all zero (expected result: 0n).

The seeds-workflow doc claimed NaN/Infinity could be persisted via a {"__special": ...} encoding, but no corresponding reviver exists in fuzz.test.ts — following the README literally would silently corrupt test input. NaN/Infinity are already in edgeNumbers and exercised every run, so persisting them adds nothing. Trim the README to match reality.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…90) Packages the fast-check fuzz targets from #41 as a Docker image for the [ChainSafe fuzz platform](https://github.com/ChainSafe/lodestar-ansible-development/tree/main/playbooks/fuzz), so they run continuously as one campaign (`zapi:round-trip`). > **Stacked on #41** (which is stacked on #83). Merge those first. ### Changes - `Dockerfile.fuzz` (+ `.dockerignore`): a build stage that downloads Zig 0.16.0 (checksum-verified) and builds `test_fuzz_numeric.node` with `ReleaseSafe` and `-Dcpu=baseline`. The runtime stage is `node:24-slim` with the addon, the tests and `node_modules`. - `tests/fuzz/platform/run-fuzz.ts`: the entrypoint, which implements the platform's container contract. It re-runs every `*.fuzz.test.ts` in a loop (100 000 cases per property by default, a fresh random seed each time) until SIGTERM. - A failing property is saved once to `$FUZZ_FAILURES/failure-<hash>.json`, de-duplicated by test name and shrunk counterexample. The file includes the seed and path for replay. Fuzzing then continues. - `$FUZZ_METRICS/status` is rewritten atomically after each iteration. It reports `execs_total`, `execs_per_second`, `crashes_total` and `last_find_timestamp_seconds`. There is no coverage or corpus field, because fast-check is not coverage-guided. - Counters persist in `$FUZZ_STATE/stats.json`. - A test file that fails to load, or a run with no tests, exits non-zero (an infrastructure failure). - `.github/workflows/fuzz-image.yml`: mirrors lodestar-z's `fuzz-image.yml`. A `build and test` job runs with read-only permissions and smoke-tests the image under the platform's restrictions (non-root uid, read-only rootfs, no capabilities, no network, `--init`). Publishing is **manual** (`workflow_dispatch`): a separate `publish` job, the only one with `packages: write`, pushes the exact tested image as `main` (run on `main`) or `branch-<name>` (any other branch), plus `sha-<short>`, and keeps the newest 20 versions. lodestar-z's AFL++-only steps (seed replay, `core_pattern`, minimize) are omitted; the platform reads none of them. - `tests/fuzz/README.md`: a section on continuous fuzzing. ### Verified locally (arm64 Docker, platform restrictions) - Runs continuously and stops on SIGTERM with exit 0. - The status file is written and updated. - Counters resume across restarts. - A planted failing property is saved once over about 20 failing iterations. - A hidden addon gives `FATAL a test suite failed to load` and exit 1. The amd64 build and the workflow itself have not run yet; this PR's CI is their first run. ### Follow-ups - After merging, publish by running the workflow on `main` from the Actions tab. Then set the GHCR package `zapi-fuzz` to **public**. The platform pulls without credentials. - Add the target to `group_vars/fuzzer/vars.yml` in lodestar-ansible-development: ```yaml - project: zapi target: round-trip image: ghcr.io/chainsafe/zapi-fuzz:main cpu_mhz: 1000 memory_mb: 1024 duration: 0 uid: 60011 gid: 60011 ``` 🤖 Generated with [Claude Code](https://claude.com/claude-code)

The fuzz-image workflow already runs the fuzz properties on pull requests, so the separate test-fuzz CI job is redundant. The fixed edge-value checks move to tests/unit and run with test:js. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

nazarhussain requested review from spiral-ladder and wemeetagain July 16, 2026 09:03

wemeetagain reviewed Jul 24, 2026

View reviewed changes

matthewkeil added this to Lodestar Team Coordination Jul 27, 2026

matthewkeil moved this to In Review in Lodestar Team Coordination Jul 27, 2026

matthewkeil assigned nazarhussain Jul 27, 2026

nazarhussain requested a review from wemeetagain August 12, 2026 09:06

matthewkeil moved this from Awaiting Author to Review Requested in Lodestar Team Coordination Sep 10, 2026

nazarhussain force-pushed the nh/fuzz-testing branch 2 times, most recently from 4b09223 to e3384ed Compare October 19, 2026 10:11

nazarhussain mentioned this pull request Oct 19, 2026

fix(js)!: make BigInt conversions exact and add a shared LossyConversion error #83

Draft

nazarhussain changed the base branch from main to nh/bigint-exact-conversions October 19, 2026 10:11

nazarhussain force-pushed the nh/fuzz-testing branch from e3384ed to 959d16d Compare October 19, 2026 10:20

nazarhussain and others added 16 commits October 19, 2026 12:31

chore(deps): add fast-check and test:fuzz script

a86ab02

chore(tests): scaffold fuzz_numeric addon module

0cc0e58

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

test(fuzz): add rtNumberF64 round-trip fuzz target

21af8ad

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

test(fuzz): add Number i32/u32/i64 round-trip properties

e578734

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

test(fuzz): add BigInt i64/u64 round-trip properties

2c7a8be

test(fuzz): add lossless-flag introspection for toI64/toU64

209b47b

test(fuzz): add BigInt word-level round-trip

57bcdde

ci: add fuzz tests job

72f6e4b

docs(fuzz): document seeds workflow and dev loop

4b67883

chore: restructure oracle code

5d0c2a3

test(fuzz): cover exact and LowBits BigInt conversions

594a4ad

test: add uint8array fuzz tests

a032b18

chore: restructure tests files

3689e3b

chore: rename fuzz tests script

3f2ee8c

test(fuzz): note getValueBigintWords overflow in rtBigIntWords

17080db

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

nazarhussain force-pushed the nh/bigint-exact-conversions branch from 0773d1c to 0dcc598 Compare October 19, 2026 10:31

nazarhussain force-pushed the nh/fuzz-testing branch from 959d16d to 17080db Compare October 19, 2026 10:31

nazarhussain mentioned this pull request Nov 5, 2026

ci(fuzz): run fast-check fuzz targets on the ChainSafe fuzz platform #90

Merged

nazarhussain and others added 2 commits November 6, 2026 10:50

nazarhussain mentioned this pull request Nov 6, 2026

feat(js): add exact BigInt conversions, toI128LowBits and fromWords #91

Open

Navigation Menu

Repository navigation

test: property-based fuzz testing for js->zig->js roundtrip - #41

test: property-based fuzz testing for js->zig->js roundtrip#41
nazarhussain wants to merge 19 commits into
nh/bigint-exact-conversionsfrom
nh/fuzz-testing

nazarhussain commented May 29, 2026 •

edited

Loading

Uh oh!

wemeetagain Jul 24, 2026

Uh oh!

wemeetagain left a comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

nazarhussain commented May 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Uh oh!

wemeetagain Jul 24, 2026

Choose a reason for hiding this comment

Uh oh!

wemeetagain left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

nazarhussain commented May 29, 2026 •

edited

Loading