Skip to content

fix: apply precision before round for decimal units - #404

Open
rahul05ranjan wants to merge 1 commit into
avoidwork:masterfrom
rahul05ranjan:fix/precision-before-round
Open

rahul05ranjan wants to merge 1 commit into
avoidwork:masterfrom
rahul05ranjan:fix/precision-before-round

Conversation

@rahul05ranjan

Copy link
Copy Markdown

Fixes #399.

The README documents that precision "takes precedence over round", but applyRounding() ran unconditionally before applyPrecisionHandling(), so:

  • digits beyond round were lost before toPrecision() could keep them — filesize(2887, { precision: 4 }) printed 2.890 kB instead of 2.887 kB;
  • boundary values were rounded twice — filesize(999499, { precision: 3 }): 999.499 pre-rounded to 999.5, toPrecision(3) then produced 1.00e+3, and the unit bumped to 1.00 MB instead of staying 999 kB;
  • round leaked into the result even with precision set — filesize(1245, { precision: 2, round: 0 }) printed 1.0 kB instead of 1.2 kB.

The change: for decimal units (e > 0) with precision > 0, the un-rounded scaled value goes straight to the precision pass. Whole bytes (e === 0) still round to integers — a byte count is an integer quantity, and the sub-1-byte outputs (0 B, 0.0 B, 1 B) are unchanged.

Verification (node --test tests/unit/, no install needed — the tests import src/ directly):

  • New regression test pins all four cases from the issue plus the two whole-byte controls. It fails on unpatched master@1a8566c (the 999 kB / 2.887 kB / 1.2 kB assertions) and passes with the fix.
  • Full suite: 274 pass / 0 fail with the fix — every pre-existing precision pin (locale, sci-notation removal, standards, bits, array/object/exponent outputs, fullform) is unchanged.

The README documents that precision overrides round, but round ran
first: 999.499 kB became 999.5, toPrecision(3) produced "1.00e+3"
and the unit bumped to MB; digits beyond round were lost before
toPrecision() could keep them (2.887 -> 2.89 -> "2.890"). Decimal
units now feed the un-rounded scaled value to the precision pass;
whole bytes still round to integers, so the sub-1-byte outputs are
unchanged.

Fixes avoidwork#399

Co-authored-by: rahul05ranjan <11793455+rahul05ranjan@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: precision is applied after round, rounding an already-rounded value

1 participant