Skip to content

👌 Speed up rendering on PyPy by joining fragments once - #458

Open
DicksonWu654 wants to merge 3 commits into
executablebooks:masterfrom
DicksonWu654:perf/pypy-renderer
Open

DicksonWu654 wants to merge 3 commits into
executablebooks:masterfrom
DicksonWu654:perf/pypy-renderer

Conversation

@DicksonWu654

Copy link
Copy Markdown

Summary

RendererHTML.render() and renderInline() build their output with result += fragment. CPython extends a local string in place, but PyPy does not, so each append copies the whole rendered prefix. For a large document this dominates rendering time on PyPy.

On PyPy only, both methods now collect fragments in a list and join once. CPython keeps exactly the current code: the implementation is chosen once at class creation, and the CPython branch is the existing method body, re-indented (its bytecode is unchanged on 3.12 and 3.14).

Behaviour kept for custom render rules:

  • If a rule returns something other than an exact str (a str subclass with its own __add__/__radd__, or an invalid object), the prefix is materialised at that point and rendering continues with +=. Overloaded addition, the returned type, rule call order and the first raised error stay as before.
  • The fragment list starts with "", so a render that produces a single fragment still returns a new object, as string addition does on PyPy. A custom reflected addition can observe that identity.

The second commit is that identity fix, so the two commits belong together.

Performance

Whole-document timings. Each pair runs master and this branch in fresh processes, pinned to one core, alternating AB/BA order. Times are warm: 25 warmup rounds on PyPy and 2 on CPython, then 10 or 6 timed rounds. Ratios are master time / branch time (median of per-pair ratios, with min-max over pairs). Documents: benchmarking/samples/spec.md (CommonMark spec), and the repository docs plus README and CHANGELOG concatenated.

Interpreter Workload Speedup Range Pairs faster
PyPy 3.11.13 render(spec.md), commonmark 2.92x 1.82-3.69 6/6
PyPy 3.11.13 render(docs), gfm-like 1.19x 1.03-1.47 6/6
PyPy 3.11.13 all three workloads (incl. parse(spec.md)) 1.77x 1.29-2.08 6/6
CPython 3.12.3 all three workloads 1.03x 0.85-1.21 8/14
CPython 3.14.8 all three workloads 1.02x 0.94-1.05 4/6

On CPython there is no change, as expected: an A/A run on the same machine (master against master) spreads just as widely, with medians between 0.93x and 1.01x.

Validation

  • Test suite passes (1053 tests, 21 of them new) on CPython 3.10, 3.11, 3.12, 3.13 and 3.14, and on PyPy 3.10 and 3.11.
  • The new tests/test_api/test_renderer_results.py covers custom rules returning str subclasses with __add__/__radd__, non-string results and the error they raise, rule call order and prefix identity. These tests also pass on master (CPython and PyPy), so they pin the existing behaviour.
  • Differential against master on CPython 3.12, CPython 3.14 and PyPy 3.11. It compares rendered HTML, Token.as_dict() streams and env keys, over the CommonMark spec examples, every test fixture (whole files and individual cases), the repository docs, and Markdown files and package metadata from an installed environment, for 3,528 documents in all. It runs them under the commonmark, default, zero, gfm-like, gfm-like2 and js-default presets, gfm-like with linkify and typographer, and commonmark with mdit-py-plugins (front matter, footnote, deflist, tasklists, dollarmath, attrs, container). All 28,224 records are identical.
  • pre-commit run --all-files passes (ruff, ruff-format, mypy).

🤖 Generated with Claude Code

DicksonWu654 and others added 3 commits November 6, 2026 17:00
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Retain the initial empty operand so buffered rendering preserves string identity visible to custom reflected addition and single-fragment callers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant