Skip to content

Fix quadratic processing time for consecutive abbreviation definitions - #1702

Open
deadstrobe5 wants to merge 3 commits into
Python-Markdown:masterfrom
deadstrobe5:perf/abbr-consecutive
Open

deadstrobe5 wants to merge 3 commits into
Python-Markdown:masterfrom
deadstrobe5:perf/abbr-consecutive

Conversation

@deadstrobe5

@deadstrobe5 deadstrobe5 commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Description

AbbrBlockprocessor.run records one definition per call and puts the rest of the block back on the stack, so a block of N definitions is re-tested by every block processor N times and the remainder is copied N times. A glossary of abbreviations appended to every page (the setup Material for MkDocs documents with pymdownx.snippets auto_append) hits this on every page.

This records the following definitions in the same pass, the same way consecutive reference definitions were handled in #1678. A later definition is only taken when the block starts with a definition, the later one fits on one line, and it is followed by another definition or the end of the block. In any other case a processor with higher priority could claim the remainder (a Setext underline, a table separator or a definition list line after it), so that definition is left to the next pass and the output does not change. A test covers the Setext case.

Median of 5 alternating runs in one process, CPython 3.14, markdown.Markdown(extensions=[...]).convert():

Document Before After
2 KB page + 50 definitions (9 extensions) 5.41 ms 2.18 ms
2 KB page + 200 definitions (9 extensions) 79.5 ms 3.92 ms
2 KB page + 200 definitions (abbr only) 30.3 ms 2.18 ms
10,000 definitions (abbr only, one run) 178 s 0.32 s

Pages without definitions are unchanged. I compared the HTML and the recorded abbreviations of the old and new code on 50,000 generated documents (definitions inside lists, block quotes and indented blocks, empty and '' titles, multi-line definitions, mixed with tables, headers and definition lists) and found no differences.

AI Assistance Disclosure

  • No AI tools were used in preparing this PR.

  • AI tools were used and I have disclosed which ones, and have fully reviewed and verified their output.

    Claude Code (Claude Opus 5.5) was used for everything ticked below. The review attestation in the box above is left for the human submitter.

    AI tools were use to assist with the following:

    • Code
    • Code Comments
    • Documentation
    • Changelog Entry
    • Commit Message
    • The Description Above
    • Other: ______________

Checklist

),
self.dedent(
"""
<h1>*[HTML]: Hyper Text Markup Language</h1>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not the correct behavior. The correct output should be:

<p>===</p>

@waylan waylan added the requires-changes Awaiting updates after a review. label Nov 10, 2026
@deadstrobe5

Copy link
Copy Markdown
Contributor Author

Agreed, fixed in 796d415 (plus c66f873 for line length). The processor now takes every consecutive definition, whatever follows, so that input gives <p>===</p>. I dropped the rule that left the last definition for another processor, which was there only to keep the old output.

Current master is inconsistent here: two definitions before === already give <p>===</p>, but with three or more, master turns the last definition into an <h1>. I added a test for the three-definition case (it fails on master and passes here). Against master, the output differs only in this pattern: 19 of 4,168 documents (the test fixtures, the docs, and random mixes of definitions, headers, lists and text), all of them definitions followed by ===. 200 definitions still go from 15.5 ms to 0.63 ms. Also merged master to fix the changelog conflict.

if m is None or '\n' in m.group():
break
matches.append(m)
for m in matches:

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that there is no code between these 2 loops, they should be able to be combined into a single loop.

Although, why not just call finditer up on line 160 and then loop through the results?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

requires-changes Awaiting updates after a review.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants