[Steve-20-ai]

Description

markdown.markdown() raises an uncaught RecursionError on a deeply nested
list, so any service rendering untrusted Markdown can be crashed by a small
input. 500 list markers on one line is about 1 KB:

>>> import markdown
>>> markdown.markdown('- ' * 500 + 'x')
RecursionError: maximum recursion depth exceeded

The same happens for an indented list of about 500 lines, and for a list inside
a blockquote. It is the class of problem #1534 covered, with a different
trigger.

A list nests one level per parseBlocks call, and the list processors had no
limit on that. BlockQuoteProcessor has had one since #799, which is why
>>>>>>... renders instead of raising:

def test(self, parent, block):
    return bool(self.RE.search(block)) and not util.nearing_recursion_limit()

This applies the same guard to the list processors. ListIndentProcessor.test()
and OListProcessor.test() now return False once nearing_recursion_limit()
is true, and UListProcessor inherits the latter. The block then falls through
to the paragraph processor, so the markers which no longer nest are kept as
text, exactly as the blockquote guard keeps its > markers. No content is lost:

Input	Before	After
'- ' * 500 + 'x' (1 KB)	RecursionError	renders, 0.04s, 447 nested <ul>
'1. ' * 500 + 'x' (1.5 KB)	RecursionError	renders, 0.02s
'> ' + '- ' * 500 + 'x' (1 KB)	RecursionError	renders, 0.04s
600 lines of ' ' * i + '- x' (354 KB)	RecursionError	renders, 0.7s

The guard alone is not enough to make that path reasonable, which is the part
of the report about rendering rather than crashing: get_items() appended every
continuation line with '{}\n{}'.format(item, line), copying the whole item on
each line, which is quadratic in the number of lines of an item. A 1200 level
list, which is one huge item once nesting stops, took 20.0s with the guard and
3.8s once the lines of an item are collected in a list and joined once. That is
the second commit of this change, and it also speeds up any list item which
spans many lines.

Output is unchanged for input which nests below the limit: all 1204 tests pass,
and rendering a corpus of 3000 random list documents produces byte-identical
output before and after.

The remaining time for a very deep list is spent in the block processors
re-scanning the tail once per level, which this change does not address.

Two notes for review:

• The depth at which nesting stops depends on sys.getrecursionlimit(), as it
  does for blockquotes. The added test uses recursionlimit() to exercise the
  guard at a low limit, in the style of TestBlockquoteBlocks.test_nesting_limit().
• I kept the changelog entry and the quadratic fix in this pull request because
  the report asks for both halves. Happy to split them if you would prefer to
  review the guard on its own.

Update after 3.11.3

The branch is rebuilt on master at 86d1e46 so the changelog entry sits in the
Unreleased section that now follows the 3.11.3 release, rather than in the
section that shipped as 3.11.3. docs/changelog.md is the only file which
differed between the old base and master; blockprocessors.py is untouched
upstream, and the diff is still +17/-8 there. The branch is now ahead 1, behind 0, and the changelog hunk is +5/-0.

Re-ran on the rebased tree, python -m unittest discover tests:

master + new test file only:  1209 tests, FAILED (failures=1, errors=3)
master + this change:         1209 tests, OK (skipped=111)

The same four failures, all RecursionError or 10 not less than 10, so the
red/green split still holds after the rebuild.

fixes #1679

AI Assistance Disclosure

• AI tools were used and I have disclosed which ones, and have fully reviewed and verified their output.

  AI tools were use to assist with the following:

  • Code
  • Code Comments
  • Documentation
  • Changelog Entry
  • Commit Message
  • The Description Above
  • Other: tests

  The tool was the WorkBuddy coding agent. I reviewed the change and ran the
  checks below myself, and I am the one responsible for it in review.

Checklist

• This PR follows the contribution guidelines.
• The code follows the Code Style Guide.
• The commit message follows the Commit Message Style Guide.
• I have added or updated relevant docs, including release notes if applicable which follow the Documentation Style Guide.
• I have added or updated relevant tests.
• I have not requested, and will not request, an automated AI review for this PR.

Checks run locally on the rebuilt branch: python -m unittest discover tests
(1209 tests, OK, 111 skipped), flake8 markdown tests (no new findings).

[waylan]

This looks good except the changelog issue mentioned below.

[Steve-20-ai]

Changelog moved into the new Unreleased section and the branch rebuilt on master (86d1e46), so it applies cleanly again. No code change in this push, only the changelog lines relocated; details in the reply on the inline thread. Ready for another look when you have a moment.

[waylan]

Thanks for doing the work on this.