Skip to content

🐛 FIX: section_ref no longer raises on very long digit runs - #168

Merged
chrisjsewell merged 3 commits into
masterfrom
maint/section-ref-digit-limit
Nov 10, 2026
Merged

chrisjsewell merged 3 commits into
masterfrom
maint/section-ref-digit-limit

Conversation

@chrisjsewell

Copy link
Copy Markdown
Member

The section_ref plugin added in #144 crashes the whole render on hostile input:
a § followed by more than 4300 digits makes int() raise ValueError,
because Python limits integer string conversion to 4300 digits by default.
For example, md.render("see §" + "1" * 5000) raises on master.
This blocks tagging 0.7.0.

The rule now declines when any dotted level of the number has more than nine digits
(leading zeros count). The whole text then renders literally:
no shorter prefix is captured, so a number is never split.
For example, §1.1234567890.2 stays plain text.
References whose levels have at most nine digits behave exactly as before.

Why nine digits:

  • every number stays below 10**9, so it fits a signed 32-bit integer for downstream consumers;
  • it is far below the conversion limit, which cannot be configured lower than 640,
    so sys.set_int_max_str_digits cannot bring the crash back;
  • no real document has a billion sections.

The regex is unchanged. It has no ambiguous quantifiers, so it scans in linear time.

Tests:

  • fixtures for a level at the bound, one past it, an over-long middle level
    (a later §3 on the same line is still captured), and leading zeros counting towards the bound;
  • § + 5000 digits renders as text with no exception;
  • meta["numbers"] at the bound, and four over-long shapes that decline to a single text token;
  • a timing check with a 3s budget on a long digit run, a long dotted run and many long runs
    (a per-character quadratic scan or a backtracking regex would take far longer).

Without the fix, ten of the new tests fail.
The full suite passes on CPython and PyPy, pre-commit passes,
and the docs build with -nW.

`§` followed by more than 4300 digits made `int()` raise ValueError
(Python's integer string conversion limit), aborting the whole render.

A level longer than nine digits now makes the rule decline, so the text
renders literally and no shorter prefix is captured. Nine digits keeps
every number below 10**9 (a signed 32-bit int) and far below the
conversion limit, which cannot be configured below 640.
Drop the many-refs timing case: it goes through the valid-reference path
this change does not touch, and on PyPy it mostly measures markdown-it-py's
renderer, whose string concatenation is quadratic there (it failed the 1s
budget, and took about 3s with --cov).

Raise the budget to 3s, which still catches a quadratic scan (tens of
seconds at 200k characters), and drop the redundant timeout marker
(pyproject.toml already sets timeout = 10).

Say in the docstring that leading zeros count towards the nine digits.
The budget catches a scan that does Python-level work per character, or
a backtracking regex. A quadratic scan built from string slices can
still pass it, so the docstring no longer claims every quadratic scan
would take tens of seconds.
@codecov

codecov Bot commented Nov 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.66%. Comparing base (d11bdaf) to head (25c8d31).
⚠️ Report is 33 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #168      +/-   ##
==========================================
+ Coverage   92.80%   94.66%   +1.86%     
==========================================
  Files          31       40       +9     
  Lines        1835     2325     +490     
==========================================
+ Hits         1703     2201     +498     
+ Misses        132      124       -8     
Flag Coverage Δ
pytests 94.66% <100.00%> (+1.86%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@chrisjsewell
chrisjsewell merged commit 7997835 into master Nov 10, 2026
13 checks passed
@chrisjsewell
chrisjsewell deleted the maint/section-ref-digit-limit branch November 10, 2026 14:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant