Repository navigation
📚 DOCS: freeze the section_ref token shape and make its grammar normative - #169
Merged
Merged
Conversation
`§` followed by more than 4300 digits made `int()` raise ValueError (Python's integer string conversion limit), aborting the whole render. A level longer than nine digits now makes the rule decline, so the text renders literally and no shorter prefix is captured. Nine digits keeps every number below 10**9 (a signed 32-bit int) and far below the conversion limit, which cannot be configured below 640.
Drop the many-refs timing case: it goes through the valid-reference path this change does not touch, and on PyPy it mostly measures markdown-it-py's renderer, whose string concatenation is quadratic there (it failed the 1s budget, and took about 3s with --cov). Raise the budget to 3s, which still catches a quadratic scan (tens of seconds at 200k characters), and drop the redundant timeout marker (pyproject.toml already sets timeout = 10). Say in the docstring that leading zeros count towards the nine digits.
The budget catches a scan that does Python-level work per character, or a backtracking regex. A quadratic scan built from string slices can still pass it, so the docstring no longer claims every quadratic scan would take tens of seconds.
Assert every field of the section_ref token (type, nesting, children, tag, attrs, markup, content and meta) in one test, and say in the plugin docstring that the token shape is part of the public API from 0.7.0.
State the grammar in the plugin docstring as rules: ASCII digits only, the ASCII-letter look-ahead (a non-ASCII letter does not block a match), no trailing dot, the nine-digit bound, the contexts where no reference forms, and that \§ is not a backslash escape (the backslash stays visible), with § and § giving a literal sign. Pin the statements the fixtures did not cover yet with fixture rows: non-ASCII digits, the two entities, an autolink and raw HTML. Note that markdown-it-py's renderInlineAsText keeps only text tokens, so a renderer that wants the reference in image alt text should use the section_ref token's content.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #169 +/- ##
==========================================
+ Coverage 92.80% 94.66% +1.86%
==========================================
Files 31 40 +9
Lines 1835 2325 +490
==========================================
+ Hits 1703 2201 +498
+ Misses 132 124 -8
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #168 (the nine-digit bound). Retarget to
masteronce #168 merges.The
section_reftoken becomes public API with 0.7.0. MyST-Parser reads it, and other implementations port the grammarfrom this plugin, so this PR pins the token shape and states the grammar as rules before the tag.
There is no behaviour change.
type="section_ref",nesting=0,children=None,tag="",attrs={},markup="§",contentas written ("§01.02") andmeta={"numbers": [1, 2]}.The docstring says "The token shape is part of the public API from 0.7.0."
autofunction):.is not taken;\§were a CommonMark escape. It is not, because§is not ASCIIpunctuation: the backslash stays visible and no reference forms.
§and§give a literal sign.renderInlineAsTextkeeps onlytexttokens, so in plain HTML a reference disappearsfrom image alt text, as inline math does. A renderer should use the token's
content.§١,§３), the two entities, an autolink, and raw HTML.Checks run:
pre-commit run --all-files;sphinx-build -nW --keep-going -b html docs/.Each new test was checked against a mutant of the behaviour it pins, and each one fails under its mutant.