Repository navigation
Fix patchedast aborting on ''' inside a double-quoted f-string - #844
marlon-costa-dc wants to merge 4 commits into
Conversation
end_quote_char() picked the closing delimiter by max(len(quote)) over every
quote char found anywhere in the f-string body, so a ''' sequence inside a
double-quoted f-string (e.g. f"cfg = '''\n{value}") was selected as the
delimiter. That produced a reconstructed token consume_joined_string() could
not locate, raising `ValueError: substring not found` -> MismatchedTokenError,
which aborts rename/inline for the entire module.
Pick the delimiter the f-string literally ends with (QUOTE_CHARS is ordered
longest-first, so """/''' win over "/' correctly) so the closing quote matches
the opening one regardless of quote characters appearing in the body.
Adds a regression test; full patchedast, rename and inline suites stay green.
There was a problem hiding this comment.
Pull request overview
Fixes patchedast f-string reconstruction when the f-string body contains quote sequences (notably ''') that previously confused _JoinedStr.end_quote_char(), causing consume_joined_string() failures and aborting refactorings for the whole module.
Changes:
- Update
_JoinedStr.end_quote_char()to select the closing delimiter based on what the source literally ends with (rather than the longest quote found in the body). - Add a regression test covering a double-quoted f-string containing
'''inside its body. - Document the fix in
CHANGELOG.md.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
rope/refactor/patchedast.py |
Fixes f-string closing delimiter detection to avoid mismatches caused by quote sequences in the f-string body. |
ropetest/refactor/patchedasttest.py |
Adds a regression test proving the bug and preventing future regressions. |
CHANGELOG.md |
Notes the bug fix and its impact on refactorings. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
# Conflicts: # CHANGELOG.md
Addresses review: end_quote_char fell back to the last character when no quote delimiter matched, contradicting its intent and masking parsing bugs. It now raises MismatchedTokenError naming the line, like the other token-matching failures in patchedast.
|
Rebased this fix onto current master (post-#896): the old The updated implementation reads the delimiter from whatever quote sequence actually terminates the region and raises |
Description
patchedast._JoinedStr.end_quote_char()selects the closing delimiter bymax(len(quote))over every quote character found anywhere in the f-string body. When a'''sequence appears inside a double-quoted f-string, e.g.:f"cfg = '''\n{value}"the
'''in the body is picked as the delimiter instead of the actual closing". That produces a reconstructed token thatconsume_joined_string()cannot locate, raising:which surfaces as
MismatchedTokenErrorand aborts rename / inline for the entire module — a single such f-string anywhere in a project breaks refactoring of that file.Root cause
end_quote_char()chose the quote by length precedence rather than by which delimiter the f-string literally ends with. The opening quote (start_quote_char) was already correct; the closing detection was not symmetric.Fix
Pick the delimiter the f-string literally ends with.
QUOTE_CHARSis ordered longest-first (""",''',",'), so triple-quoted strings are still matched before single-character quotes, and the closing quote now always matches the opening one regardless of quote characters in the body.Reproducer (before the fix)
Testing
test_handling_format_strings_with_triple_quote_in_bodyinropetest/refactor/patchedasttest.py(red before, green after).ropetest/refactor/= 944 passed, 7 skipped, 1 xfailed.patchedasttest.py= 141 passed, 6 skipped.Checklist