gh-99064: Ignore the encoding declaration when parsing decoded text - #156734
Conversation
…text ElementTree.parse() with a text file mis-decoded the text in the C implementation: _parse_whole() encoded it as UTF-8, but left expat to honor the encoding declared in the document. It now overrides the encoding, as XMLParser.feed() already does for str data.
serhiy-storchaka
added
needs backport to 3.13
vstinner
left a comment
vstinner
left a comment
There was a problem hiding this comment.
LGTM.
_elementtree.XMLParser.feed() already calls EXPAT(st, SetEncoding)(self->parser, "utf-8"); if the input is an Unicode string, so doing the same in _elementtree.XMLParser._parse_whole() sounds correct to me.
Co-authored-by: Victor Stinner <vstinner@python.org>
|
Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14, 3.15. |
|
Sorry, @serhiy-storchaka, I could not cleanly backport this to |
|
Sorry, @serhiy-storchaka, I could not cleanly backport this to |
|
Sorry, @serhiy-storchaka, I could not cleanly backport this to |
|
GH-156750 is a backport of this pull request to the 3.15 branch. |
bedevere-app
Bot
removed
the
needs backport to 3.15
|
GH-156751 is a backport of this pull request to the 3.14 branch. |
|
GH-156752 is a backport of this pull request to the 3.13 branch. |
… text (GH-156734) (GH-156751) ElementTree.parse() with a text file mis-decoded the text in the C implementation: _parse_whole() encoded it as UTF-8, but left expat to honor the encoding declared in the document. It now overrides the encoding, as XMLParser.feed() already does for str data. (cherry picked from commit c83013c)
… text (GH-156734) (GH-156752) ElementTree.parse() with a text file mis-decoded the text in the C implementation: _parse_whole() encoded it as UTF-8, but left expat to honor the encoding declared in the document. It now overrides the encoding, as XMLParser.feed() already does for str data. (cherry picked from commit c83013c)
2 participants
ElementTree.parse()mis-decodes a text file in the C implementation:_parse_whole()encodes the read string as UTF-8, but leaves expat to honor the encoding declared in the document.XMLParser.feed()overrides the encoding forstrdata (bpo-16986), but_parse_whole()was never changed, soparse()without an explicit parser has been the odd one out. It now overrides it too, and a text source declaring a non-ASCII-compatible encoding (UTF-16, UTF-32) is no longer aParseError.