[TwigComponent] Twig PreLexer gets confused by ICU MessageFormat plural syntax

[balbinus]

I have run into an edge case where I’m using ICU messages in a translatable string, and the message has plural cases in it. See here for a more complete documentation, but here’s an example string:

{count, plural, 
   one {# item}
   other {# items}
}

This trips up the Twig PreLexer and makes it look for an end-of-comment #} that it may or may not find, and ignore all <twig:Blocks> until it finds one.

Here's a minimal template as example:

There are {{ '{count, plural, one {# item} other {# items}}'|trans({'count': 42}) }}!

<twig:Test />

It outputs:

There are {42, plural, one {# item} other {# items}}!

<twig:Test />#}

The ICU syntax isn’t processed in the raw string, but this doesn’t matter in practice since it’s only here as a placeholder for translators. Real strings from XLIFF files are handled correctly. Most importantly, the HTML tag is present in the output… Now, if I modify the template to add a “fake” comment:

There are {{ '{count, plural, one {# item} other {# items}}'|trans({'count': 42}) }}!
{# Comment #}
<twig:Test />

The component is now rendered:

There are {42, plural, one {# item} other {# items}}!
TEST COMPONENT

I understand this is niche/edge case, and in any case thanks a lot for your hard work on this project and thanks for your time!

[Kocal]

Hi, and thanks for your words!

Indeed, I don't think anyone would have thought of that, but more importantly... who actually uses the message itself as a translation key? 😜

Joking aside, we will see what we can do without breaking anything :)

[Amoifr]

Hi @Kocal! 👋

I gave this a shot — turns out the PreLexer was treating any {# as a comment opener, even when it appeared inside a Twig output expression like {{ '...{# items}}' | trans(...) }}. So when no #} was found later, the rest of the template (including <twig:Test />) got swallowed as a fake comment. Classic case of "who would write a translation key like that?" meeting "someone, somewhere, did" 😄

The fix adds a top-level {{ ... }} handler in the PreLexer that consumes the expression body verbatim, with string-literal-aware scanning so that }} (or {#) inside a single/double-quoted string don't end the expression early. It also ensures the implicit {% block content %} injection still kicks in when the expression is inside a component.

Draft PR: #3518 (against 2.x) — 332 tests green (5 new regression cases for ICU plural / quote escapes / nested }} in strings).

One scope question for you before I un-draft it: {% set foo = '{# bar #}' %} would technically have the same latent issue, but I kept the patch tight on {{ }} (which is what the report describes). Want me to extend it to {% %} in the same PR, or keep it focused?

Thanks for everything you do on this project! 🙏

[balbinus]

Hahaha I agree this is not necessarily totally standard, but I'm really struggling to find a workflow for l10n that makes sense... I'm using JMS for extraction, and this trick (+ their desc() Twig function) allows me to have message keys (user.action.open, etc) + a reference translation when extracting ^^ I figured some other people may run into this issue, that's why I decided to report it ^^