fandango convert drops ANTLR non-greedy quantifiers (*?, +?, ??)

[Nuhiat-Arefin]

When converting ANTLR grammars to .fan, fandango convert strips the non-greedy ? from EBNF suffixes and keeps only the greedy quantifier.

Converter code causing the issue:

def visitEbnfSuffix(self, ctx):
    suffix = ctx.getText() if ctx else ""
    if len(suffix) > 1:
        self.addNote(f"was '{suffix}'")
        suffix = suffix[0]
    return suffix

Because of this logic:

• *? becomes *
• +? becomes +
• ?? becomes ?

Minimal Repro

Grammar (bug_lazy.g4):

grammar BugLazy;
start: BlockComment;
BlockComment: '/*' .*? '*/';

Command:

fandango convert -o bug_lazy.fan bug_lazy.g4

Converted output:

<start> ::= <BlockComment> 
<BlockComment> ::= '/*' r'.'* '*/'  # NOTE: was '*?'

This also affects real-world grammars. For example, in g_c.g4:

MultiLineMacro: '#' (~[\n]*? '\\' '\r'? '\n')+ ~[\n]+ -> channel(HIDDEN);

is converted to:

<MultiLineMacro> ::= '#' (r'[^\n]'* '\\' '\r'? '\n')+ r'[^\n]'+  # NOTE: was '*?'; was '-> channel(HIDDEN)'

This is not semantics-preserving. In ANTLR, .*? is lazy/non-greedy; after conversion, it silently becomes greedy r'.'*.

[andreas-zeller]

Alas, Fandango does not support non-greedy quantifiers (for now), neither in grammars nor (AFAIK) in regexes. The ANTLR-to-Fandango converter outputs the "Note" comment for exactly this reason. So, yes, it drops the non-greedy part; but it does not do so silently.

To fix this, one would have to extend the Fandango producers and parsers accordingly; then, the ANTLR converter could pass the non-greedy operators through.