Specify code generation target?

[Korporal]

Does the Antlr4 grammar allow me to specify lexer or parser code on a per-language basis?

e.g., consider:

@lexer::members {
    
	// This code MUST compile fine as Java and C#

	public String KeywordLexiconCode = "ru";

	public Boolean Lexicon(String Code)
	{
		return Code == KeywordLexiconCode;
	}
}

Is there any way to do this:

@lexer::members (java) {
    
	// This code MUST compile fine as Java and C#

	public String KeywordLexiconCode = "ru";

	public Boolean Lexicon(String Code)
	{
		return Code == KeywordLexiconCode;
	}
}

and

@lexer::members (csharp) {
    
	// This code MUST compile fine as Java and C#

	public String KeywordLexiconCode = "ru";

	public Boolean Lexicon(String Code)
	{
		return Code == KeywordLexiconCode;
	}
}

or something like that?

[KvanTTT]

No. Somebody suggested the similar feature some time ago: #1045 (comment)

[kaby76]

You can do this easily right now by preprocessing with a few tree rewrites with Trash. You'd have to create a syntax that fits the @whatever1::whatever2 { ... } syntax, e.g.:

$ cat FooBar.g4
grammar FooBar;
@csharplexer::member {
public bool Fu() { return true; }
}
@javalexer::member {
public boolean Fu() { return true; }
}
start : 'my' 'a' 'b' 'c\'' EOF;

Kenne@DESKTOP-DL44R7B MINGW64 ~/a1
$ trparse FooBar.g4 | trdeltree ' //action_[actionScopeName/identifier/RULE_REF/text()="csharplexer"]' | trreplace ' //action_/actionScopeName/identifier/RULE_REF' 'lexer' | trtext
CSharp 0 FooBar.g4 success 0.0451983
grammar FooBar;
@lexer::member {
public boolean Fu() { return true; }
}
start : 'my' 'a' 'b' 'c\'' EOF;

Voila. You have an Antlr grammar per target of choice. Really, we should not be using Antlr visitors and listeners. Way too slow.

At some point, you have to decide whether these little extensions are worth the effort of adding code into the Antlr parser generator itself, vs creating an extension that can be refactored into Antlr code.

[Korporal]

You can do this easily right now by preprocessing with a few tree rewrites with Trash. You'd have to create a syntax that fits the @whatever1::whatever2 { ... } syntax, e.g.:

$ cat FooBar.g4
grammar FooBar;
@csharplexer::member {
public bool Fu() { return true; }
}
@javalexer::member {
public boolean Fu() { return true; }
}
start : 'my' 'a' 'b' 'c\'' EOF;

Kenne@DESKTOP-DL44R7B MINGW64 ~/a1
$ trparse FooBar.g4 | trdeltree ' //action_[actionScopeName/identifier/RULE_REF/text()="csharplexer"]' | trreplace ' //action_/actionScopeName/identifier/RULE_REF' 'lexer' | trtext
CSharp 0 FooBar.g4 success 0.0451983
grammar FooBar;
@lexer::member {
public boolean Fu() { return true; }
}
start : 'my' 'a' 'b' 'c\'' EOF;

Voila. You have an Antlr grammar per target of choice. Really, we should not be using Antlr visitors and listeners. Way too slow.

At some point, you have to decide whether these little extensions are worth the effort of adding code into the Antlr parser generator itself, vs creating an extension that can be refactored into Antlr code.

The Antlr grammar could be altered to allow optional additional metadata:

From:

action_
   : AT (actionScopeName COLONCOLON)? identifier actionBlock
   ;

to:

action_
   : AT (actionScopeName COLONCOLON)? identifier actionArgs? actionBlock
   ;

action_args
   : LPAR identifier (, identifier)* RPAR
   ;

Or something like that. Then everything needed to generate the code from the g4 is inside the g4, also because the addition is optional it would be totally backward compatible.

Something that makes Antlr so powerful is the ease with which such changes can be safely made, so it strikes me as low hanging fruit...

[kaby76]

Yes, you can instead change the Antlr4 grammar to make your extension and use trgen to create a standardized parser for it. This can then be used by trparse to parse your input with @lexer::members (java) extension syntax, then use trdelete, insert, rewrite, etc., to refactor it, and trtree to reconstruct the text. I've wanted to integrate an XLST engine to Trash for full tree rewrite capabilities, but I haven't had the time.

[Korporal]

I devised a simpler workaround, I need only Java and C# to be generated:

@lexer::members 
{    
	// This code MUST compile for both Java and C#
        // Some constructs here require support code be
	// defined as C# partial and extension classes.
    
	private String KeywordLexiconCode = "en";

	public Boolean Lexicon(String Code)
	{
		return Code == KeywordLexiconCode;
	}

	public void SetLexicon(String Code)
	{
		KeywordLexiconCode = Code;
	}

	public void ReverseLexeme()
	{
		StringBuilder builder = new StringBuilder();
		builder.append(getText());
		builder.reverse();
		setText(builder.toString());
	}
}

That is pure, standard Java but C# has no reverse (C# uses Reverse) or toString and getText (C# has properties) are only present in the generated Java code. So since the generated lexer class is "partial" for C# I just added methods getText and setText and also created extension methods for the String and Stringbuilder methods that have lower case names.

Now that generated code above compiles fine for Java and C#.

[kaby76]

You are trying to "solve" the "target agnostic grammar" problem. There really isn't a solution.

The action code in @members can certainly share syntax between a few targets that you intend to support, e.g., Java and C#. But, the grammar is compromised by the fact that someone could modify the action with syntax that is target specific, thus removing one of the targets you carefully tried to share syntax you tried to support.

I would suggest you adopt "target agnostic format", which forces the action to be of the most basic syntax possible--a method call--and push all target-specific code into base classes per target. You can then use the entire target language and runtime to support your computation rather than guess "Is this ok syntax between languages X, Y, and Z?". Yes, there's a lot more "boilerplate" code you will have to write. This is the format adopted in the V4 grammars. The advantage of your original idea is that one wouldn't need to write the boilerplate code ("import/using/#include, namespace/package, class/def/...).