Consider using rule versioning to associate code with grammar rules

[sharwell]

The fork of ANTLR used by this parser provides a rule versioning feature intended to help developers avoid making mistakes during development of larger projects. Consider the following trivial and hypothetical example:

Motivating example

Suppose a language includes two primitive types: int and float:

literalType
  : INT
  | FLOAT
  ;

In the compiler for this language, code like the following is found:

LiteralTypeContext literalType = ...;
if (literalType.INT() != null) {
  return true;
} else {
  // Must be a float
  return false;
}

Now assume the developer adds a new type, short:

literalType
  : INT
  | SHORT
  | FLOAT
  ;

It's easy to see that the Java code now contains an incorrect assumption about the form of the grammar, but the change in generated code is not sufficient to produce a compiler error.

Rule versioning

Rule versioning is my proposed strategy for ensuring situations like the above produce a compilation error if the Java code is not updated to match the grammar. This functionality was instrumental in the development of GoWorks on a short timeline by helping ensure every IDE feature (completion, indentation, navigation, etc.) was updated to reflect changes made in the grammar.

An overview of this feature can be found here:
https://github.com/sharwell/antlr4/wiki/Rule-Versioning

I've found that the feature has a relatively high cost of initial adoption (understanding exactly how it works, how to effectively use the annotations, and taking the time to document existing code), but a remarkably low incremental/maintenance cost once it's used. Subjectively I found that GoWorks had a much lower rate of functionality regressions caused by grammar changes than projects I've worked on in the past, suggesting that rule versioning contributed to the long-term quality of the product.

If you are interested in incorporating this functionality, let me know and I can help with the initial adoption process. The only thing I would ask in return is detailed feedback regarding the impact it has on code maintenance and a review of the documentation so I can better explain this feature to other users.

[daniellansun]

The parser project is quite large. Rule versioning looks good. Glad to get your help to adopt the feature :-)

[daniellansun]

Apart from the code maintenance, could you please provide me some advice to improve the performance further?

[sharwell]

I may have some time Monday to look. I have a short attention span so feel free to remind me.

[daniellansun]

OK

[daniellansun]

@sharwell ping :-)

[sharwell]

❓ What editor are you using for this project? I tried opening the code with NetBeans but it keeps hanging the IDE.

[daniellansun]

Sam, I use IntelliJ IDEA. It is a project based on gradle. You can use IDE to setup a gradle project using the "build.gradle" file ;-)

P.S. I have just waken up...

[sharwell]

Thanks. I took a look and my first area of focus is the handling/placement of nls and sep in the grammar. I'll file a separate issue.

[daniellansun]

OK. Thanks for your help in advance :-)