Dynamic token name

[Serhioromano]

I have token like this

const variable = createToken({ name: "simbol", pattern: /[a-zA-Z]+[a-zA-Z0-9_]*/ });

But I use it in different cases with $.CONSUME. In one place it is the name of the function or variable or function parameter, in another place it is name of namespace or program or a name of a custom type. In my parsed tree I want to know a different type of this symbol. Can I use one Regex to use to parse different types of tokens?

[bd82]

You can modify the resulting CST (Concrete Syntax Tree) by using LABEL

• chevrotain/packages/cst-dts-gen-test/test/snapshots/alternation_label/input.ts

  Line 23 in efdaf49

  { ALT: () => this.CONSUME(Token1, { LABEL: "item" }) }

• https://chevrotain.io/documentation/10_4_2/interfaces/ConsumeMethodOpts.html#LABEL

[Serhioromano]

I used it but it did not change label in the diagram.

[bd82]

Looks like a bug in the diagrams creations, likely somewhere in this file:

• https://github.com/Chevrotain/chevrotain/blob/master/packages/chevrotain/diagrams/src/diagrams_builder.js

[Serhioromano]

Ш could fix that probably but I cannot find it. I use Chevrotain in the other project and I installed it with npm i chevrotain and how I nave ./node_modules/chevrotain and inside I do not see any packages folder although there is node_modules/chevrotain/diagrams/src/diagrams_builder.js file. Although If I place console.log(123) in that file is is not shows when I generate diagram like this.

const parser = new StParser();
parser.input = lexingResult.tokens;
parser.iecst();

if (parser.errors.length > 0) {
    console.log(parser.errors[0]);
} else {
    //console.log(parser);
}


const serializedGrammar = parser.getSerializedGastProductions();
const htmlText = chevrotain.createSyntaxDiagramsCode(serializedGrammar);
const outPath = path.resolve(__dirname, "./")
fs.writeFileSync(outPath + "/generated_diagrams.html", htmlText);

I would start with getSerializedGastProductions but I cannot find it.

[bd82]

The diagrams_builder.js is only executed when the html is rendered.

• https://github.com/Chevrotain/chevrotain/blob/master/packages/chevrotain/src/diagrams/render_public.ts#L31

The createSyntaxDiagramsCode() function creates an HTML source code that once rendered in a browser will display the diagrams.

[bd82]

It seems that the labels for Terminals vs Non-Terminals are represented with different property names in the
serialized GAST productions.

          {
            "type": "Alternative",
            "definition": [
              {
                "type": "NonTerminal",
                "name": "array",
                "idx": 0,
                "label": "JSON_ARR"
              }
            ]
          },
          {
            "type": "Alternative",
            "definition": [
              {
                "type": "Terminal",
                "name": "True",
                "label": "True",
                "idx": 0,
                "terminalLabel": "BOL_TRUE",
                "pattern": "true"
              }
            ]
          },

Although it also seems like in both cases these labels are ignored in the diagrams...

[Serhioromano]

Sounds like not what I could fix on my own :)