Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert semantic token types to TextMate scopes #21

Merged
merged 4 commits into from
Oct 4, 2024

Conversation

sungshik
Copy link
Collaborator

@sungshik sungshik commented Oct 4, 2024

Consider:

lexical Digit = @category="number" [0-9];
  • Before this PR, the generated TextMate rule for Digit would just get scope name number. However, while number is a semantic token type, it is not one of the standard TextMate scope names in VS Code (VS Code reports "No theme selector" and doesn't highlight). For semantic highlighting, VS Code does the conversion from incoming semantic token types to TextMate scope names itself, but scope names in TextMate grammars are uninterpreted. Thus, we need to make sure that the Rascal-Textmate converter already converts semantic token types (that occur in a Rascal grammar) to standard TextMate scopes before inserting them in a TextMate grammar.
  • After this PR, the generated TextMate rule gets scope name constant.numeric

Additionally, this PR also moves the conversion of legacy semantic token types into the general converter (used to be done only for Rascal's own grammar, but it should always be done). Tests were updated too.

@sungshik sungshik marked this pull request as ready for review October 4, 2024 10:00
Copy link
Member

@DavyLandman DavyLandman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Smart to do this for all grammars out there 👍🏼.

Also, note that the usethesource/rascal#1928 issue exists that should at some point in the future normalize these categories. (and in turn cause extra mapping code to the lookup tables in this PR)

@sungshik sungshik merged commit 09aab8c into main Oct 4, 2024
2 checks passed
@sungshik sungshik deleted the convert-semantic-token-types-to-textmate-scopes branch October 4, 2024 12:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants