Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add crude type parsing for jai tokenizer #106

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

rainbowism
Copy link

It'll highlights the types in most situations. It isn't a very robust solution but after some testing it seems to work well enough.

@focus-editor
Copy link
Owner

Thanks for the PR! I tried to do it myself but decided against it because there are too many special cases (see screenshots)

image

image

I'm honestly not sure it's even worth it to try to highlight user types using tokens at this point. The plan was to leverage the compiler eventually to get a list of types asynchronously and put it in a hash map for highlighting (and other things)

@rainbowism
Copy link
Author

That's fair. I have actually resolved those 2 scenarios but if the plan is for using the compiler rather than a bespoke tokenizer then that seems like a better plan.

@focus-editor
Copy link
Owner

There's more than 2 unfortunately. Also I checked and it slows down the parser by 5-10%, which is not great because we're parsing on every keystroke.

Here's some more (handling these cases would slow it down even further).

image

image

image

Would you agree to wait until we try it with the compiler?

@rainbowism
Copy link
Author

rainbowism commented Jul 23, 2023

That's fine with me. I can just use this personally for now in the meanwhile, or somebody can pull in my branch if they wish to use it themselves.

@focus-editor
Copy link
Owner

Great. Thank you for your understanding!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants