A first party treesitter parser for laravel blade

[unansweredocd]

Hey,

I haven't seen anyone work on this yet so I figured I'd ask the community / the laravel team whether it's viable and whether anyone else has the issue I'm having.

Developers use a lot of different ways to write their laravel code, whether that's emacs, sublime text, vim, neovim, phpstorm and I suppose notepad++. Some of the aforementioned names have great laravel blade support but are behind a paywall which not everyone can afford, or they choose to ignore those options since they're a vim / emacs for life kind of person.

Wouldn't it be great if every editor had great support for laravel blade, by utilizing treesitter?

What's treesitter? Well.

Tree-sitter is a parser generator tool and an incremental parsing library. It can build a concrete syntax tree for a source file and efficiently update the syntax tree as the source file is edited. Tree-sitter aims to be:

General enough to parse any programming language

Fast enough to parse on every keystroke in a text editor

Robust enough to provide useful results even in the presence of syntax errors

Dependency-free so that the runtime library (which is written in pure C) can be embedded in any application

TL;DR, proper highlighting in blade.php files for everyone. No matter their choice of editor.

Here's my problem, I find it rather difficult to do it myself and I'm going to need some help with this. I've tried and failed many times but maybe someone in the community, or someone in the laravel team can work on this.

It would greatly improve the developer experience of everyone if Laravel itself could point to an official treesitter parser that takes the pain out of highlighting in blade.php files without pointing to expensive options like phpstorm.

Note, if you've solved this issue without treesitter be sure to let everyone know in the comments. I figured I'd ask the community since I still think that an official treesitter parser would be awesome for everyone.

[unansweredocd]

@jessarcher @driesvints

I've been in touch with you two before on other matters and I would love your input on this, maybe your large following can give it some more attention if you think it deserves that.

[jessarcher]

Hey @unansweredocd, this would be cool. GitHub uses tree-sitter as well, so it's not just editors that would benefit from it.

Unfortunately, I still haven't gotten the hang of tree-sitter queries. I wonder if there is another similar HTML-based templating language that could be used as a guide.

Presumably, this would also be helpful for creating a Blade linter/formatter as well.

[Sophist-UK]

https://github.com/tree-sitter/tree-sitter-html is a parser for HTML.

This can presumably be used as a starting point for a Blade parser - indeed since Blade is a superset of HTML you probably want to base a Blade Parser by forking and extending this.

[unansweredocd]

I recently found https://github.com/gbprod/tree-sitter-twig which helps for the projects that use twig. I bet we could take a lot from that, but I'm no pro at this tree-sitter query stuff either.

[calebdw]

Actually, I found an embedded-template parser which I think would be better to fork from. The twig parser implements a lot of rules that blade should just pass off to php and html.

[yaegassy]

It would be very useful to have a tree-sitter of blade. Recently, there seems to be one for Python's Django, so I'll paste the URL. https://github.com/interdependence/tree-sitter-htmldjango

[KaranAhlawat]

I have started work on an tree-sitter-blade parser. For now it's experimental. You can find it here: https://github.com/KaranAhlawat/tree-sitter-blade.

The commit history may be a bit confusing, sorry about that.
The support for now includes parsing Blade echo tags. Next thing I'd like to tackle are the directives Blade has, so we'll see how that goes.
All help is welcome! I'm pretty new to this stuff too and not entirely sure what I'm doing at times, esp since I'm new to Blade and PHP.
Looking for testers too, who'll do their best to try and break the current grammar :)

[yaegassy]

@KaranAhlawat Excellent!

[yaegassy]

There seems to be a repository that is actively developing a parser for blade. I'll post the URL here in case it will be of some use.

• PHP: https://github.com/Stillat/blade-parser
• TypeScript: https://github.com/Stillat/blade-parser-typescript

[KaranAhlawat]

Maybe if someone is interested in writing a language server, this could be useful. I'll keep them tabbed though, thank you!

[EmranMR]

Hello everybody and @unansweredocd I have been working on a grammar for a couple of months now, happy to announce that it is now public 🎉 . you can find it here EmranMR/tree-sitter-blade on github.

I would really appreciate if everyone can have a look and provide feedback. It is pretty much complete and up to date as of Laravel 10.x. I use Nova and it worked great there when I was running the tests(unpublished syntax extension). However I recognise tree-sitter-blade would be predominantly used in NeoVim and I have zero experience with that editor, or who knows maybe github as well! So It would be great to have your inputs/feedbacks esp @jessarcher.

I wrote it in a way so it is easy to extend the grammar as Laravel grows. This is beta, and have the following features:

• injecting html and php with simple queries
• code folding capability for nested directives if your editor supports folding
• code highlighting

Future plans:

• Next up for me is to write the tests
• javascript code injections, inertia, Livewire and alpineJS support and see where the community takes the project to.
• better documentation for contribution.

Happy Coding
Emran

[JohnathonKoster]

The structures like if/switch are also where I stopped experimenting with TS a while ago, since I couldn't come up with a clean way to apply them to custom conditions (without knowing what they are ahead of time):

@disk('local')
    <!-- The application is using the local disk... -->
@elsedisk('s3')
    <!-- The application is using the s3 disk... -->
@else
    <!-- The application is using some other disk... -->
@enddisk
 
@unlessdisk('local')
    <!-- The application is not using the local disk... -->
@enddisk

That, and the following would produce valid compiled PHP output, but makes it challenging to do any sort of parsing without some information up-front:

@disk('local')
    <!-- The application is using the local disk... -->
@elsedisk('s3')
    <!-- The application is using the s3 disk... -->
@elseif('something')
   <!-- Just a wrench. -->
@else
    <!-- The application is using some other disk... -->
@endif
 
@unlessdisk('local')
    <!-- The application is not using the local disk... -->
@endif

I am very interested to see how this might be tackled with TS.

[EmranMR]

@unansweredocd i will have a look and see how I can register the parser, thanks for the link!

@JohnathonKoster custom directives/ if-statements are in my to-do list 😉

[KaranAhlawat]

I think the way to go would be, something along the lines of

1. If you encounter a @, it maybe a directive or a verbatim (raw?) echo tag.
2. If it's an echo tag, well and good
3. If it's not, we assume it's a directive. If it's not one of the predefined directives, we assume it's a custom directive.

Also would probably run into a wall of how to differentiate between a custom if statement and just a custom directive. Anyways, these are just my first thoughts, maybe it's way easier

[JohnathonKoster]

@KaranAhlawat

That approach is similar to what I did in the TypeScript version of my parser, with the added step of having a list of things that should never be considered a Blade directive. This list is mainly comprised of CSS things like @media, @keyframes, @viewport, etc. If it passes that list, and the @ is not preceded by the escape character, doesn't seem like it might be an email address, etc. - the parser just assumes it must be a Blade directive.

After that, if I need the structural information it begins attempting to locate pairs by checking for @end<directive_name>. If it finds one it will then analyze any directives that are inside of it - if it finds any that begin with @else it assumes it must be a custom condition. Once a candidate has been flagged as just a regular paired directive, a switch statement, or any type of condition, it invokes different pairing logic to find the start/ends along the way.

Not sure how transferrable (or ideal) that approach is, but thought I'd share how the TypeScript version works (the structural analysis in the PHP version is largely the same, but it has the benefit of being able to retrieve the list of all known directives/conditions at runtime).

[KaranAhlawat]

Well, I'm definitely thinking we could use the external scanner here. Taking the approach of what many other grammars do, we could simply maintain a stack of the directive we have seen before. It might end up being a linear time operation though, since the order of usage is not fixed.

[EmranMR]

@EmranMR,

Directives

The multiple directives is really what made things tricky, take for instance the @if statements:

You could have multiple @elseif directives and those along with the @else directive should be parsed as directives but they are not the start nor end.

Same thing goes for @switch directives

@empty

This one is especially tricky because it can be used standalone or as part of a @foreach (@forelse when used with @empty)

PHP Parser

I submitted a PR to the php repo (tree-sitter/tree-sitter-php#177) to allow top level statements like the Go parser , however, it was a flop due to the aforementioned limitation of the parser. My last comment on that PR suggested that the php parser be split into separate parsers (like markdown and markdown-inline) one for just php (maybe call it phpo for php-only, or even just php-only) and one for combined php/html which would reference the php-only parser to prevent duplication

Anyways, should probably move this discussion over to the blade repo---congrats again and good job!

Splitting the php parser into html/standalone is exactly what is needed. Even the old RegEx based syntax grammars had a separate php and base-php. So hopefully that comes to life sometimes soon... 🤞

Thanks for explaining the @empty in fact I just learnt something new as I did not know that is possible! 🙏
I will get back to you on both once I am back at my computer and can run some tests. I will prob raise and issue and tag you in the discussion on the tree-sitter-blade repo so that you could kindly test things with neovim and feedback 😊

[end3r-man]

is their any blade formatter for nvim??

[end3r-man]

• https://github.com/shufo/blade-formatter

There's also a prettier plug-in for it

hello i having some issue with phpactor, while using a model like User::create im getting Method not found in App\Models\User

whats issue??

[end3r-man]

like this

[end3r-man]

[calebdw]

@end3r-man, you already posted that question on the phpactor side of things

[end3r-man]

@end3r-man, you already posted that question on the phpactor side of things

yes, forgot