White spaces ignored after keywords

[madchicken]

I have a simple grammar with few keywords:

find     =  { ^"find" }
in = { ^"in" }
name = @{ ('a'..'z'|'A'..'Z')+ }
​
in_clause = { in ~ name}
find_clause   = { find ~ name ~ in_clause? }
expr           = { find_clause+ }
query          = { SOI ~ expr ~ END }
​
begin_comment =  { "#" }
COMMENT       = _{ begin_comment ~ (!NEWLINE ~ ANY)* ~ NEWLINE? }
WHITESPACE    = _{ " " | "\t" | NEWLINE }
END           = _{ ";"+ ~ EOI }
​

It works nicely but ignores whitespaces AFTER keywords, so it's able to parse also things like FINDfoo inbar;
The only way I found to prevent this behavior is to change the keyword definitions into

find = ${ ^"find" ~ WHITESPACES+ }
in = ${ ^"in" ~ WHITESPACES+ }

but it doesn't sound right to me. Also, the parsed rule for keywords now contains trailing spaces.
What am I missing here?

[rix0rrr]

I have the same problem. Naively I thought I could do this:

rule = { "keyword" ~ WHITESPACE ~ "next_word" }

But of course that doesn't work, because ~ will have eaten all available whitespace already so WHITESPACE will never match anymore.

Ignore tokens

The keywords now also take up space in the token stream, even though I don't care about them (the keyword is mostly implied from the rule that was matched). This seems to be unsolvable, since in order to both force at least one whitespace AND the token to not appear I need a rule that is both atomic (to suppress the behavior of ~) and silent (to suppress the token), and it doesn't seem possible to make a rule both atomic and silent.

Forcing a whitespace separator

It seems unnecessarily hard to force at least a single whitespace separator. It seems like the Pest parser should give us more control here?

• A new operator like ~~, which translates to WHITESPACE+ instead of WHITESPACE* would have done it: "keyword" ~~ "next_word")
• A new operator like - (proposed here would have done it: "keyword" - WHITESPACE ~ "next_word"
• Even a lookbehind operator would have done it: "keyword" ~ <&WHITESPACE ~ "next_word" (though lookbehind is probably hard/bad to implement)

Requiring a whitespace separator seems like it should be simple, but it's borderline impossible (without resorting to dropping the implicit WHITESPACE feature and taking full ws control... which rather defeats the point of this feature)

[cathaysia]

I also encountered a similar problem. My rules are as follows:

char = {"char"}
scoped_name = { ASCII_ALPHA+ }

type = { char | scoped_name }

Then pest always parse char8 as rule char. I think this is because pest ends prematurely while parsing.

@tomtau did you have any idea about it?

[cathaysia]

the following code can solve this:

char = {"char" ~ !(!WHITE_SPACE)}
scoped_name = { ASCII_ALPHA+ }

type = { char | scoped_name }

[tomtau]

I think this is because pest ends prematurely while parsing.
@tomtau did you have any idea about it?

https://pest.rs/book/grammars/peg.html#eagerness / https://pest.rs/book/grammars/peg.html#non-backtracking