Add to support load multiple grammars for derive generator.

[huacnlee]

Resolve #197 #333

What New

Allows load multiple grammar files for derive generator.

Example:

#[derive(Parser)]
#[grammar = "base.pest"]
#[grammar = "json.pest"]
pub struct JSONParser;

By this changes, we can split the sharing rules in to a single file, and then load it for other grammar files.

In my AutoCorrect case, this will useful, because there have so many duplicate rules (e.g. space, comment, newline) defined in each grammars.

https://github.com/huacnlee/autocorrect/tree/v2.5.5/autocorrect/grammar

[tomtau]

Great work, thanks! I think that multi-derive generator approach (i.e. this commit 157f159 ) is simple and can address the basic use cases, so I'd be happy to merge that right away -- perhaps could you split it into two PRs?

For the meta grammar extension (i.e. this commit 30c15cf ), I think it's fine, but would want more feedback from other folks. If my understanding of this use <filename> feature is correct, it'll simply concat all definitions into one grammar (i.e. it's unlike that previous work by @bobbbay #660 that tried to add module-level scoping). Is that correct? If so, what is the advantage of writing use ... inside a grammar over writing multiple derive annotations?

If this meta grammar extension is to be added, it's maybe worth adding more complex tests (e.g. use statements across multiple files A.pest -> B.pest -> C.pest, or multiple use statements in one file) and documenting this feature in the grammar reference in rustdocs/readme/book (especially its limitations or what one needs to watch for... I assume for this concat approach, there can't be a cycle / mutual dependency, one needs to make sure each grammar file is included only once...?).

[huacnlee]

I have splitted #759

[CAD97]

I agree that using concat for multiple #[grammar] attributes makes sense.

I'm pretty firmly against implementing a use via textual inclusion (Rust include!, C++ #include), though. At a minimum, use/import needs some “#pragma once” behavior, so you can have a diamond use¹. Ideally, there's some sort of module namespacing done, but plenty of languages manage to get away without that, and we need a flat list of items for the Rule enum anyway, unless we want to do name mangling indirection through a macro.

If it's spelled include!("base.pest") I'm less against it, since the dumb nature is more obvious.

Footnotes

1. a uses b and c, and b and c both use d, and this doesn't result in the multiple definition of items from d. ↩

[tomtau]

lgtm, the meta grammar extension can comments can continue on #759