Document common issues, and how to solve them.

[oxcrow]

Hi everyone,

I'm loving using pest for building a compiler.

I would request to collect the common issues that happen while using pest from users, and documenting them.

To help pest maintainers in this effort, we can collect the common issues we faced, along with how we solved them.

Using our testimonials, pest maintainers can prepare better documentation for future users.

If it's okay with everyone, I will start.

How to parse recursive nested AST nodes using rust/pest/pest-ast?

Using pest-ast, it took me few hours to figure out how to represent a grammar of "A module can contain modules and functions"

Initially I thought this representation would work, but it doesn't:

#[derive(Debug, Clone, FromPest)]
#[pest_ast(rule(Rule::module))]
pub struct Module {
    id: Id,
    modules: Vec<Module>,
    functions: Vec<Functions>,
}

This makes the parser only collect module AST nodes first into the modules vector, then once its finished, the parser collects the function AST nodes into the functions vector.

This is problematic, and restricts the language I want to develop.

Mainly, code like this, fail to parse ...

mod core {
    fn x() {
        // okay so far
    }

    mod y {
        // error: parser crashes, 
        // since it expects all modules to exist before functions, 
        // so it can not process this random module that exists out of place.
    }
}

After many hours of trial and error, I figured out how to solve this.

We need to represent the nested items as an enum, and store those items into a vector.

This works perfectly!

#[derive(Debug, Clone, FromPest)]
#[pest_ast(rule(Rule::module))]
pub struct Module {
    id: Id,
    items: Vec<ModuleItem>,
}

#[derive(Debug, Clone, FromPest)]
#[pest_ast(rule(Rule::module_item))]
pub enum ModuleItem {
    Module(Module),
    Function(Function),
}

So, if it's okay with pest maintainers,
I would request other users to also report their issues, and also report if/how they fixed them.

Thanks!

[tomtau]

Sounds good. In addition to that, it could be perhaps added to some sections of the book: https://github.com/pest-parser/book/
(The AST conversion sounds more like something that could have gone to this unfinished project chapter: https://github.com/pest-parser/book/blob/master/src/examples/awk.md )