Parser with passive input?

[cppden]

PEGTL allows to parse data by chunks (partial input) but it's only possible when parser drives the input, i.e. doing active reads when it needs more data to parse. It works fine for files but consider HTTP or similar when data comes from a socket and we can't do blocking read of that socket. Instead we can only read some chunks of data when it's received and pass a chunk to the parser.
For now I had to split HTTP grammar in 2 parts - head and body, and detect end of head externally then pass whole chunk of input received for head to head-parser, then do even more nasty stuff for body because of problems with chunks.
Perhaps it's possible for PEGTL to support a special mode when it would parse a given chunk of data till it can and then return the size of data not parsed and preserve state of parser so that when more data arrives a user could pass the data not parsed on previous step with additional data chunk just received.

[ColinH]

The state of the parser consists of the State arguments and the current call stack with all of its automatic variables. This makes it rather impossible to return from a partial parsing run so that it can be resumed later. The classical way to achieve this kind of producer-consumer coupling is with coroutines (or with proper continuations, if available, which of course make everything easy). As soon as C++20 coroutines are fully functional in GCC or Clang I'll check them out in detail and hopefully they'll allow us to put something together.

[cppden]

Thanks, Colin. I expected major problems but thought it might be possible to identify (even with help of a user, e.g. like in Lua with yield but it supports couroutines natively) some points where state could be collapsed to a managable size. But if everything is hold on stack then no way of course...
And yet if you ever consider implemeting this even with coroutines I'd be even more thankful that I am now. :) Thanks!

[ColinH]

I'm very much looking forward to my compiler implementing standard coroutines, and this will be the first thing I'll use them for.