Interactive reads

[jjttjj]

Addresses #438

I had mainly wanted to be able to upgrade my clojure repl to a convex repl and that has been working well for me. But there could be things I'm missing, and edge cases and implications of this change that I'm still missing. Any feedback is welcome. Below are a few things I'm unsure of

EOF handling

One thing I'm unsure of is if there should be better EOF handling with this feature. Should there be an EOF exception thrown when a read is attempted on EOF? The following currently just throws a ParseException:

(let [r (java.io.PushbackReader. (java.io.StringReader. "(+ 1 2 3)"))]
  [(AntlrReader/readOne r)
   (AntlrReader/readOne r)])

Should there be more options for EOF handling, as the clojure.core/read allows?

Testing

For now the tests I added are fairly minimal, just checking for things I ran into during development. I have tried out using readOne in place of read for all the relevant test cases and verified that things work as expected. I could try to refactor the reader tests to do in all appropriate tests as well. Or just copy and paste several other of the read tests to a readOne version.

[jjttjj]

Something like this is required to properly throw a ParseException on 1.0e0.1234, instead of just reading 1.0 and leaving the rest on the stream.

[jjttjj]

Allows for a parse excepton to be thrown on : input

[jjttjj]

These two consume implementations are the same as the Unbuffered(Char|Token)Stream class that they extend except without the sync call at the end, which consumes more characters/tokens from the input

[jjttjj]

Even after disabling the syncing in the streams with our custom consume implementations below, there is a single additional character that is read from the stream when a form is parsed. I'm currently unsure of the exact mechanism that causes this but I believe it involves consume calls in antlr's LexerATNSimulator class. It's possible that there's a better way to handle this there. But just unreading from the stream here seems to work well.

[jjttjj]

The built in CommonTokenStream that is used for the other reads filters for tokens only from the default channel. This feature doesn't use that token stream and instead modifies UnbufferedTokenStream, which doesn't include channel filtering.

It would be possible to add that functionality back into the token stream we use, but since we are not doing anything with the hidden token channel, it should be fine to just skip these tokens instead.

[mikera]

This looks very good, happy to merge when you think it is ready. Are you on Discord @jjttjj and if so what is your handle?

Regarding EOF handling, I think some way to control this might be useful. I can see a couple of options:

• Optional extra argument to say what happens when EOF is reached. Unfortunately, just retuning null isn't useful because that is also a valid form but
• Have a static dummy ACell instance that acts as a marker? This wouldn't be a valid ACell in any way but a singleton that can be used to detect EOF
• Have a function like hasAnotherForm that detects a next form. Might be inefficient though if used on large input...