Wondering if there is a better way to represent the following

[DamianReeves]

After hitting a stackoverflow trying to use many like the EBNF requires I tried applying the thinking of chain to this though, it didn't quite 100% translate from the expression example.

Is there a better way to represent this?

//> using dep com.github.j-mie6::parsley:4.5.2
//> using dep "com.lihaoyi::pprint:0.9.0"

import parsley.Parsley.{atomic, many, some}
import parsley.character.{endOfLine, letter, hexDigit, oneOf}
import parsley.syntax.character.{charLift, stringLift}
import parsley.character
import parsley.expr.chain

// LANGTAG	::=	'@' [a-zA-Z]+ ('-' [a-zA-Z0-9]+)*
val asciiLowerChar = oneOf('a' to 'z')
val asciiUpperChar = oneOf('A' to 'Z')
val asciiChar = asciiLowerChar | asciiUpperChar
val digit = oneOf('0' to '9')
val alphaNumeric = asciiChar | digit
val start = atomic('@' ~> some(asciiChar).map(_.mkString("@", "", "")))
lazy val tail = {
  val identifier = some(alphaNumeric).map(_.mkString)
  val rest = chain.left1(identifier, '-' as (_ + '-' + _))
  rest
}
val langtag = (start <~> ('-' *> tail)).map(_ + '-' + _) | start

pprint.pprintln(start.parse("@en"))
pprint.pprintln(tail.parse("GB-01-oxendict-1997"))
pprint.pprintln(langtag.parse("@en-GB01-oxendict-1997"))

[j-mie6]

You were hitting a stack overflow with many? that doesn't sound right... can you share that original parser and the version you're running on?

as an aside, your start would work nicely using stringOfSome(asciiChar).map('@' + _), or better yet, atomic('@' ~> some(asciiChar)).span

[j-mie6]

ditto val identifier = stringOfSome(alphaNumeric)

[DamianReeves]

Original was:

  val LANGTAG = {
    val asciiLowerChar = oneOf('a' to 'z')
    val asciiUpperChar = oneOf('A' to 'Z')
    val asciiChar = asciiLowerChar | asciiUpperChar
    val digit = oneOf('0' to '9')
    val start = '@' ~> some(asciiChar).map(_.mkString("@", "", ""))
    val tail =
      many('-' ~> some(asciiChar).map(_.mkString("-", "", "")))
        .foldLeft("")(_ + _)
    val langtag = (start <~> tail).map(_ + _)
    langtag
  }

[j-mie6]

I've used the improved debugger functionality in parsley-debug found in the latest 5.0.0 snapshots, and found this:

scala> detectDivergence(LANGTAG.langtag).parse("@en")
parsley.exceptions.ParsleyException:
tail is looping unproductively as pure can succeed having not consumed input.

More precise names for the body can be sourced using Collector.names
or the `named` combinator.

looking at your tail definition, though, I don't see how this can be unproductively looping (it's infinite looping by the way, not stack overflow!)

I imagine this is a parsley bug and I'll investigate it now :)

[j-mie6]

Ahhhh wait no, I see exactly what it is, the debugger was reporting something a bit unusually (good test case for the traces!)
you are using the .foldLeft combinator on a many, whereas you meant .map(_.foldLeft...) on the list. many can succeed having not consumed input, which makes foldLeft loop forever (I should have figured this out when I saw the original parent name was infix.left1, which is the combinator used to implement foldLeft).

again, what I'd do for this is to write:

val tail = many('-' ~> some(asciiChar)).span

[DamianReeves]

Thanks so much, and great catch.

[j-mie6]

no problem, this was a useful thing to see, because it'll be good to make the debugger a bit more sane for this kind of divergence: the crux is that foldLeft looks a bit like loop(pure(f) <*> this) in some sense, so the debugger notices the infinite loop when taking a snapshot of the pure and not the this, which leads to a rather uninformative message... Hopefully I can find a way to make it ignore "internal" combinators or something like that -- this would have made the error from the debugger:

tail is looping unproductively as many can succeed having not consumed input.

which is a lot more useful for understanding this problem!

[DamianReeves]

Using 0.4.5.2