Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ArrayIndexOutOfBoundsException when running more than one process in parallel #2034

Open
apauley opened this issue Apr 12, 2023 · 5 comments · May be fixed by #3399
Open

ArrayIndexOutOfBoundsException when running more than one process in parallel #2034

apauley opened this issue Apr 12, 2023 · 5 comments · May be fixed by #3399
Assignees
@Gedochao
Labels
Bloop Issues tied with Bloop integration. bug Something isn't working scripting Issues tied to *.sc script inputs.

Comments

@apauley
Copy link

apauley commented Apr 12, 2023

Version(s)
1.0.0-RC1 and 0.2.1, under both Linux and Mac OS X

Describe the bug
When spawning more than one scala-cli process in parallel, some of the processes fail with:

Error: java.lang.ArrayIndexOutOfBoundsException: Index 6 out of bounds for length 0

To Reproduce
Given any scala-cli script, e.g:

$ cat printargs.sc
println(args.mkString(" "))

And the following bash script which spawns the first process in the background:

$ cat run
#!/usr/bin/env bash

scala-cli printargs.sc -- First run in background &
scala-cli printargs.sc -- Second run in parallel

It frequently exits with the following error:

$ ./run
Deduplicating compilation of project_103be31561-9b85cc8ba0 from bsp client 'scala-cli 1.0.0-RC1' (since 0.071s)
Error: java.lang.ArrayIndexOutOfBoundsException: Index 6 out of bounds for length 0
For more details, please see '/home/andreas/tmp/scala-cli/.scala-build/stacktraces/1681305420-14395538719813530497.log'

The error is more likely to occur if you spawn more background processes.

Full stack trace:

java.lang.ArrayIndexOutOfBoundsException: Index 6 out of bounds for length 0
  org.objectweb.asm.ClassReader.readShort(ClassReader.java:3608)
  org.objectweb.asm.ClassReader.<init>(ClassReader.java:197)
  org.objectweb.asm.ClassReader.<init>(ClassReader.java:180)
  org.objectweb.asm.ClassReader.<init>(ClassReader.java:166)
  org.objectweb.asm.ClassReader.<init>(ClassReader.java:287)
  scala.build.postprocessing.AsmPositionUpdater$.postProcess$$anonfun$3(AsmPositionUpdater.scala:59)
  scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
  scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
  scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575)
  scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573)
  scala.collection.AbstractIterator.foreach(Iterator.scala:1300)
  scala.build.postprocessing.AsmPositionUpdater$.postProcess(AsmPositionUpdater.scala:70)
  scala.build.postprocessing.ByteCodePostProcessor$.postProcess(ByteCodePostProcessor.scala:15)
  scala.build.Build$.$anonfun$63(Build.scala:1139)
  scala.collection.immutable.List.flatMap(List.scala:293)
  scala.collection.immutable.List.flatMap(List.scala:79)
  scala.build.Build$.postProcess(Build.scala:1140)
  scala.build.Build$.doPostProcess$1$$anonfun$2(Build.scala:252)
  scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
  scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
  scala.Option.foreach(Option.scala:437)
  scala.build.Build$.doPostProcess$1(Build.scala:253)
  scala.build.Build$.doBuild$1$$anonfun$1(Build.scala:362)
  scala.build.EitherCps$Helper.apply(EitherCps.scala:19)
  scala.build.Build$.doBuild$1(Build.scala:369)
  scala.build.Build$.buildScopes$1$$anonfun$1(Build.scala:373)
  scala.build.EitherCps$Helper.apply(EitherCps.scala:19)
  scala.build.Build$.buildScopes$1(Build.scala:400)
  scala.build.Build$.build$$anonfun$1(Build.scala:402)
  scala.build.EitherCps$Helper.apply(EitherCps.scala:19)
  scala.build.Build$.build(Build.scala:414)
  scala.build.Build$.build$$anonfun$3(Build.scala:575)
  scala.build.compiler.ScalaCompilerMaker.withCompiler(ScalaCompilerMaker.scala:31)
  scala.build.compiler.ScalaCompilerMaker.withCompiler$(ScalaCompilerMaker.scala:7)
  scala.build.compiler.BloopCompilerMaker.withCompiler(BloopCompilerMaker.scala:11)
  scala.build.Build$.build(Build.scala:597)
  scala.cli.commands.run.Run$.runCommand(Run.scala:294)
  scala.cli.commands.default.Default.runCommand(Default.scala:69)
  scala.cli.commands.default.Default.runCommand(Default.scala:48)
  scala.cli.commands.ScalaCommand.run(ScalaCommand.scala:365)
  scala.cli.commands.ScalaCommand.run(ScalaCommand.scala:352)
  caseapp.core.app.CaseApp.main(CaseApp.scala:150)
  scala.cli.commands.ScalaCommand.main(ScalaCommand.scala:337)
  caseapp.core.app.CommandsEntryPoint.main(CommandsEntryPoint.scala:120)
  scala.cli.ScalaCliCommands.main(ScalaCliCommands.scala:124)
  scala.cli.ScalaCli$.main0(ScalaCli.scala:203)
  scala.cli.ScalaCli$.main(ScalaCli.scala:107)
  scala.cli.ScalaCli.main(ScalaCli.scala)

Expected behaviour
Multiple concurrent scala-cli processes should reliably and successfully execute independently of other running processes.
@apauley apauley added the bug Something isn't working label Apr 12, 2023
@jiramares
Copy link

jiramares commented Dec 13, 2024 •
edited
Loading

Same here :-(

I have some *.sc files to operate with csv files but when i start following bash command

./csv-distribute.sc -i file.csv -e cp1250 -h -k 3,1 -v 4-5 -x 4 -f $DISTR_FILE -y 1,10 -a 2-4 -w 7-8 -l "Head1:Head2:Head3" | \
  ./csv-map.sc -e cp1250 -h -K 1 | \
  tee temp.csv | \
  ./csv-map.sc -e cp1250 -h -K 1 | \
  ./csv-sum.sc -e cp1250 -h -l Celkem -s 5-6 -o output.csv

I get the same error (but not all the time). So compilation doesn't work well in parallel. If I run commands using scala-cli --server=false command is much slower but works (never got such an error). Is it problem of compilation server?

I am running JVM 21, scala 3.3.3 and scala-cli 1.5.4

Any workarounds? Is there the way to build *.sc file into jar file not to compile at runtime at all?

@Gedochao
Copy link
Contributor

Hmm... If --server=false is causing it, then this does look like it might be tied to Bloop not handling multiple processes well...
The parallel processes makes me think if it could be tied to scala/scala3#21947 smh, but the error is different.
@tgodzik any ideas on Bloop's side?
@jiramares can you provide the stack trace you're getting? is it the same as in the issue description?

@Gedochao Gedochao added Bloop Issues tied with Bloop integration. scripting Issues tied to *.sc script inputs. labels Dec 17, 2024
@jiramares
Copy link

jiramares commented Dec 17, 2024 •
edited
Loading

NNN, It's working well with --server=false and doesn't work without it.

@Gedochao
Copy link
Contributor

but when it doesn't work, what error are you getting?

@jiramares
Copy link

jiramares commented Dec 17, 2024 •
edited
Loading

Same as described in the issue.

I have 2 stacktraces (they are quite same but not identical):

java.lang.ArrayIndexOutOfBoundsException: Index 6 out of bounds for length 0
  org.objectweb.asm.ClassReader.readShort(ClassReader.java:3621)
  org.objectweb.asm.ClassReader.<init>(ClassReader.java:198)
  org.objectweb.asm.ClassReader.<init>(ClassReader.java:180)
  org.objectweb.asm.ClassReader.<init>(ClassReader.java:166)
  org.objectweb.asm.ClassReader.<init>(ClassReader.java:288)
  scala.build.postprocessing.AsmPositionUpdater$.postProcess$$anonfun$3(AsmPositionUpdater.scala:59)
  scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
  scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
  scala.collection.IterableOnceOps.foreach(IterableOnce.scala:619)
  scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:617)
  scala.collection.AbstractIterator.foreach(Iterator.scala:1303)
  scala.build.postprocessing.AsmPositionUpdater$.postProcess(AsmPositionUpdater.scala:55)
  scala.build.postprocessing.ByteCodePostProcessor$.postProcess(ByteCodePostProcessor.scala:17)
  scala.build.Build$.$anonfun$61(Build.scala:1217)
  scala.collection.immutable.List.flatMap(List.scala:294)
  scala.collection.immutable.List.flatMap(List.scala:79)
  scala.build.Build$.postProcess(Build.scala:1210)
  scala.build.Build$.doPostProcess$1$$anonfun$2(Build.scala:296)
  scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
  scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
  scala.Option.foreach(Option.scala:437)
  scala.build.Build$.doPostProcess$1(Build.scala:287)
  scala.build.Build$.doBuild$1$$anonfun$1(Build.scala:406)
  scala.build.EitherCps$Helper.apply(EitherCps.scala:19)
  scala.build.Build$.doBuild$1(Build.scala:308)
  scala.build.Build$.buildScopes$1$$anonfun$1(Build.scala:417)
  scala.build.EitherCps$Helper.apply(EitherCps.scala:19)
  scala.build.Build$.buildScopes$1(Build.scala:416)
  scala.build.Build$.build$$anonfun$1(Build.scala:446)
  scala.build.EitherCps$Helper.apply(EitherCps.scala:19)
  scala.build.Build$.build(Build.scala:281)
  scala.build.Build$.build$$anonfun$3$$anonfun$1(Build.scala:626)
  scala.util.Either.flatMap(Either.scala:360)
  scala.build.compiler.ScalaCompilerMaker.withCompiler(ScalaCompilerMaker.scala:37)
  scala.build.compiler.ScalaCompilerMaker.withCompiler$(ScalaCompilerMaker.scala:9)
  scala.build.compiler.BloopCompilerMaker.withCompiler(BloopCompilerMaker.scala:14)
  scala.build.Build$.build$$anonfun$3(Build.scala:611)
  scala.build.EitherCps$Helper.apply(EitherCps.scala:19)
  scala.build.Build$.build(Build.scala:596)
  scala.cli.commands.run.Run$.runCommand(Run.scala:334)
  scala.cli.commands.shebang.Shebang$.runCommand(Shebang.scala:33)
  scala.cli.commands.shebang.Shebang$.runCommand(Shebang.scala:26)
  scala.cli.commands.ScalaCommand.run(ScalaCommand.scala:390)
  scala.cli.commands.ScalaCommand.run(ScalaCommand.scala:371)
  caseapp.core.app.CaseApp.main(CaseApp.scala:166)
  scala.cli.commands.ScalaCommand.main(ScalaCommand.scala:356)
  caseapp.core.app.CommandsEntryPoint.main(CommandsEntryPoint.scala:370)
  scala.cli.ScalaCliCommands.main(ScalaCliCommands.scala:125)
  scala.cli.ScalaCli$.main0(ScalaCli.scala:320)
  scala.cli.ScalaCli$.main(ScalaCli.scala:124)
  scala.cli.ScalaCli.main(ScalaCli.scala)

java.lang.ArrayIndexOutOfBoundsException: Index 6 out of bounds for length 0
  org.objectweb.asm.ClassReader.readShort(ClassReader.java:3621)
  org.objectweb.asm.ClassReader.<init>(ClassReader.java:198)
  org.objectweb.asm.ClassReader.<init>(ClassReader.java:180)
  org.objectweb.asm.ClassReader.<init>(ClassReader.java:166)
  org.objectweb.asm.ClassReader.<init>(ClassReader.java:288)
  scala.build.internal.MainClass$.findInClass(MainClass.scala:51)
  scala.build.internal.MainClass$.findInClass(MainClass.scala:48)
  scala.build.internal.MainClass$.find$$anonfun$1(MainClass.scala:93)
  scala.collection.Iterator$$anon$10.nextCur(Iterator.scala:594)
  scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:608)
  scala.collection.mutable.Growable.addAll(Growable.scala:61)
  scala.collection.mutable.Growable.addAll$(Growable.scala:57)
  scala.collection.immutable.VectorBuilder.addAll(Vector.scala:1825)
  scala.collection.immutable.Vector$.from(Vector.scala:1397)
  scala.collection.IterableOnceOps.toVector(IterableOnce.scala:1452)
  scala.collection.IterableOnceOps.toVector$(IterableOnce.scala:1452)
  scala.collection.AbstractIterator.toVector(Iterator.scala:1303)
  scala.build.internal.MainClass$.find(MainClass.scala:91)
  scala.build.Build$Successful.mainClassesFoundInProject$lzyINIT1(Build.scala:62)
  scala.build.Build$Successful.mainClassesFoundInProject(Build.scala:62)
  scala.build.Build$Successful.foundMainClasses(Build.scala:68)
  scala.cli.commands.run.Run$.maybeRun$1$$anonfun$1(Run.scala:161)
  scala.build.EitherCps$Helper.apply(EitherCps.scala:19)
  scala.cli.commands.run.Run$.maybeRun$1(Run.scala:160)
  scala.cli.commands.run.Run$.runCommand(Run.scala:344)
  scala.cli.commands.shebang.Shebang$.runCommand(Shebang.scala:33)
  scala.cli.commands.shebang.Shebang$.runCommand(Shebang.scala:26)
  scala.cli.commands.ScalaCommand.run(ScalaCommand.scala:390)
  scala.cli.commands.ScalaCommand.run(ScalaCommand.scala:371)
  caseapp.core.app.CaseApp.main(CaseApp.scala:166)
  scala.cli.commands.ScalaCommand.main(ScalaCommand.scala:356)
  caseapp.core.app.CommandsEntryPoint.main(CommandsEntryPoint.scala:370)
  scala.cli.ScalaCliCommands.main(ScalaCliCommands.scala:125)
  scala.cli.ScalaCli$.main0(ScalaCli.scala:320)
  scala.cli.ScalaCli$.main(ScalaCli.scala:124)
  scala.cli.ScalaCli.main(ScalaCli.scala)

@Gedochao Gedochao self-assigned this Dec 17, 2024
@Gedochao Gedochao moved this from To do to In progress in Issue Board Dec 31, 2024
This was referenced Dec 31, 2024
Draft
Open
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Gedochao Gedochao

Labels
Bloop Issues tied with Bloop integration. bug Something isn't working scripting Issues tied to *.sc script inputs.
Projects
Issue Board
Status: In progress
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[WIP] Improve handling for parallel Scala CLI runs Gedochao/scala-cli
3 participants
@apauley @jiramares @Gedochao