Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix normalized URI generation for UNC path #374

Draft
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

Friendseeker
Copy link
Member

@Friendseeker Friendseeker commented Nov 2, 2024

Issue

IO.directoryURI and Path.absolute both calls toURI.normalize to convert a java.io.File to a normalized URI, which generates invalid URI for UNC path. As a result, sbt crashes when we try to open a project in a WSL directory.

Example: say we have a dir: java.io.File representing Ubuntu-24.04/home/jerrytan/testwsl/ in WSL, then calling dir.toPath.toUri.normalize yields file:/wsl$/Ubuntu-24.04/home/jerrytan/testwsl/ , which then leads to

java.lang.RuntimeException: Invalid build URI (no handler available): file:/wsl$/OracleLinux_9/home/user/project/
        at scala.sys.package$.error(package.scala:30)
        at sbt.internal.Load$.$anonfun$builtinLoader$1(Load.scala:489)
        at sbt.internal.BuildLoader.$anonfun$apply$3(BuildLoader.scala:244)
        at scala.Option.getOrElse(Option.scala:189)
        at sbt.internal.BuildLoader.apply(BuildLoader.scala:244)
        at sbt.internal.Load$.loadURI$1(Load.scala:554)
        at sbt.internal.Load$.loadAll(Load.scala:570)
        at sbt.internal.Load$.loadURI(Load.scala:500)
        at sbt.internal.Load$.load(Load.scala:479)

Fix

Call toPath.normalize.toUri instead. For above example, it yields file:////wsl$/Ubuntu-24.04/home/jerrytan/testwsl/ which can be loaded by sbt. Note that the new file URL still do not follow the best practice as documented in Eugene's blog post, but it indeed do not crash.

Closes sbt/sbt#7135

Credit to reporter of sbt/sbt#7135 for detailed investigation of the issue.

@Friendseeker Friendseeker changed the title Handle UNC path gracefully Fix normalized URI generation for UNC path Nov 2, 2024
@Friendseeker Friendseeker marked this pull request as ready for review November 2, 2024 01:46
@@ -1160,7 +1160,7 @@ object IO {
*/
def directoryURI(dir: File): URI = {
assertAbsolute(dir)
directoryURI(dir.toURI.normalize)
directoryURI(dir.toPath.normalize.toUri)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add unit test for this plz?

@@ -1177,7 +1177,7 @@ object IO {
else
new URI(str + "/")

dirURI.normalize
new File(dirURI).toPath.normalize.toUri
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The creation of the first URI object seems wasteful here. See also #132, which states that URI object can lead to hundreds of MB in heap.

Copy link
Member Author

@Friendseeker Friendseeker Nov 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall investigate this once I regain access to a Windows Machine.

Indeed there is something fishy there. If dirURI follows u2 notation, then calling dirURI.normalize should be correct. While I think the other changes in the PR makes sense, this particular change is just a bandaid that somehow fixes the sbt crash without fixing the underlying issue.

I now actually suspect the hidden issue might be with IO.toURL, which is called by ProjectRef.apply on sbt side. This is just a hypothesis though.

Copy link
Member Author

@Friendseeker Friendseeker Nov 3, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Btw going slightly off topic but your blogpost about file URI schema is so helpful! Without it I probably need to spend a lot of time researching about file URI.

@Friendseeker Friendseeker marked this pull request as draft November 5, 2024 02:12
@Friendseeker
Copy link
Member Author

Friendseeker commented Nov 5, 2024

I must say this is more complex than I anticipated...

The PR, as of its current state, is inadequate.

The issue is that, the dollar sign actually matters when parsing a URI. Probably since $ is a reserved character.

val path = "\\\\wsl$\\Ubuntu-24.04\\home\\jerrytan\\testwsl"
val wslDir = new File(path)
println(wslDir.toPath.toUri)

// Output: file:////wsl$/Ubuntu-24.04/home/jerrytan/testwsl/

val path = "\\\\wsl.localhost\\Ubuntu-24.04\\home\\jerrytan\\testwsl"
val wslDir = new File(path)
println(wslDir.toPath.toUri)

// Output: file://wsl.localhost/Ubuntu-24.04/home/jerrytan/testwsl/

The u4 URI is fragile. After calling .normalize() on it, it becomes file:/wsl.localhost/Ubuntu-24.04/home/jerrytan/testwsl/ which sbt cannot parse. The right conversion is to convert it to u2, which does not break (remains u2) after normalization.

So here's the current state:

  • Path.toURI generates invalid URI for UNC path containing $ sign
  • IO.toFile is correct, other than the fact that it does not append "/" to the end of URI when file is directory
  • 'IO.directoryURI(dir: File)` is broken.
  • IO.directoryURI(uri: URI) is probably correct, as long as the uri follows spec.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Sbt doesn't recognise WSL UNC path - Invalid build URI (no handler available)
2 participants