JULIA_REVISE_POLL=1 needed on WSL2

[ericphanson]

The new "Windows subsystem for linux 2" (WSL2, which has some docs here: https://docs.microsoft.com/en-us/windows/wsl/wsl2-index) is nice in that it lets you use the same filesystem for your windows install and for linux running on top (compared to WSL1 which used a separate filesystem for linux). It seems that one needs JULIA_REVISE_POLL=1 since I think it works like an NTFS mounted network drive. I think the docs in https://timholy.github.io/Revise.jl/stable/config/#Polling-and-NFS-mounted-code-directories:-JULIA_REVISE_POLL-1 could then be updated for WSL2.

[timholy]

See the note at the very bottom of the page: https://timholy.github.io/Revise.jl/stable/config/#Polling-and-NFS-mounted-code-directories:-JULIA_REVISE_POLL-1

[ericphanson]

Ah, I hadn't read the note carefully enough and did mix up NTFS and NFS.

Consider this a bug report then: Revise does not work for me on the new WSL2 without polling, but works once I enable polling. (Note that WSL2 is different than WSL1 and as far as I remember, Revise works just fine on WSL1).

[timholy]

Interesting. Thanks for the bug report, then. Might be microsoft/WSL#4293? Do you see any warnings?

[ericphanson]

No, it just silently didn’t update. And during my debugging attempts, I saw src folder of the package I was trying to revise was correctly listed in Revise.watched_files, so I don’t think it was a configuration problem either.

[timholy]

Having it in Revise.watched_files won't help if the OS doesn't provide notification. This seems very unlikely, but it's worth checking, does Julia identify as Sys.islinux() on such a system?

Revise.jl/src/Revise.jl

Line 49 in 8336b10

if Sys.islinux() && err isa Base.IOError && err.code == -28 # ENOSPC

Does FileWatching pass its tests on this system?

[ericphanson]

does Julia identify as Sys.islinux()?

Yep:

julia> Sys.islinux()
true

julia> versioninfo()
Julia Version 1.4.2
Commit 44fa15b150* (2020-05-23 18:35 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-8.0.1 (ORCJIT, skylake)
Environment:
  JULIA_REVISE_POLL = 1
  JULIA_NUM_THREADS = 3

shell> uname -r
4.19.104-microsoft-standard

Does FileWatching pass its tests on this system?

Yep:

eric@DESKTOP-S2O7JNE:/mnt/c/Users/eric/julia$ make test-FileWatching

    JULIA test/FileWatching
Test     (Worker) | Time (s) | GC (s) | GC % | Alloc (MB) | RSS (MB)
FileWatching  (1) |        started at 2020-07-08T01:56:13.882
FileWatching  (1) |    33.43 |   0.74 |  2.2 |     183.90 |   264.44

Test Summary: | Pass  Total
  Overall     |  506    506
    SUCCESS

However, makeing Julia may have revealed a clue: it gave me warnings about clock skew. I'll try fixing that and test out revise again tomorrow.

[ericphanson]

However, makeing Julia may have revealed a clue: it gave me warnings about clock skew. I'll try fixing that and test out revise again tomorrow.

So I checked with date and the clock in the WSL2 environment seems fine, and ran sudo hwclock -s as mentioned in microsoft/WSL#4245 (comment), then tested Revise again and it did not pick up the changes. So I think it's not a clock issue then.

Anything else I can try?

[timholy]

The plot thickens 😄. Perhaps the next easy step would be to set Revise.watching_files[] = true before you load packages. Then make a revision and see whether it picks anything up.

Because it may be frustrating to wait for me to respond to individual queries, perhaps I should let you know that the main watching code is here. You could sprinkle @show lines throughout and try to figure out whether there's any form of notification. The docstrings should explain at least some of what's going on, be sure to read the one for revise_file_queued.

[nhabedi]

I came across this bug report because I'm having the same kind of problem with Revise and WSL2. I didn't investigate Revise in detail, but I think the fact that FileWatching passes its tests is misleading. Below is what I can reproduce. The gist of it seems to be that FileWatching works within the "native" file system of WSL, but ceases to work in the normal Windows file system. (One funny thing is that it makes a slight difference if the file is modified from a WSL program - but only if you watch the directory it's in. That's the last working case below.)

julia> using FileWatching

julia> watch_file("/tmp/")
# change /tmp/foo from WSL bash
FileWatching.FileEvent(false, true, false) 

julia> watch_file("/tmp/foo")
# change /tmp/foo from WSL bash
FileWatching.FileEvent(false, true, false)

julia> watch_file("/tmp/")
# change /tmp/foo from Windows-Emacs
FileWatching.FileEvent(false, true, false) 

julia> watch_file("/tmp/foo")
# change /tmp/foo from Windows-Emacs
FileWatching.FileEvent(false, true, false)

julia> watch_file("/mnt/c/Users/edi/Desktop")
# change c:/Users/edi/Desktop/foo from WSL bash
FileWatching.FileEvent(false, true, false)

julia> watch_file("/mnt/c/Users/edi/Desktop/foo")
# change c:/Users/edi/Desktop/foo from WSL bash
# NOTHING HAPPENS

julia> watch_file("/mnt/c/Users/edi/Desktop")
# change c:/Users/edi/Desktop/foo from Windows-Emacs
# NOTHING HAPPENS

julia> watch_file("/mnt/c/Users/edi/Desktop/foo")
# change c:/Users/edi/Desktop/foo from Windows-Emacs
# NOTHING HAPPENS

julia> versioninfo()
Julia Version 1.4.2
Commit 44fa15b150* (2020-05-23 18:35 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-8.0.1 (ORCJIT, skylake)

[timholy]

@nhabedi, thanks for the update, and sorry I never responded. It seems like it might be good to file this as a Julia bug. Would you have time to do so? I can copy/paste your report but then I can't easily perform any followup.

[nhabedi]

@nhabedi, thanks for the update, and sorry I never responded. It seems like it might be good to file this as a Julia bug. Would you have time to do so? I can copy/paste your report but then I can't easily perform any followup.

Sure, no problem. I just did that: JuliaLang/julia#37029

[timholy]

See also #615 (closed as a dup but it does have some extra information).

[thchr]

I've been hit by this for the past year or so, and only found the solution here now: I think it would be nice to add to the documentation that (as long as microsoft/WSL#4739 is an upstream WSL2 bug) Revise will not work properly when edits are made in the local/native Windows file system and Julia is running under WSL2.

I.e., Revise will work fine under WSL2 if the tracked packages are in the associated Linux filesystem (e.g., under ~/[...]), but not if the tracked packages are in /mnt/c/[...] etc. But setting ENV["JULIA_REVISE_POLL"] = "1" in the Julia startup.jl file will make the latter work, albeit with a small polling delay on changes.