curl_easy_setopt: 4 when using Pkg
#244
Comments
|
That error code indicates |
|
By doing Also by doing using Libdl
filter!(contains("curl"), dllist())I get this: |
|
using Downloads
Downloads.Curl.CURL_VERSION_STR
Downloads.download("https://pkg.julialang.org/registries", verbose=true) |
|
First of all sorry if I made a mistake on opening too many issues. Then thanks for the help. julia> using Downloads
julia> Downloads.Curl.CURL_VERSION_STR
"libcurl/8.4.0 Schannel zlib/1.2.13 libssh2/1.11.0 nghttp2/1.52.0"
julia> Downloads.download("https://pkg.julialang.org/registries", verbose=true)
┌ Error: curl_easy_setopt: 4
└ @ Downloads.Curl C:\Users\aless\.julia\juliaup\julia-1.10.3+0.x64.w64.mingw32\share\julia\stdlib\v1.10\Downloads\src\Curl\utils.jl:50
* WARNING: failed to open cookie file ""
* Couldn't find host pkg.julialang.org in the .netrc file; using defaults
* Hostname pkg.julialang.org was found in DNS cache
* Trying [2a04:4e42::729]:443...
* Connected to pkg.julialang.org (2a04:4e42::729) port 443
* schannel: disabled automatic use of client certificate
* using HTTP/1.x
> GET /registries HTTP/1.1
Host: pkg.julialang.org
Accept: */*
User-Agent: curl/8.4.0 julia/1.10
* schannel: server closed the connection
< HTTP/1.1 301 EU internal redirect trigger
< Connection: close
< Content-Length: 0
< Server: Varnish
< Retry-After: 0
< Location: https://eu-central.pkg.julialang.org/registries
< x-geo-continent: EU
< x-geo-country: IT
< x-geo-region: GO
< Accept-Ranges: bytes
< Date: Wed, 29 May 2024 22:55:41 GMT
< Via: 1.1 varnish
< X-Served-By: cache-mxp6960-MXP
< X-Cache: HIT
< X-Cache-Hits: 0
< X-Timer: S1717023341.111815,VS0,VE0
<
* Closing connection
* schannel: shutting down SSL/TLS connection with pkg.julialang.org port 443
* Issue another request to this URL: 'https://eu-central.pkg.julialang.org/registries'
* Couldn't find host eu-central.pkg.julialang.org in the .netrc file; using defaults
* Found bundle for host: 0x185fba58320 [serially]
* Can not multiplex, even if we wanted to
* Re-using existing connection with host eu-central.pkg.julialang.org
> GET /registries HTTP/1.1
Host: eu-central.pkg.julialang.org
Accept: */*
User-Agent: curl/8.4.0 julia/1.10
< HTTP/1.1 302 Moved Temporarily
< Server: nginx/1.26.0
< Date: Wed, 29 May 2024 22:55:41 GMT
< Transfer-Encoding: chunked
< Connection: keep-alive
< Location: /registries.conservative
< X-lb-strategy: pkgservers_roundrobin
<
* Ignoring the response-body
* Connection #1 to host eu-central.pkg.julialang.org left intact
* Issue another request to this URL: 'https://eu-central.pkg.julialang.org/registries.conservative'
* Couldn't find host eu-central.pkg.julialang.org in the .netrc file; using defaults
* Found bundle for host: 0x185fba58320 [serially]
* Can not multiplex, even if we wanted to
* Re-using existing connection with host eu-central.pkg.julialang.org
> GET /registries.conservative HTTP/1.1
Host: eu-central.pkg.julialang.org
Accept: */*
User-Agent: curl/8.4.0 julia/1.10
< HTTP/1.1 200 OK
< Server: nginx/1.26.0
< Date: Wed, 29 May 2024 22:55:41 GMT
< Content-Type: application/octet-stream
< Content-Length: 88
< Connection: keep-alive
< Last-Modified: Wed, 29 May 2024 22:43:47 GMT
< ETag: "6657afa3-58"
< Accept-Ranges: bytes
< X-lb-strategy: pkgservers_roundrobin
<
* Connection #1 to host eu-central.pkg.julialang.org left intact
"C:\\Users\\aless\\AppData\\Local\\Temp\\jl_TcXmhkFutW" |
|
Any news? issue is still ongoing, I want to help but I cannot figure out what is the problem... |
|
So I have no idea what is happening here. You seem to have the correct version of libcurl that we ship. Everything seems to be working correctly as well. But you're getting those error messages indicating that a curl option is being set that libcurl doesn't understand. I'll have to think about what to do next to debug this... |
|
Ok, please try this: using Downloads
@eval Downloads.Curl function setopt(easy::Easy, option::Integer, value)
puts("setopt: $option = $value")
@check curl_easy_setopt(easy.handle, option, value)
end
Downloads.download("https://pkg.julialang.org/registries")This monkey-patches the definition of |
|
This is the result of the script provided (thank you by the way!!!!) setopt: 10103 = Ptr{Nothing} @0x0000024d0aef8320
setopt: 10010 = Ptr{UInt8} @0x0000024d0ab88818
setopt: 20312 = Ptr{Nothing} @0x0000024d4eaf39e0
setopt: 10313 = Ptr{Nothing} @0x0000024d0aef8320
setopt: 20079 = Ptr{Nothing} @0x0000024d4eaf3c80
setopt: 10029 = Ptr{Nothing} @0x0000024d0aef8320
setopt: 20011 = Ptr{Nothing} @0x0000024d4eaf3ee0
setopt: 10001 = Ptr{Nothing} @0x0000024d0aef8320
setopt: 20219 = Ptr{Nothing} @0x0000024d4eaf4140
setopt: 10057 = Ptr{Nothing} @0x0000024d0aef8320
setopt: 99 = true
setopt: 52 = true
setopt: 68 = 50
setopt: 161 = 7
setopt: 10018 = curl/8.4.0 julia/1.10
setopt: 51 = 1
setopt: 10031 =
setopt: 216 = 8
setopt: 78 = 30
setopt: 20 = 20
setopt: 19 = 1
setopt: 10153 = C:\Users\aless\.ssh\id_rsa
setopt: 10152 = C:\Users\aless\.ssh\id_rsa.pub
setopt: 10026 = Ptr{Nothing} @0x0000000000000000
setopt: 10002 = https://pkg.julialang.org/registries
setopt: 64 = true
setopt: 10183 = C:\Users\aless\.ssh\known_hosts
setopt: 13 = 0
setopt: 41 = false
setopt: 20094 = Ptr{Nothing} @0x0000000000000000
setopt: 10095 = Ptr{Nothing} @0x0000000000000000
setopt: 10023 = Ptr{Downloads.Curl.curl_slist_t} @0x0000024d7d7844a0
setopt: 44 = false
"C:\\Users\\aless\\AppData\\Local\\Temp\\jl_AhgFAPMHkN" |
|
@StefanKarpinski side note, maybe |
|
Does it not show the error anymore when you do that? I was hoping for a setopt message right before the error and then we could infer that the option printed immediately prior was the one it's complaining about. |
|
I just noted one thing, if this code here: using Downloads
@eval Downloads.Curl function setopt(easy::Easy, option::Integer, value)
puts("setopt: $option = $value")
@check curl_easy_setopt(easy.handle, option, value)
end
Downloads.download("https://pkg.julialang.org/registries")is run from either the julia REPL or in the terminal by calling And the error is present. But if I run from VS Code using the Is there something related with Powershell? I'm using Powershell 7.4.2 in Windows Terminal 1.20.11381.0. |
|
Do you have one of the following environment variables set?
|
|
Yeah this error can be reduced on Windows to setting SSL_CERT_DIR to an empty folder. The Conda activation script of OpenSSL started setting The easiest way to reproduce is by installing pixi, go to an empty folder and run: The And the SSL_CERT_DIR that it sets under |
|
What do you think the best mitigation would be? Should we check if the directory is empty and not apply the option if it is? |
|
Actually, this error seems to indicate that setting the directory to anything at all might not be supported. Does it only happen when the directory is empty? |
|
I can confirm that with my install of miniforge (which is just conda + mamba) and having done |
|
Is |
|
At least for me, the
I don't really know, I don't fully understand the issue. The
This may explain why we get a CURLE_NOT_BUILT_IN (4) return code. So does that mean we should not set ca roots path at all on Windows? I'm no expert though. |
|
I think we may want to print a warning there if its set but we can't honor it, at least once so the user can understand why the setting isn't respected. |
|
It would be less scary to have 1 warn log instead of many error logs for sure. If I'm reading this correctly it's what curl tools also do. Though I don't see this warning pop up when I do a curl command. I tried to look into why this option doesn't work on Windows, but I can just see that it's been in the docs for over a decade. The code mentions Windows as well: https://github.com/curl/curl/blob/282b9fe8ff3d23c1c975b6e29eb3eb083fac6751/lib/setopt.c#L2161-L2173 So if this is documented not to work on Windows perhaps even a warn log is too much? |
The ssl folder is structured as follows in my pc I also agree that maybe logging this is not necessary if it's a known windows issue |
|
It may be a known curl issue, but if someone sets one of the relevant options via NetworkOptions and we just ignore it in Downloads then I think the user deserves a warning. Otherwise they may be confused about why they're using the wrong CA carts. They don't know or care that we use libcurl. |
|
I don't really understand why they set SSL_CERT_DIR to an empty dir in conda-forge/openssl-feedstock#157. Would be nice if only SSL_CERT_FILE or some other mechanism would also work for them. |
|
But it doesn't really matter does it? The fact that the directory is empty is not actually relevant here. Whatever the contents of the directory, we cannot support it, so we should warn and not try setting the option. |
|
No indeed not for this repo. I meant perhaps we can ask them not to set SSL_CERT_DIR. In the Anaconda managed OpenSSL recipe they did a similar PR earlier on where they only set SSL_CERT_FILE. But that's best discussed on https://github.com/conda-forge/openssl-feedstock. |
Due to an issue between the curl and openssl libraries the `pixi run build` tasks failed. In JuliaLang/Downloads.jl#244 the cause of the issue is gfound to be environment variables being set that conflict. This commit adds an environment variable to the tasks failing, clearing the SSL variables --------- Co-authored-by: Martijn Visser <[email protected]>
|
Hmm the top post states that everything with Pkg is working correctly, but I just read https://discourse.julialang.org/t/how-to-fix-ssl-cert-issues-in-pkg/115495 and realize operations involving git like Here I'm again setting SSL_CERT_DIR to an empty directory just like conda openssl does, to simulate what most users starting julia from a conda environment on Windows will see: The GitError comes from here. |
|
I see in the NetworkOptions.ca_roots function that SSL_CERT_DIR is searched before SSL_CERT_FILE. Can this be reversed? The documentation states
More closely following openssl here should fix this issue. |
This aligns better with the behavior of openssl. See JuliaLang/Downloads.jl#244 (comment)
In windows with a fresh install of
juliaupwith Julia 1.10.3 when using any command in the Pkg REPL gives out this message:Everything is working correctly but there is this error continuously
The text was updated successfully, but these errors were encountered: