Allow skipping download of lazy artifacts

[rbvermaa]

When using PackageCompiler.jl, we ran into an issue with TimeZone.jl [0] where it downloaded all timezone data (200+ files). These files are listed as lazy. We would like to avoid downloading them, and in the issue linked, it was mentioned we should probably add an include_lazy_artifacts option.

I tried to add an include_lazy_artifacts option [1], however, it is probably not a correct solution, as it ignores the situation where there is a Dict in the Artifact section. See first part of the if at https://github.com/RelationalAI-oss/PackageCompiler.jl/blob/rv-include-lazy/src/PackageCompiler.jl#L722 .

This patch, although probably incorrect, solved our immediate issue (we ger blocked by the server when downloading too many files).

I think this probably only can be fixed in Pkg.jl, but not sure how. Any idea how to get this fixed properly?

[0] JuliaTime/TimeZones.jl#300
[1] master...RelationalAI-oss:rv-include-lazy

[NHDaly]

CC: @staticfloat - i think we talked with you and @KristofferC about this on slack. Do you think something like an include_lazy_artifacts option is the best approach? Should it be along the lines of @rbvermaa's patch, above?

Thanks!

[KristofferC]

It makes sense to me to have that option, at least.

[IanButterworth]

This is quite painful for CUDA.jl given all of the many large artifacts are lazy https://github.com/JuliaGPU/CUDA.jl/blob/master/Artifacts.toml

And a include_lazy_artifacts flag wouldn't help here, I believe. I think PackageCompiler will need to know which lazy artifacts it needs to grab in advance

[Sacha0]

Bumping as this issue bit us again just now. Given

It makes sense to me to have that option, at least.

I'll roll Rob's branch into a pull request so that we can begin iterating something. Thanks folks! :)

[KristofferC]

This is now merged into PackageCompiler