You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Shallow realizations map a basic drv (no inputDrvs) and output name to a content address.
Suppose we have a dependency graph like CompilerA -> CompilerB -> Library. These are only build-time dependencies: the outputs of each build will not depend on this dependency. For sake of argument, CompilerA is "plain old data", (like a bootstrap binary), and just uploaded as-is.
Suppose we have built all 2 derivations and uploaded the results, shallow realisations, but not deep realisations to a remote store.
Now, in another store, configured to substitute from that remote store, one tries to build Library.
Currently, this will happen:
Want to obtain Library
There is no deep realization in the cache keyed unresolved derivation
We don't know any content-addressed store object we try to download.
Wants to build Library
Want to obtain CompilerB
Finds shallow trace for CompilerB (since CompilerA is plain old data, CompilerB's derivation is already resolved)
Downloads CompilerB
Resolves Library derivation
Finds shallow trace for Library Derivation
Downloads Library
This works, but note that we downloaded CompilerB even though it is not in the runtime closure of Library.
Instead I would want something like this:
Want to obtain Library
There is no deep realization in the cache keyed unresolved derivation
Wants to resolve Library derivation
Wants resolution for CompilerB
Finds shallow trace for CompilerB
Resolves Library derivation
Finds shallow trace for Library Derivation
Downloads Library
Now we don't bother downloading CompilerB.
The way to make the second sequence of steps reality is to have "obtaining a realisation" a goal in and of itself, separate from obtaining a store object and building one. In the case where the cache doesn't have the realisation, it falls back on to just building it, but in the case where it does it doesn't need to fall back on downloading store objects. Dependencies between these goals would allow us to resolve derivations through arbitrary many inputDrv edges without downloading any store objects.
Before doing this, we should attempt #11927 so this code is not nearly as annoying to work with.
The text was updated successfully, but these errors were encountered:
Shallow realizations map a basic drv (no
inputDrvs
) and output name to a content address.Suppose we have a dependency graph like
CompilerA
->CompilerB
->Library
. These are only build-time dependencies: the outputs of each build will not depend on this dependency. For sake of argument,CompilerA
is "plain old data", (like a bootstrap binary), and just uploaded as-is.Suppose we have built all 2 derivations and uploaded the results, shallow realisations, but not deep realisations to a remote store.
Now, in another store, configured to substitute from that remote store, one tries to build
Library
.Currently, this will happen:
Library
Library
CompilerB
CompilerB
(sinceCompilerA
is plain old data,CompilerB
's derivation is already resolved)CompilerB
Library
derivationLibrary
DerivationLibrary
This works, but note that we downloaded
CompilerB
even though it is not in the runtime closure ofLibrary
.Instead I would want something like this:
Library
Library
derivationCompilerB
CompilerB
Library
derivationLibrary
DerivationLibrary
Now we don't bother downloading
CompilerB
.The way to make the second sequence of steps reality is to have "obtaining a realisation" a goal in and of itself, separate from obtaining a store object and building one. In the case where the cache doesn't have the realisation, it falls back on to just building it, but in the case where it does it doesn't need to fall back on downloading store objects. Dependencies between these goals would allow us to resolve derivations through arbitrary many
inputDrv
edges without downloading any store objects.Before doing this, we should attempt #11927 so this code is not nearly as annoying to work with.
The text was updated successfully, but these errors were encountered: