Track built artifacts in the build system

[IsaacWoods]

Invoking cargo for each component, even if it doesn't need to recompile, is getting fairly slow, and will only get worse. I wonder if we can add a layer of tracking above it, where we check the modification date of a top-level directory for example to see if a component needs to be re-built. This could also be useful for checking if we have artifacts for e.g. device trees or OpenSBI built for the correct platform, or to built them.

Obviously, the downside is that this adds an extra layer that, if it ever goes wrong, could produce major confusion if an artifact is not recompiled correctly. I think keeping it to the most basic implementation should make this fairly unlikely.

[bjorn3]

The mtime of a directory is only modified when files are added or removed from the directory. Looking at just the mtime of the top-level directory works for atomic replaces of direct child files (like most editors do), but not in-place modification (like echo foo > my_file) and more importantly doesn't work if the file that changed is inside a subdirectory.

[IsaacWoods]

Huh, interesting, thanks! Do you know of a sensible way to check for the correct behaviour (any change to files in a directory or any subdirectory) / is there no meaningful improvement to be made over Cargo's behaviour here?

[bjorn3]

Cargo has to resolve all dependencies and parse all relevant fingerprint files before it can do mtime checks. So doing a per-file mtime check inside the poplar build system similar to cargo before actually invoking cargo may still be faster. If that still isn't fast enough, you did probably have to start using persistent build system process with a file watcher.

[IsaacWoods]

Great, thanks for the info! I'll play around with this at some point but I guess will need to time builds to see if it's faster across normal development patterns for me.