publish: define how "historic" metadata and artifacts should work

[jku]

The way tuf-on-ci works right now is that only current data is published:

• only metadata versions that are part of the current delegation tree
• only targetpaths listed in current metadata delegation tree

The way GitHub Pages publishing handles this is that only the current data is made available (previous uploads are wiped). There's two potential issues with this that I can see:

1. the idea that repository content should stay available as long as timestamp was valid (that if you have non-expired metadata you should be able to use that to download an artifact, even if a newer version of metadata does not contain that artifact). I think this may be a logical flaw: if you can download the artifact, you absolutely should first download the current metadata
2. The short term race condition: if a downloader client is doing a metadata refresh just as the metadata gets updated by the repository, the client will currently get a 404 when using GitHub Pages. This is a real problem.

I'm not yet sure how I'd like to address this:

• on one hand it's clear that the metadata should not disappear from under the downloader client: this is essentially the API breaking
• on the other, I really dislike storing old, currently invalid, metadata in git: it's clutter that makes it harder to see the relevant data. I would really like to somehow make this a publisher problem, not a repository problem
• I suppose we could make the metadata/root_history approach work for all metadata.