Simplify tagging multiple repositories in a tree of externals

[rsdunlapiv]

If you have a hierarchy of external repositories, creating a tag across the full hierarchy is a tedious, error-prone process. For example, assume you have a top level umbrella repository pointing to several external repos, and each of those point to an additional level. Now you are working in your sandbox of repos and you are at a point where you want to tag the whole system. This means committing and pushing a tag to each repository involved and then updating all of the Externals.cfg files to point to these new tags. This is very time consuming and tedious.

We should explore whether this entire process can be automated or if the user can at least be assisted. An automated approach would actually create the tags in each repo and update Externals.cfg with the tags. If this is too hard (or the semantics are ambiguous) another approach might be to assist the user by generating a set of git commands and printing them to the screen. The user can then copy these commands, modify if needed, and apply them manually.

[billsacks]

This seems like a very useful feature. I should mention that, in the initial development of manage_externals, we very deliberately avoided having manage_externals coordinate any kind of committing or pushing, because we felt that was a can of worms for which it would be hard to implement completely robust solutions. That decision could definitely be revisited, though: I'm not sure if it was really based on any analysis of what it would take, or more just a general sense of "that would be hard; let's not go there".

[billsacks]

One challenge here is how to determine where to push the changes to. It is often the case that you want to push changes back to a different repository from the one you retrieved the code from, and your various externals may not be set up with the appropriate remotes. Another challenge related to pushing is that of authentication. I remember that Ben really struggled with this authentication issue.

I still think this is a good idea, but just wanted to raise some points that will require consideration.

[tclune]

We're a bit deeper into routinely using ME, and this issue is now becoming one of great frustration to our users. These are also related to another issue #130 that I opened yesterday.

One of our developers suggested that we (somehow) automate the generation of a config file that represents the cloned sandbox rather than the origin. E.g. for simple externals.cfg this would swap the GitHub URL with a path on the local cluster. Then, if there were a mechanism to recursively tag the hierarchy, the generated config could be (automatically?) updated to use those tags rather than force our developers to push the tags to GitHub. (And I'm just now realizing that the generation step could probably wait until the recursive tag step.)

I don't think the suggestion is impossible to implement, but parts would be tricky. One issue to face is whether the generated file is itself tracked under git. If not, then the clone is insufficient for the next user. And if it does, it means that ME would be editing source files, which is not generally advisable.

Interestingly this is one of the few issues where there is what I would consider a fundamental difference between ME and git submodules. In most scenarios there is a 1-1 correspondence with just differences in how integrated things are with git and how easily changeable they are by humans. But the very lack of separation in git submodules would really interfere with any attempt to treat the nested local sandbox as the full remote for another user.