External protocol for secret store adaptors

[mih]

This issues replaces and continues from #224. It combines the idea from #684 with the need for versatile credential store support.

Problem

datalad-next already provides a credential manager that supersedes the credential handling in datalad-core. However, it does not provide a proper credential abstraction that would allow for using arbitrary secret stores. Instead it places the secrets into whatever python-keyring is configured to use, and all non-secret credential properties into the git-config. This is neat, but if one doesn't like it, there is no way around it.

Potential solution

• Use the idea from External protocol specifications with shell based "driver" implementation #684 and
• Identify all operations on credentials that are relevant (key aspect of RF CredentialManager to have a formal backend class #224)
• Define a protocol to trigger such operations, where the actual execution is deferred to an external helper
• have a credential frontend command and a CredentialManager as a lean interface to drive (any number of) helpers that interface a a particular secret store
• Such helpers can be implemented in any language, be provided by 3rd-parties and need not be cross-platform, but can be tuned to particular environments (incl. specific institutional setups)

Challenges

The capabilities of secret stores vary a lot. We need to re-evaluate our requirements and the operations we really really need vs those a helper could perform. Few thoughts:

• the protocol very likely needs the concept of capability reporting (ie. what does a specific adapter provide in terms of operations)
• we need to be able to query for and retrieve a credential, we need not necessarily be able to set/store credentials (in all adapters)
• somehow credentials need to be identified, and how that is done varies by secret store implementation. Some use implicit identifiers, some explicit (the one in datalad-next uses an explicit credential name)
• query by credential property should be possible. Properties could be hostname, HTTP realm, or a credential name (which could be just a property rather than a full identifier)
• the protocol needs to handle multiple query hits
• candidate operations are (from RF CredentialManager to have a formal backend class #224)
  • query/get: retrieve one or more credentials that match a query (get existed, because a credential name was a proper identifier)
  • set: store/update a credential (update would require some kind of identifier handling to make sure that a particular credential is updated), when multiple "backends" are active simulataneously, we would need to be able to say which backend should be used.
  • list: this is equivalent to a query-for-all operation and may not be necessary to have a dedicated implementation. It is very helpful for maintenance and UI aspects (see what exists)