add external platforms

[axlbonnet]

The goal is to allow to reference files from external storage platforms as input and output of executions.
This should be considered as a draft and I made some questionable choice that are open to discussion.

So to reference a external file as input, one should sent on POST /executions

{ "pipelineIdentifier":"somePipeline",
  "inputValues" : {
    "inputFile" : "anOtherPlatform:/path/to/file"  }  }

So just prefix the path with some platform identifier. The current format (/path/to/file) is of course still supported to reference a local file on the requested platform.

Then to output data on an external platform, there is a new optional field when creating an execution :

{ "pipelineIdentifier":"somePipeline",
  "inputValues" : {
    "inputFile" : "myPlatform:/path/to/file"  },
  "resultsLocation" : "anOtherPlatform:/path/to/folder"  }

Again, the field content has the same format. A local folder can also be specified. That's what we already do in VIP with a mandatory input parameter on every pipeline. The fact that it is optional implies no change for the platforms which do not want to let the users choose the output location.

It is up to the CARMIN platform to download and upload the files to the external platforms. To do so, the user can only reference external platforms that are supported. A Carmin platform exposes the external platforms it supports through GET /externalPlatforms

[ { "identifier" : "anOtherPlatform",
    "description" : "A nice storage platform that we support",
    "url" : "https://link.to.platform.homepage.org" },
  { "identifier" : "yetAnotherPlatform",
    "description" : "A less nice platform" } }

This declaration means the CARMIN platform can exchange data whith those platform, whether it is through a REST API, some SFTP link or any other possible way. But before using an external platform, an user should configure how to identify him on it. This is done with the method `PUT /user/externalKeys' :

{ "platformIdentifier":"anOtherPlatform",
  "APIKey" : "fsdfsdkjfsdljfsdlkjfl" }

(I could be wrong with the REST url, maybe it should be /user/externalKeys/{platformIdentifier} )

Waiting for your remarks :)

[glatard]

It looks very good to me, in particular since a platform can always not implement this feature and return {} on GET /externalplatforms. I am just wondering if transferring the API key is the right way to do this but I don't have a better idea yet... To reference the files, maybe using URLs with a carmin prefix would be useful.

[glatard]

Discussed at the July 9th meeting. The impact on method getResults and maybe others should be investigated. Won't be merged until there is an implementation (in VIP).