Plugins that help to pass credentials for S3 and GCS to remote cluster workers #438
Labels
provider/aws/ec2
Cluster provider for AWS EC2 Instances
provider/gcp/vm
Cluster provider for GCP Instances
question
Further information is requested
I didn't find a simple way to pass credentials to remote workers, such as S3 and GCS, while both are widely used to store data frames.
In this ticket's scope, I propose creating plugins that will help distribute the required keys to remote workers.
GCP credentials
GCP credentials file path is stored in
GOOGLE_APPLICATION_CREDENTIALS
env variable. The plugin has to create a remote file and pass an env variable with a proper path to workers.S3 credentials
Like GCP, we must update credential files and store them on each worker.
PR: #439
The text was updated successfully, but these errors were encountered: