From 9c459ab78513ca36b9b6a6338f11775e706dfa95 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jan=20Kr=C3=BCger?= Date: Wed, 5 Jan 2022 08:17:56 +0100 Subject: [PATCH] Release 2.3.1 (#322) * fix : print possible configuration errors to stdout to be more specific that "an error occurs" * fix #222: print possible configuration errors to stdout * enhancement #281: add support for cloud site region (Openstack) * Fix/#275 - Integrated 'useHostname' config parameter (#280) * Adjusted ideConf Readme * Extended configuration by useHostname parameter, adjusted AnsibleHostsConfig writing if useHostname is enabled * Added useHostname config parameter to config schema, adjusted default parameters marked with "" * Fixed 'useHostname' instead of 'useHostnames' typo * Added description to 'useHostnames' parameter * enhancement #283: add option to overwrite service CIDR mask * enhancement #283: adjust changelog * enhancement #283: adjust documentation * Moved serviceCIDR validation from config to validator, added method description for validateServiceCIDR(string), Added string pattern description to CONFIGURATION_SCHEMA.md * bump version to 2.2.2 (main bugfix release) - readd missing 'break' in intentMode switch/case directive * fix/#217 : - adjust theia task - remove unused (unlinked) cloud9 task * #285: move CIDR mask validation into Configuration setter method * security/#291: change default ip range for opened ports from 0.0.0.0/0 to current * enhancement/#289 : - bump node version manager to 0.37.0 - bump prebuild theia-version to 1.8 * fix/#297: disable auto-upgrade as early as possible (#298) * fix/#297: disable auto-upgrade as early as possible * #297 add check for /var/lib/dpkg/lock * Move client2 module/#299 (#300) * Adjusted ideConf Readme * Moved client to providerModule * update ChangeLog.md * bump version from 2.2.2 to 2.3 * Determine quotas/#257 (#306) * Adjusted ideConf Readme * Moved Intents for Openstack to separate directory * Changed interface OSClient to correct implementation via OSClientV3 * ProviderTypes have not been tested and remained unused, check config independently only in -ch and -c case * Moved config validation to validator * Removed unnecessary duplicate block in checking instance and image resources, added documentation to ValidateIntent * Renamed ValidatorOpenstack method validate in validateCredentials to be more precise * Added OpenStack Implementation of ValidateIntent * Check quotas exceeded before launching worker instances in scaling up, Changed scale up method createWorkerInstances into createAdditionalWorkerInstances, Assign clusterId in runIntent() in StartUp in forehand, Removed help intentMode from runIntent(), since it is handled before, Added ValidateIntentOpenstack, Added todos * Created configureClusterInstances() including master and worker instance configuration to check quotas in forehand, changed list of map entries for instanceTypes to map, removed unnecessary 'prepare' value permanently, Fixed some issues in google cloud, aws and azure and other warnings * Fixed ValidateIntent checkQuotas() wrong parameters issue * Fixed issue with master and worker same providerType, enhanced logging * adjust ChangeLog.md bump jackson databind to 2.9.10.7 * adjust ChangeLog.md * Bump guava from 28.0-jre to 29.0-jre in /bibigrid-openstack Bumps [guava](https://github.com/google/guava) from 28.0-jre to 29.0-jre. - [Release notes](https://github.com/google/guava/releases) - [Commits](https://github.com/google/guava/commits) Signed-off-by: dependabot[bot] * Bump snakeyaml from 1.25 to 1.26 in /bibigrid-core Bumps [snakeyaml](https://bitbucket.org/asomov/snakeyaml) from 1.25 to 1.26. - [Commits](https://bitbucket.org/asomov/snakeyaml/branches/compare/snakeyaml-1.26..snakeyaml-1.25) --- updated-dependencies: - dependency-name: org.yaml:snakeyaml dependency-type: direct:production ... Signed-off-by: dependabot[bot] * Fix ide config check/#293 (#313) * Adjusted ideConf Readme * Remove ConfigurationFile.java which has been fully replaced by Configuration * Moved CommandLine interpretation to separate class. * Moved config validation into validate and create only, minor fixes * Removed deprecated slave instances configuration * Added missing loadConfiguration() method doc * Added opportunity to install IDE subsequently * Added cluster initialization in CreateCluster * Fixed rest api createCluster components * Minor enhancements * Moved loaded cluster to IdeIntent instead of loading in intent * Moved Yaml Interpretation into separate model, minor bug fixes * Enhanced YamlInterpreter logging * Moved isIpAdressFile() to YamlInterpreter * Added LoadIdeConfiguration Method in LoadClusterConfigurationIntent, minor code enhancements * Fix rebase merging errors * error handling when ide not installed, separate installSubsequently() method to install afterwards (not yet implemented) * Abort IDE start when configuration not loaded successfully * fix(BibigridInfoIdGetHandler):just added a space to avoid confusion * Fixes missing clusterMap loading in BibigridTerminateIdDeleteHandler (#316) * Added ansible ping check in shell script, Removed unnecessary warn logging (#319) * add workaround for buggy openstack4j identity api v3 handling (#321) * bump version to 2.3.1 Co-authored-by: tdilger <39946465+tdilger@users.noreply.github.com> Co-authored-by: tdilger Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dweinholz --- bibigrid-aws/pom.xml | 2 +- bibigrid-core/pom.xml | 4 +- .../bibigrid/core/intents/CreateCluster.java | 87 ++-- .../intents/CreateClusterEnvironment.java | 2 +- .../bibigrid/core/intents/CreateIntent.java | 34 +- .../bibigrid/core/intents/IdeIntent.java | 62 ++- .../LoadClusterConfigurationIntent.java | 52 ++- .../bibigrid/core/intents/PrepareIntent.java | 7 +- .../core/intents/ScaleWorkerIntent.java | 26 +- .../core/intents/TerminateIntent.java | 68 +-- .../bibigrid/core/model/AnsibleConfig.java | 85 +--- .../cebitec/bibigrid/core/model/Cluster.java | 22 +- .../bibigrid/core/model/Configuration.java | 35 +- .../bibigrid/core/model/ProviderModule.java | 3 +- .../bibigrid/core/model/YamlInterpreter.java | 71 ++++ .../bibigrid/core/util/ConfigurationFile.java | 94 ----- .../core/util/ShellScriptCreator.java | 6 + .../playbook/roles/additional/kubernetes | 1 + bibigrid-googlecloud/pom.xml | 4 +- bibigrid-light-rest-4j/pom.xml | 6 +- .../handler/BibigridCreatePostHandler.java | 11 +- .../handler/BibigridInfoIdGetHandler.java | 2 +- .../BibigridTerminateIdDeleteHandler.java | 12 +- bibigrid-main/pom.xml | 20 +- .../bibigrid/CommandLineInterpreter.java | 137 ++++++ .../de/unibi/cebitec/bibigrid/StartUp.java | 285 ++++++------- bibigrid-main/tests.log | 48 +++ bibigrid-openstack/pom.xml | 6 +- .../bibigrid/openstack/ClientOpenstack.java | 15 +- .../CreateClusterEnvironmentOpenstack.java | 8 +- .../openstack/CreateClusterOpenstack.java | 13 +- .../openstack/ProviderModuleOpenstack.java | 4 +- .../intents/TerminateIntentOpenstack.java | 9 +- pom.xml | 2 +- tests.log | 393 ++++++++++++++++++ 35 files changed, 1116 insertions(+), 520 deletions(-) create mode 100644 bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/model/YamlInterpreter.java delete mode 100644 bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/util/ConfigurationFile.java create mode 160000 bibigrid-core/src/main/resources/playbook/roles/additional/kubernetes create mode 100644 bibigrid-main/src/main/java/de/unibi/cebitec/bibigrid/CommandLineInterpreter.java create mode 100644 bibigrid-main/tests.log create mode 100644 tests.log diff --git a/bibigrid-aws/pom.xml b/bibigrid-aws/pom.xml index cd3dfefc3..bcc6a7a94 100644 --- a/bibigrid-aws/pom.xml +++ b/bibigrid-aws/pom.xml @@ -5,7 +5,7 @@ bibigrid de.unibi.cebitec.bibigrid - 2.3 + 2.3.1 4.0.0 diff --git a/bibigrid-core/pom.xml b/bibigrid-core/pom.xml index 6308b0443..23a813b9e 100644 --- a/bibigrid-core/pom.xml +++ b/bibigrid-core/pom.xml @@ -18,7 +18,7 @@ bibigrid de.unibi.cebitec.bibigrid - 2.3 + 2.3.1 bibigrid-core @@ -49,7 +49,7 @@ org.yaml snakeyaml - 1.25 + 1.26 diff --git a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/CreateCluster.java b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/CreateCluster.java index 01e9fb5e3..6784d9c6c 100644 --- a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/CreateCluster.java +++ b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/CreateCluster.java @@ -38,9 +38,7 @@ public abstract class CreateCluster extends Intent { protected final ProviderModule providerModule; protected final Configuration config; - protected final String clusterId; - protected Instance masterInstance; - protected List workerInstances; + protected Cluster cluster; protected CreateClusterEnvironment environment; protected DeviceMapper masterDeviceMapper; @@ -54,12 +52,13 @@ public abstract class CreateCluster extends Intent { * @param clusterId optional if cluster already started and has to be scaled */ protected CreateCluster(ProviderModule providerModule, Configuration config, String clusterId) { - this.clusterId = clusterId != null ? clusterId : generateClusterId(); + String cid = clusterId != null ? clusterId : generateClusterId(); + cluster = new Cluster(cid); this.providerModule = providerModule; this.config = config; this.interruptionMessageHook = new Thread(() -> LOG.error("Cluster setup was interrupted!\n\n" + - "Please clean up the remains using: -t {}\n\n", this.clusterId)); + "Please clean up the remains using: -t {}\n\n", cluster.getClusterId())); } /** @@ -78,10 +77,6 @@ static String generateClusterId() { return clusterIdBase64.substring(0, len).toLowerCase(Locale.US); } - public String getClusterId() { - return clusterId; - } - /** * The environment creation procedure. For a successful environment creation * you will need an Environment-Instance which implements the @@ -115,7 +110,6 @@ public CreateClusterEnvironment createClusterEnvironment() throws ConfigurationE public boolean configureClusterInstances() { Map instanceTypes = new HashMap<>(); instanceTypes.put(config.getMasterInstance().getProviderType(), 1); - LOG.warn("Put master instance type {} into quotas map.", config.getMasterInstance().getProviderType()); for (Configuration.WorkerInstanceConfiguration worker : config.getWorkerInstances()) { InstanceType providerType = worker.getProviderType(); if (instanceTypes.containsKey(providerType)) { @@ -123,7 +117,6 @@ public boolean configureClusterInstances() { } else { instanceTypes.put(worker.getProviderType(), worker.getCount()); } - LOG.warn("Put worker instance type {} into quotas map.", worker.getProviderType()); } if (providerModule.getValidateIntent(config). checkQuotasExceeded(instanceTypes)) { @@ -163,12 +156,12 @@ public void configureClusterWorkerInstance() { */ public boolean launchClusterInstances() { try { - String masterNameTag = MASTER_NAME_PREFIX + SEPARATOR + clusterId; - masterInstance = launchClusterMasterInstance(masterNameTag); + String masterNameTag = MASTER_NAME_PREFIX + SEPARATOR + cluster.getClusterId(); + Instance masterInstance = launchClusterMasterInstance(masterNameTag); if (masterInstance == null) { return false; } - workerInstances = new ArrayList<>(); + List workerInstances = new ArrayList<>(); int totalWorkerInstanceCount = config.getWorkerInstanceCount(); if (totalWorkerInstanceCount > 0) { LOG.info("Requesting {} worker instance(s) with {} different configurations...", @@ -178,7 +171,7 @@ public boolean launchClusterInstances() { Configuration.WorkerInstanceConfiguration instanceConfiguration = config.getWorkerInstances().get(i); LOG.info("Requesting {} worker instance(s) with same configuration...", instanceConfiguration.getCount()); - String workerNameTag = WORKER_NAME_PREFIX + SEPARATOR + clusterId; + String workerNameTag = WORKER_NAME_PREFIX + SEPARATOR + cluster.getClusterId(); List workersBatch = launchClusterWorkerInstances(i, instanceConfiguration, workerNameTag); if (workersBatch == null) { return false; @@ -188,9 +181,14 @@ public boolean launchClusterInstances() { } else { LOG.info("No Worker instance(s) requested!"); } + // set cluster initialization values + cluster.setMasterInstance(masterInstance); + cluster.setWorkerInstances(workerInstances); + cluster.setPublicIp(masterInstance.getPublicIp()); + cluster.setPrivateIp(masterInstance.getPrivateIp()); // just to be sure, everything is present, wait x seconds sleep(4); - LOG.info("Cluster (ID: {}) successfully created!", clusterId); + LOG.info("Cluster (ID: {}) successfully created!", cluster.getClusterId()); final String masterIp = config.isUseMasterWithPublicIp() ? masterInstance.getPublicIp() : masterInstance.getPrivateIp(); configureAnsible(); @@ -205,7 +203,8 @@ public boolean launchClusterInstances() { } if (Configuration.DEBUG) { logFinishedInfoMessage( - config.isUseMasterWithPublicIp() ? masterInstance.getPublicIp() : masterInstance.getPrivateIp()); + config.isUseMasterWithPublicIp() ? + cluster.getMasterInstance().getPublicIp() : cluster.getMasterInstance().getPrivateIp()); } Runtime.getRuntime().removeShutdownHook(this.interruptionMessageHook); @@ -216,17 +215,13 @@ public boolean launchClusterInstances() { } /** - * TODO Should not rely on config, since configurationFile might not be valid for current cluster, keypair + ssh user? * Adds additional worker instance(s) with specified batch to cluster. * Adopts the configuration from the other worker instances in batch * @return true, if worker instance(s) created successfully */ - public boolean createAdditionalWorkerInstances(int batchIndex, int count) { - LoadClusterConfigurationIntent loadIntent = providerModule.getLoadClusterConfigurationIntent(config); - loadIntent.loadClusterConfiguration(clusterId); - Cluster cluster = loadIntent.getCluster(clusterId); + public boolean createWorkerInstances(int batchIndex, int count) { if (cluster == null) { - LOG.error("No cluster with specified clusterId {} found", clusterId); + LOG.error("No cluster with specified clusterId {} found", cluster.getClusterId()); return false; } try { @@ -291,18 +286,20 @@ public boolean createAdditionalWorkerInstances(int batchIndex, int count) { } LOG.info("Creating {} worker " + (count == 1 ? "instance" : "instances") + " for batch {}.", count, batchIndex); instanceConfiguration.setCount(count); - String workerNameTag = WORKER_NAME_PREFIX + "-" + clusterId; + String workerNameTag = WORKER_NAME_PREFIX + "-" + cluster.getClusterId(); int workerIndex = workersBatch.size() + 1; List additionalWorkers = - launchAdditionalClusterWorkerInstances(cluster, batchIndex, workerIndex, instanceConfiguration, workerNameTag); + launchAdditionalClusterWorkerInstances(batchIndex, workerIndex, instanceConfiguration, workerNameTag); if (additionalWorkers == null) { LOG.error("No additional workers could be launched."); channelSftp.disconnect(); sshSession.disconnect(); return false; } else { - workerInstances = cluster.getWorkerInstances(); - workerInstances.addAll(additionalWorkers); + // loadIntent cluster ... + // workerInstances = cluster.getWorkerInstances(); + // workerInstances.addAll(additionalWorkers); + cluster.addWorkerInstances(additionalWorkers); } config.getClusterKeyPair().setName(CreateCluster.PREFIX + cluster.getClusterId()); config.getClusterKeyPair().load(); @@ -352,10 +349,11 @@ public boolean createAdditionalWorkerInstances(int batchIndex, int count) { * @return List of worker Instances */ protected abstract List launchAdditionalClusterWorkerInstances( - Cluster cluster, int batchIndex, int workerIndex, Configuration.WorkerInstanceConfiguration instanceConfiguration, String workerNameTag); + int batchIndex, int workerIndex, + Configuration.WorkerInstanceConfiguration instanceConfiguration, String workerNameTag); protected String buildWorkerInstanceName(int batchIndex, int workerIndex) { - return WORKER_NAME_PREFIX + SEPARATOR + (batchIndex) + SEPARATOR + (workerIndex) + "-" + clusterId; + return WORKER_NAME_PREFIX + SEPARATOR + (batchIndex) + SEPARATOR + (workerIndex) + "-" + cluster.getClusterId(); } private void logFinishedInfoMessage(final String masterPublicIp) { @@ -376,9 +374,9 @@ private void logFinishedInfoMessageWindows(final String masterPublicIp) { .append("putty -i ") .append(Configuration.KEYS_DIR).append("\\").append(config.getClusterKeyPair().getName()) .append(" ").append(config.getSshUser()).append("@%BIBIGRID_MASTER%\n\n"); - sb.append("The cluster id of your started cluster is: ").append(clusterId).append("\n\n"); + sb.append("The cluster id of your started cluster is: ").append(cluster.getClusterId()).append("\n\n"); sb.append("You can easily terminate the cluster at any time with:\n") - .append("./bibigrid -t ").append(clusterId).append(" "); + .append("./bibigrid -t ").append(cluster.getClusterId()).append(" "); if (config.isAlternativeConfigFile()) { sb.append("-o ").append(config.getAlternativeConfigPath()).append(" "); } @@ -394,9 +392,9 @@ private void logFinishedInfoMessageUnix(final String masterPublicIp) { .append("ssh -i ") .append(Configuration.KEYS_DIR).append("/").append(config.getClusterKeyPair().getName()) .append(" ").append(config.getSshUser()).append("@$BIBIGRID_MASTER\n\n"); - sb.append("The cluster id of your started cluster is: ").append(clusterId).append("\n\n"); + sb.append("The cluster id of your started cluster is: ").append(cluster.getClusterId()).append("\n\n"); sb.append("You can easily terminate the cluster at any time with:\n") - .append("./bibigrid -t ").append(clusterId).append(" "); + .append("./bibigrid -t ").append(cluster.getClusterId()).append(" "); if (config.isAlternativeConfigFile()) { sb.append("-o ").append(config.getAlternativeConfigPath()).append(" "); } @@ -410,7 +408,7 @@ private void saveGridPropertiesFile(String masterPublicIp) { gp.setProperty("BIBIGRID_MASTER", masterPublicIp); gp.setProperty("SSHPublicKeyFile", config.getSshPublicKeyFile()); gp.setProperty("SSHPrivateKeyFile", config.getSshPrivateKeyFile()); - gp.setProperty("clusterId", clusterId); + gp.setProperty("clusterId", cluster.getClusterId()); if (config.isAlternativeConfigFile()) { gp.setProperty("AlternativeConfigFile", config.getAlternativeConfigPath()); } @@ -430,8 +428,8 @@ private void saveGridPropertiesFile(String masterPublicIp) { * ConfigurationException, close the sshSession and throw a new ConfigurationException */ private void configureAnsible() throws ConfigurationException { - final String masterIp = config.isUseMasterWithPublicIp() ? masterInstance.getPublicIp() : - masterInstance.getPrivateIp(); + final String masterIp = config.isUseMasterWithPublicIp() ? cluster.getPublicIp() : + cluster.getPrivateIp(); LOG.info("Now configuring..."); boolean sshPortIsReady = SshFactory.pollSshPortIsAvailable(masterIp); if (sshPortIsReady) { @@ -565,7 +563,7 @@ private void uploadAnsibleToMaster(Session sshSession) throws JSchException, Con AnsibleConfig.writeSiteFile(channel.put(site_file), customMasterRoles, customWorkerRoles); - AnsibleConfig.writeHostsFile(channel, config.getSshUser(), workerInstances, config.useHostnames()); + AnsibleConfig.writeHostsFile(channel, config.getSshUser(), cluster.getWorkerInstances(), config.useHostnames()); // files for common configuration String login_file = channel_dir + AnsibleResources.COMMONS_LOGIN_FILE; @@ -581,7 +579,7 @@ private void uploadAnsibleToMaster(Session sshSession) throws JSchException, Con // write files using stream AnsibleConfig.writeLoginFile(login_stream, config); - AnsibleConfig.writeInstancesFile(instances_stream, masterInstance, workerInstances, masterDeviceMapper, providerModule.getBlockDeviceBase()); + AnsibleConfig.writeInstancesFile(instances_stream, cluster.getMasterInstance(), cluster.getWorkerInstances(), masterDeviceMapper, providerModule.getBlockDeviceBase()); AnsibleConfig.writeConfigFile(config_stream, config, environment.getSubnet().getCidr()); // TODO network should be written in instance configuration when initializing // security group und server group @@ -594,7 +592,7 @@ private void uploadAnsibleToMaster(Session sshSession) throws JSchException, Con } // Write worker instance specific configuration file - for (Instance worker : workerInstances) { + for (Instance worker : cluster.getWorkerInstances()) { String filename = channel_dir + AnsibleResources.CONFIG_ROOT_PATH + "/" + worker.getPrivateIp() + ".yml"; AnsibleConfig.writeSpecificInstanceFile(channel.put(filename), worker, providerModule.getBlockDeviceBase()); @@ -691,19 +689,16 @@ private String getSingleFileName(String roleFile) { return pathway[pathway.length - 1]; } - - public Configuration getConfig() { return config; } - public Instance getMasterInstance() { - return masterInstance; + public Cluster getCluster() { + return cluster; } - public List getWorkerInstances() { - return workerInstances; + public void setCluster(Cluster cluster) { + this.cluster = cluster; } - } diff --git a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/CreateClusterEnvironment.java b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/CreateClusterEnvironment.java index 579ac880c..0d48f5473 100644 --- a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/CreateClusterEnvironment.java +++ b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/CreateClusterEnvironment.java @@ -50,7 +50,7 @@ private void generateClusterKeyPair() throws ConfigurationException { bao = new ByteArrayOutputStream(); keypair.writePublicKey(bao, "generated by BiBiGrid"); clusterKeyPair.setPublicKey(bao.toString()); - clusterKeyPair.setName(CreateCluster.PREFIX+cluster.clusterId); + clusterKeyPair.setName(CreateCluster.PREFIX+cluster.cluster.getClusterId()); clusterKeyPair.store(); } catch (Exception ex) { if (Configuration.DEBUG) { diff --git a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/CreateIntent.java b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/CreateIntent.java index cc0edef46..ccc3e2e7b 100644 --- a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/CreateIntent.java +++ b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/CreateIntent.java @@ -2,7 +2,7 @@ import de.unibi.cebitec.bibigrid.core.Constant; import de.unibi.cebitec.bibigrid.core.DataBase; -import de.unibi.cebitec.bibigrid.core.model.Client; +import de.unibi.cebitec.bibigrid.core.model.Cluster; import de.unibi.cebitec.bibigrid.core.model.Configuration; import de.unibi.cebitec.bibigrid.core.model.ProviderModule; import de.unibi.cebitec.bibigrid.core.model.exceptions.ClientConnectionFailedException; @@ -13,6 +13,9 @@ import org.slf4j.LoggerFactory; import org.slf4j.MDC; +import java.util.HashMap; +import java.util.Map; + /** * CreateIntent for starting a cluster. @@ -34,43 +37,48 @@ public CreateIntent(ProviderModule module, Configuration config) { this.config = config; String clusterId = null; cluster = module.getCreateIntent(config, clusterId); - db.status.put(cluster.clusterId,new Status(Status.CODE.Preparing)); + db.status.put(cluster.getCluster().getClusterId(), new Status(Status.CODE.Preparing)); } public String getClusterId(){ - return cluster.clusterId; + return cluster.getCluster().getClusterId(); } public void create(){ try { // configure environment - db.status.put(cluster.clusterId, new Status(Status.CODE.Configuring, "Configuring environment.")); + db.status.put(cluster.getCluster().getClusterId(), new Status(Status.CODE.Configuring, "Configuring environment.")); + cluster.createClusterEnvironment() .createNetwork() .createSubnet() .createSecurityGroup() .createKeyPair() .createPlacementGroup(); - db.status.put(cluster.clusterId, new Status(Status.CODE.Configuring, "Configuring instances.")); + db.status.put(cluster.getCluster().getClusterId(), new Status(Status.CODE.Configuring, "Configuring instances.")); boolean success = cluster.configureClusterInstances(); - db.status.put(cluster.clusterId, new Status(Status.CODE.Creating)); + db.status.put(cluster.getCluster().getClusterId(), new Status(Status.CODE.Creating)); + // configure cluster success = success && cluster.launchClusterInstances(); if (success) { - db.status.put(cluster.clusterId, new Status(Status.CODE.Running)); + db.status.put(cluster.getCluster().getClusterId(), new Status(Status.CODE.Running)); } else { /* In DEBUG mode keep partial configured cluster running, otherwise clean it up */ if (Configuration.DEBUG) { - db.status.put(cluster.clusterId, new Status(Status.CODE.Error, Constant.KEEP)); + db.status.put(cluster.getCluster().getClusterId(), new Status(Status.CODE.Error, Constant.KEEP)); } else { - db.status.put(cluster.clusterId, new Status(Status.CODE.Error, Constant.ABORT_WITH_INSTANCES_RUNNING)); - TerminateIntent cleanupIntent = module.getTerminateIntent(config); - cleanupIntent.terminate(cluster.clusterId); + db.status.put(cluster.getCluster().getClusterId(), new Status(Status.CODE.Error, Constant.ABORT_WITH_INSTANCES_RUNNING)); + Map clusterMap = new HashMap<>(); + clusterMap.put(cluster.getCluster().getClusterId(), cluster.getCluster()); + TerminateIntent cleanupIntent = module.getTerminateIntent(config, clusterMap); + cleanupIntent.terminate(cluster.getCluster().getClusterId()); } } } catch (ConfigurationException ex) { - db.status.put(cluster.clusterId,new Status(Status.CODE.Error,"Failed to create cluster. "+ex.getMessage())); + db.status.put(cluster.getCluster().getClusterId(),new Status(Status.CODE.Error,"Failed to create cluster. "+ex.getMessage())); + // print stacktrace only in verbose mode if (VerboseOutputFilter.SHOW_VERBOSE) { LOG.error("Failed to create cluster. {} {}", ex.getMessage(), ex); @@ -88,7 +96,7 @@ public void run() { module.client.authenticate(); create(); } catch (ClientConnectionFailedException ex) { - db.status.put(cluster.clusterId,new Status(Status.CODE.Error,"Client connection failed. "+ex.getMessage())); + db.status.put(cluster.getCluster().getClusterId(),new Status(Status.CODE.Error,"Client connection failed. "+ex.getMessage())); if (VerboseOutputFilter.SHOW_VERBOSE) { LOG.error("Client connection failed. {} {}", ex.getMessage(), ex); } diff --git a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/IdeIntent.java b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/IdeIntent.java index e07c16f28..14658980c 100644 --- a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/IdeIntent.java +++ b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/IdeIntent.java @@ -1,13 +1,10 @@ package de.unibi.cebitec.bibigrid.core.intents; -import com.jcraft.jsch.JSch; import com.jcraft.jsch.JSchException; import com.jcraft.jsch.Session; -import de.unibi.cebitec.bibigrid.core.model.Client; import de.unibi.cebitec.bibigrid.core.model.Cluster; import de.unibi.cebitec.bibigrid.core.model.Configuration; import de.unibi.cebitec.bibigrid.core.model.ProviderModule; -import de.unibi.cebitec.bibigrid.core.util.JSchLogger; import de.unibi.cebitec.bibigrid.core.util.SshFactory; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -18,8 +15,7 @@ import java.net.InetSocketAddress; import java.net.ServerSocket; import java.net.URL; -import java.nio.file.Paths; -import java.util.Map; +import java.util.Scanner; /** * Intent for starting and tunneling the Web-IDE installation (theia-ide or cloud9) on a cluster. @@ -32,30 +28,22 @@ public class IdeIntent extends Intent { public static final int DEFAULT_IDE_PORT = 8181; public static final int DEFAULT_IDE_PORT_END = 8383; - private final ProviderModule providerModule; - private final String clusterId; + private final Cluster cluster; private final Configuration config; private int idePort = DEFAULT_IDE_PORT; private int idePortLast = DEFAULT_IDE_PORT_END; - public IdeIntent(ProviderModule providerModule, String clusterId, Configuration config) { - this.providerModule = providerModule; - this.clusterId = clusterId; + public IdeIntent(Cluster cluster, Configuration config) { + this.cluster = cluster; this.config = config; } public void start() { - if (clusterId == null) { + if (cluster == null || cluster.getClusterId() == null) { LOG.error("ClusterId not found. Please provide a valid cluster id."); return; } - LoadClusterConfigurationIntent loadIntent = providerModule.getLoadClusterConfigurationIntent(config); - loadIntent.loadClusterConfiguration(clusterId); - Cluster cluster = loadIntent.getCluster(clusterId); - if (cluster == null) { - return; - } String masterIp = config.isUseMasterWithPublicIp() ? cluster.getPublicIp() : cluster.getPrivateIp(); boolean sshPortIsReady = SshFactory.pollSshPortIsAvailable(masterIp); @@ -63,13 +51,47 @@ public void start() { LOG.error("Failed to poll master ssh port."); return; } - if (!config.isIDE()) { - LOG.error("IDE not set in configuration."); - return; + + // TODO Write in config when loaded from remote + boolean ideEnabled = config.isIDE(); + + if (!ideEnabled) { + // Ask to install WebIDE subsequently + if (!installSubsequently()) return; } startPortForwarding(masterIp); } + /** + * Asks to install IDE subsequently, if not yet enabled. + * Installs afterwards. + * @return true, if successfully installed subsequently + * TODO implement installation afterwards + */ + public static boolean installSubsequently() { + boolean install_successful; + String install_ide = ""; + String INSTALL = "yes"; + String NOT_INSTALL = "no"; + while (!(install_ide.equals(INSTALL) || install_ide.equals(NOT_INSTALL))) { + LOG.info("IDE not set in configuration. Should it be installed subsequently? [yes, NO]"); + Scanner in = new Scanner(System.in); + install_ide = in.nextLine(); + if (install_ide.equals("")) { + install_ide = NOT_INSTALL; + } + } + if (!install_ide.equals(INSTALL)) { + LOG.error("IDE cannot be started. Aborting ..."); + install_successful = false; + } else { + // TODO Install IDE subsequently + LOG.error("Install IDE subsequently not implemented, yet."); + install_successful = false; + } + return install_successful; + } + /** * To use the IDE in a browser locally, ports have to be forwarded to connect to the remote. * @param masterIp IP of the master instance diff --git a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/LoadClusterConfigurationIntent.java b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/LoadClusterConfigurationIntent.java index 42df0c115..21e7505ac 100644 --- a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/LoadClusterConfigurationIntent.java +++ b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/LoadClusterConfigurationIntent.java @@ -1,9 +1,16 @@ package de.unibi.cebitec.bibigrid.core.intents; +import com.jcraft.jsch.ChannelSftp; +import com.jcraft.jsch.JSchException; +import com.jcraft.jsch.Session; +import com.jcraft.jsch.SftpException; import de.unibi.cebitec.bibigrid.core.model.*; +import de.unibi.cebitec.bibigrid.core.util.AnsibleResources; +import de.unibi.cebitec.bibigrid.core.util.SshFactory; import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import java.io.InputStream; import java.time.format.DateTimeFormatter; import java.util.*; @@ -66,10 +73,53 @@ private void loadSingleClusterConfiguration(List clusterInstances, Str LOG.info(V, "Configuration for instance {} loaded successfully.", instance.getName()); } LOG.info(V, "Initialize cluster with id {} ...", clusterId); - initCluster(clusterId, clusterInstances); + this.initCluster(clusterId, clusterInstances); LOG.info("Cluster with id {} initialized successfully.\n", clusterId); } + /** + * Loads IdeConf from remote common_configuration.yml file into current configuration. + * @return true, if IDE configuration loaded successfully and IDE can be started + * TODO Replace strings in this method and AnsibleConfig -> getIdeConfMap() + */ + public boolean loadIdeConfiguration(String clusterIP) { + LOG.info("Loading IdeConfiguration from master instance ..."); + Configuration.IdeConf ideConf = new Configuration.IdeConf(); + try { + Session sshSession = SshFactory.createSshSession( + config.getSshUser(), + config.getClusterKeyPair(), + clusterIP); + sshSession.connect(); + ChannelSftp channel = (ChannelSftp) sshSession.openChannel("sftp"); + channel.connect(); + String common_config_file = channel.getHome() + "/" + AnsibleResources.COMMONS_CONFIG_FILE; + InputStream in = channel.get(common_config_file); + Map configMap = YamlInterpreter.readFromInputStream(in); + Map ideConfigMap = (Map) configMap.get("ideConf"); + if (!configMap.containsKey("ideConf") || ideConfigMap == null) { + LOG.error("ideConf not set in cluster configuration."); + if (!IdeIntent.installSubsequently()) { + channel.disconnect(); + sshSession.disconnect(); + return false; + } + } + ideConf.setIde((Boolean) ideConfigMap.get("ide")); + ideConf.setWorkspace((String) ideConfigMap.get("workspace")); + ideConf.setPort_start((Integer) ideConfigMap.get("port_start")); + ideConf.setPort_end((Integer) ideConfigMap.get("port_end")); + ideConf.setBuild((Boolean) ideConfigMap.get("build")); + config.setIdeConf(ideConf); + LOG.info("IdeConfiguration loaded successfully."); + return true; + } catch (JSchException | SftpException e) { + LOG.error("Could not load IdeConfiguration successfully."); + e.printStackTrace(); + return false; + } + } + /** * Initializes map of clusterIds and corresponding instances. * @return instanceMap diff --git a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/PrepareIntent.java b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/PrepareIntent.java index 8a6dbb7db..ef29b77bd 100644 --- a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/PrepareIntent.java +++ b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/PrepareIntent.java @@ -23,13 +23,14 @@ protected PrepareIntent(ProviderModule providerModule, Client client, Configurat /** * Prepare cluster images for the cluster with id in configuration. + * TODO No usage? * * @return Return true in case of success, false otherwise */ public boolean prepare(final CreateCluster cluster) { - Instance masterInstance = cluster.getMasterInstance(); - List workerInstances = cluster.getWorkerInstances(); - String clusterId = cluster.getClusterId(); + Instance masterInstance = cluster.cluster.getMasterInstance(); + List workerInstances = cluster.cluster.getWorkerInstances(); + String clusterId = cluster.cluster.getClusterId(); LOG.info("Stopping {} instances...", workerInstances.size()); stopInstance(masterInstance); for (Instance instance : workerInstances) { diff --git a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/ScaleWorkerIntent.java b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/ScaleWorkerIntent.java index 8cd280493..22477fd0d 100644 --- a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/ScaleWorkerIntent.java +++ b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/ScaleWorkerIntent.java @@ -1,6 +1,7 @@ package de.unibi.cebitec.bibigrid.core.intents; import de.unibi.cebitec.bibigrid.core.DataBase; +import de.unibi.cebitec.bibigrid.core.model.Cluster; import de.unibi.cebitec.bibigrid.core.model.Configuration; import de.unibi.cebitec.bibigrid.core.model.ProviderModule; import de.unibi.cebitec.bibigrid.core.model.exceptions.ClientConnectionFailedException; @@ -11,13 +12,16 @@ import org.slf4j.LoggerFactory; import org.slf4j.MDC; +import java.util.HashMap; +import java.util.Map; + import static de.unibi.cebitec.bibigrid.core.model.IntentMode.SCALE_DOWN; import static de.unibi.cebitec.bibigrid.core.model.IntentMode.SCALE_UP; public class ScaleWorkerIntent implements Runnable { private static final Logger LOG = LoggerFactory.getLogger(CreateIntent.class); - private final CreateCluster cluster; + private final CreateCluster createCluster; private final int batchIndex; private final int count; private final ProviderModule module; @@ -32,26 +36,28 @@ public ScaleWorkerIntent(ProviderModule module, Configuration config, String clu this.count = count; this.config = config; this.scaling = scaling; - cluster = module.getCreateIntent(config, clusterId); + createCluster = module.getCreateIntent(config, clusterId); } public String getClusterId() { - return cluster.clusterId; + return createCluster.cluster.getClusterId(); } private void scaleDown() { - db.status.put(cluster.clusterId, new Status(Status.CODE.Scale_Down, "Scaling down the cluster by " + count + " worker!")); - module.getTerminateIntent(config) - .terminateInstances(cluster.clusterId, batchIndex, count); - db.status.put(cluster.clusterId, new Status(Status.CODE.Running)); + db.status.put(createCluster.cluster.getClusterId(), new Status(Status.CODE.Scale_Down, "Scaling down the cluster by " + count + " worker!")); + Map clusterMap = new HashMap<>(); + clusterMap.put(createCluster.cluster.getClusterId(), createCluster.cluster); + module.getTerminateIntent(config, clusterMap) + .terminateInstances(createCluster.cluster.getClusterId(), batchIndex, count); + db.status.put(createCluster.cluster.getClusterId(), new Status(Status.CODE.Running)); } private void scaleUp() { - db.status.put(cluster.clusterId, new Status(Status.CODE.Scale_Up, "Scaling up the cluster by " + count + " worker!")); - cluster.createAdditionalWorkerInstances(batchIndex, count); - db.status.put(cluster.clusterId, new Status(Status.CODE.Running)); + db.status.put(createCluster.cluster.getClusterId(), new Status(Status.CODE.Scale_Up, "Scaling up the cluster by " + count + " worker!")); + createCluster.createWorkerInstances(batchIndex, count); + db.status.put(createCluster.cluster.getClusterId(), new Status(Status.CODE.Running)); } diff --git a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/TerminateIntent.java b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/TerminateIntent.java index f1854b729..09a2aa76d 100644 --- a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/TerminateIntent.java +++ b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/intents/TerminateIntent.java @@ -26,7 +26,8 @@ public abstract class TerminateIntent extends Intent { private static final Logger LOG = LoggerFactory.getLogger(TerminateIntent.class); private final ProviderModule providerModule; protected final Client client; - private final Configuration config; + final Configuration config; + final Map clusterMap; /* The terminateResponse attribute is used to save the response of the terminate intent and make it accessible to other @@ -35,10 +36,14 @@ public abstract class TerminateIntent extends Intent { */ private String terminateResponse = "Internal server error!"; - protected TerminateIntent(ProviderModule providerModule, Client client, Configuration config) { + protected TerminateIntent(ProviderModule providerModule, + Client client, + Configuration config, + Map clusterMap) { this.providerModule = providerModule; this.client = client; this.config = config; + this.clusterMap = clusterMap; } public String getTerminateResponse() { @@ -58,17 +63,30 @@ public boolean terminate(String[] parameters) { return true; } + /** + * Terminates specified cluster. + * @param cluster cluster to be terminated + * @return true, if cluster termination successful. + */ + public boolean terminate(Cluster cluster) { + String clusterId = cluster.getClusterId(); + if (terminateCluster(cluster)) { + deleteClusterKeyPair(cluster); + terminateResponse = "Cluster '" + clusterId + "' terminated!"; + LOG.info(I, "Cluster '{}' terminated!", clusterId); + return true; + } else { + terminateResponse = "Failed to terminate cluster '" + clusterId + "'!"; + LOG.error("Cluster '{}' could not be terminated successfully.", clusterId); + return false; + } + } + /** * Terminates the clusters with the specified id or specified user. * @param parameter user-id or cluster-id */ public boolean terminate(String parameter) { - LoadClusterConfigurationIntent loadIntent = providerModule.getLoadClusterConfigurationIntent(config); - loadIntent.loadClusterConfiguration(parameter); - final Map clusterMap = loadIntent.getClusterMap(); - if (clusterMap.isEmpty()) { - return false; - } List toRemove = new ArrayList<>(); if (clusterMap.containsKey(parameter)) { Cluster provided = clusterMap.get(parameter); @@ -89,19 +107,13 @@ public boolean terminate(String parameter) { return false; } + boolean success = true; for (Cluster cluster: toRemove) { String clusterId = cluster.getClusterId(); LOG.info("Terminating cluster with ID '{}' ...", clusterId); - if (terminateCluster(cluster)) { - delete_Key(cluster); - terminateResponse = "Cluster '" + clusterId + "' terminated!"; - LOG.info(I, "Cluster '{}' terminated!", clusterId); - } else { - terminateResponse = "Failed to terminate cluster '" + clusterId + "'!"; - LOG.error("Cluster '{}' could not be terminated successfully.", clusterId); - } + if (!terminateCluster(cluster)) success = false; } - return true; + return success; } /** @@ -120,8 +132,8 @@ public void terminateInstances(String clusterId, int workerBatch, int count) { List workers = cluster.getWorkerInstances(workerBatch); if (workers.isEmpty() || workers.size() < count) { if (count == 1) { - LOG.error("Could not terminate {} worker with specified workerBatch in cluster.\n" - + "There is currently {} worker node running with workerBatch {}.", count, workers.size(), workerBatch); + LOG.error("Could not terminate worker with specified workerBatch in cluster.\n" + + "There is currently no worker node running with workerBatch {}.", workerBatch); } else { LOG.error("Could not terminate {} workers with specified workerBatch in cluster.\n" + "There are currently {} worker nodes running with workerBatch {}.", count, workers.size(), workerBatch); @@ -195,12 +207,18 @@ public void terminateInstances(String clusterId, int workerBatch, int count) { */ protected abstract boolean terminateCluster(Cluster cluster); - private void delete_Key(Cluster cluster) { + /** + * Deletes Private and public key automatically generated by BiBiGrid in key dir. + * @param cluster deleted cluster + */ + private void deleteClusterKeyPair(Cluster cluster) { + boolean success = true; + Path keyPath = Paths.get(Configuration.KEYS_DIR + System.getProperty("file.separator") + cluster.getKeyName());; try { - Path p = Paths.get(Configuration.KEYS_DIR + System.getProperty("file.separator") + cluster.getKeyName()); - Files.delete(p); - LOG.info("Private key {} deleted.",p.toString()); + Files.delete(keyPath); + LOG.info("Private key {} deleted.", keyPath.toString()); } catch (IOException e) { + success = false; // ignore exception } @@ -209,8 +227,10 @@ private void delete_Key(Cluster cluster) { Files.delete(p); LOG.info("Public key {} deleted.",p.toString()); } catch (IOException e) { + success = false; // ignore exception } - + if (!success) + LOG.error("An error occurred during key deletion for cluster {}. Please remove the remains in {} directory.", cluster.getClusterId(), keyPath); } } diff --git a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/model/AnsibleConfig.java b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/model/AnsibleConfig.java index 9c9fdcd71..86fe812fc 100644 --- a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/model/AnsibleConfig.java +++ b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/model/AnsibleConfig.java @@ -7,15 +7,10 @@ import de.unibi.cebitec.bibigrid.core.util.DeviceMapper; import org.slf4j.Logger; import org.slf4j.LoggerFactory; -import org.yaml.snakeyaml.DumperOptions; -import org.yaml.snakeyaml.Yaml; -import org.yaml.snakeyaml.nodes.Tag; import java.io.*; import java.nio.charset.StandardCharsets; import java.util.*; -import java.util.regex.Matcher; -import java.util.regex.Pattern; /** * Wrapper for the {@link Configuration} class with extra fields. @@ -114,7 +109,7 @@ public static void writeSiteFile(OutputStream stream, roles.add("additional/" + role_name); } workers.put("roles", roles); - writeToOutputStream(stream, Arrays.asList(master, workers)); + YamlInterpreter.writeToOutputStream(stream, Arrays.asList(master, workers)); } /** @@ -123,7 +118,7 @@ public static void writeSiteFile(OutputStream stream, */ public static void writeAnsibleVarsFile(OutputStream stream, Map vars) { if (vars != null && !vars.isEmpty()) { - writeToOutputStream(stream, vars); + YamlInterpreter.writeToOutputStream(stream, vars); } } @@ -154,14 +149,14 @@ public static void writeRequirementsFile(OutputStream stream, List map = AnsibleConfig.readFromInputStream(in); + Map map = YamlInterpreter.readFromInputStream(in); for (Object val : map.values()) { Map batchMap = (Map) val; int index = Integer.parseInt(String.valueOf(batchMap.get(WorkerSpecification.INDEX.name()))); @@ -310,7 +305,7 @@ private static void updateSpecificInstanceFiles ( Vector vars_files = channel.ls("*.yml"); for(Object file : vars_files) { String filename = ((ChannelSftp.LsEntry) file).getFilename(); - if (isIPAddressFile(filename)) { + if (YamlInterpreter.isIPAddressFile(filename)) { ip_files.add(filename); } } @@ -324,23 +319,6 @@ private static void updateSpecificInstanceFiles ( } } - /** - * Checks with RegEx if ip could be valid address. - * @param file name of file to be checked - * @return true, if file contains valid ipv4-address - */ - private static boolean isIPAddressFile(String file) { - String ip = file.replace(".yml", ""); - final String IPV4_REGEX = - "^(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\." + - "(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\." + - "(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\." + - "(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$"; - final Pattern IPV4_PATTERN = Pattern.compile(IPV4_REGEX); - Matcher matcher = IPV4_PATTERN.matcher(ip); - return matcher.matches(); - } - /** * Loads instances.yml file from remote and adds or removes workers. * @param channel sftp channel to exchange files @@ -355,11 +333,11 @@ private static void rewriteInstancesFile( String blockDeviceBase) throws SftpException { String instances_file = channel.getHome() + "/" + AnsibleResources.COMMONS_INSTANCES_FILE; InputStream in = channel.get(instances_file); - Map map = readFromInputStream(in); + Map map = YamlInterpreter.readFromInputStream(in); map.replace("workers", getWorkerMap(workerInstances, blockDeviceBase)); - map.replace("deletedWorkers",getWorkerMap(deletedInstances,blockDeviceBase)); + map.replace("deletedWorkers",getWorkerMap(deletedInstances, blockDeviceBase)); OutputStream out = channel.put(instances_file); - writeToOutputStream(out, map); + YamlInterpreter.writeToOutputStream(out, map); } /** @@ -401,44 +379,7 @@ public static void writeConfigFile(OutputStream stream, Configuration config, St if (config.hasCustomAnsibleGalaxyRoles()) { map.put("ansible_galaxy_roles", getAnsibleGalaxyRoles(config.getAnsibleGalaxyRoles())); } - writeToOutputStream(stream, map); - } - - /** - * Uses stream to write map on remote. - * @param stream OutputStream to remote instance - * @param map (yml) file content - */ - private static void writeToOutputStream(OutputStream stream, Object map) { - try (OutputStreamWriter writer = new OutputStreamWriter(stream, StandardCharsets.UTF_8)) { - if (map instanceof Map) { - writer.write(new Yaml().dumpAsMap(map)); - } else { - writer.write(new Yaml().dumpAs(map, Tag.SEQ, DumperOptions.FlowStyle.BLOCK)); - } - } catch (IOException e) { - e.printStackTrace(); - } - } - - /** - * Parses yaml file from remote into map. - * @param stream InputStream - * @return map of Yaml syntax - */ - private static Map readFromInputStream(InputStream stream) { - StringBuilder yamlContent = new StringBuilder(); - try (Reader reader = new BufferedReader(new InputStreamReader(stream))) { - int c; - while ((c = reader.read()) != -1) { - yamlContent.append((char) c); - } - } catch (IOException e) { - LOG.error("Could not read instances file from remote."); - e.printStackTrace(); - } - Yaml yaml = new Yaml(); - return yaml.load(yamlContent.toString()); + YamlInterpreter.writeToOutputStream(stream, map); } private static void addBooleanOption(Map map, String option, boolean value) { diff --git a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/model/Cluster.java b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/model/Cluster.java index 9f0d616ba..d1acf04ef 100644 --- a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/model/Cluster.java +++ b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/model/Cluster.java @@ -102,14 +102,32 @@ public List getWorkerInstances(int batchIndex) { return workers; } - public void setWorkerInstances(List workerInstances) { - this.workerInstances = workerInstances; + public void setWorkerInstances(List instances) { + this.workerInstances = instances; } + /** + * Todo doc + * @param instances + */ + public void addWorkerInstances(List instances) { + for (Instance instance : instances) { + this.addWorkerInstance(instance); + } + } + + /** + * Todo doc + * @param instance + */ public void addWorkerInstance(Instance instance) { workerInstances.add(instance); } + /** + * Todo doc + * @param instance + */ public void removeWorkerInstance(Instance instance) { workerInstances.remove(instance); } diff --git a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/model/Configuration.java b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/model/Configuration.java index a31bed327..968cf5bfb 100644 --- a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/model/Configuration.java +++ b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/model/Configuration.java @@ -1,7 +1,6 @@ package de.unibi.cebitec.bibigrid.core.model; -import com.fasterxml.jackson.databind.annotation.JsonDeserialize; import de.unibi.cebitec.bibigrid.core.intents.IdeIntent; import de.unibi.cebitec.bibigrid.core.model.exceptions.ConfigurationException; import org.slf4j.Logger; @@ -10,7 +9,6 @@ import org.yaml.snakeyaml.error.YAMLException; import java.io.*; -import java.nio.charset.Charset; import java.nio.charset.StandardCharsets; import java.nio.file.*; import java.nio.file.attribute.FileAttribute; @@ -19,11 +17,10 @@ import java.security.MessageDigest; import java.security.NoSuchAlgorithmException; import java.util.*; -import java.util.regex.Pattern; import static de.unibi.cebitec.bibigrid.core.util.VerboseOutputFilter.V; -@SuppressWarnings({"WeakerAccess", "unused"}) +@SuppressWarnings({"WeakerAccess"}) public abstract class Configuration { /* public const */ public static boolean DEBUG = false; @@ -53,6 +50,13 @@ public Configuration() throws IOException { } } + /** + * Loads configuration from config yaml file. + * @param configurationClass provider dependent configuration class + * @param path path to config file + * @return loaded configuration + * @throws ConfigurationException error in config + */ public static Configuration loadConfiguration(Class configurationClass, String path) throws ConfigurationException{ Path propertiesFilePath = null; @@ -83,11 +87,9 @@ public static Configuration loadConfiguration(Class con } catch (YAMLException e) { throw new ConfigurationException("Failed to parse configuration file. "+e.getMessage(), e); } - } /* properties */ -// protected static final Logger LOG = LoggerFactory.getLogger(Configuration.class); // private static final String DEFAULT_WORKSPACE = "$HOME"; private String mode; private String user = System.getProperty("user.name"); @@ -98,7 +100,7 @@ public static Configuration loadConfiguration(Class con private List sshPublicKeyFiles = new ArrayList<>(); private List sshPublicKeys = new ArrayList<>(); private String id; - private ClusterKeyPair clusterKeyPair = new ClusterKeyPair(); + private final ClusterKeyPair clusterKeyPair = new ClusterKeyPair(); @Deprecated private String sshPrivateKeyFile; private String alternativeConfigPath; @@ -122,11 +124,12 @@ public static Configuration loadConfiguration(Class con private boolean ganglia; private boolean zabbix; private ZabbixConf zabbixConf = new ZabbixConf(); - private List nfsShares = new ArrayList<>(Arrays.asList("/vol/spool")); + private List nfsShares = new ArrayList<>(Collections.singletonList("/vol/spool")); private List masterMounts = new ArrayList<>(); private List extNfsShares = new ArrayList<>(); private FS localFS = FS.XFS; private boolean debugRequests; + @Deprecated private Properties ogeConf = OgeConf.initOgeConfProperties(); private List ansibleRoles = new ArrayList<>(); private List ansibleGalaxyRoles = new ArrayList<>(); @@ -191,8 +194,6 @@ public ClusterKeyPair getClusterKeyPair() { return clusterKeyPair; } - - public String getSshPublicKeyFile() { return sshPublicKeyFile; } @@ -253,19 +254,6 @@ public void setMasterInstance(InstanceConfiguration masterInstance) { } } - @Deprecated - public List getSlaveInstances() { - LOG.warn("Property 'slaveInstances' is deprecated and will be removed in next major release. It is replaced 1:1 by 'workerInstances'."); - return getWorkerInstances(); - } - - @Deprecated - public void setSlaveInstances(List workerInstances) { - LOG.warn("Property 'slaveInstances' is deprecated and will be removed in next major release. It is replaced 1:1 by 'workerInstances'."); - setWorkerInstances(workerInstances); - } - - public List getWorkerInstances() { return workerInstances; } @@ -800,6 +788,7 @@ public void setOgeConf(Properties ogeConf) { /** * Provides support for GridEngine global configuration. */ + @Deprecated public static class OgeConf extends Properties { public static final String GRIDENGINE_FILES = "/playbook/roles/master/files/gridengine/"; public static final String GLOBAL_OGE_CONF = GRIDENGINE_FILES + "global.conf"; diff --git a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/model/ProviderModule.java b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/model/ProviderModule.java index 1dd5580d3..160f90ba0 100644 --- a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/model/ProviderModule.java +++ b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/model/ProviderModule.java @@ -29,7 +29,6 @@ public abstract class ProviderModule { */ public Client client; - /** * Get the configuration implementation for the specified provider, that have provider specific parameters. * @@ -59,7 +58,7 @@ public Client getClient() { public abstract ListIntent getListIntent(Map clusterMap); - public abstract TerminateIntent getTerminateIntent(Configuration config); + public abstract TerminateIntent getTerminateIntent(Configuration config, Map clusterMap); public abstract PrepareIntent getPrepareIntent(Configuration config); diff --git a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/model/YamlInterpreter.java b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/model/YamlInterpreter.java new file mode 100644 index 000000000..950100538 --- /dev/null +++ b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/model/YamlInterpreter.java @@ -0,0 +1,71 @@ +package de.unibi.cebitec.bibigrid.core.model; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.yaml.snakeyaml.DumperOptions; +import org.yaml.snakeyaml.Yaml; +import org.yaml.snakeyaml.nodes.Tag; + +import java.io.*; +import java.nio.charset.StandardCharsets; +import java.util.Map; +import java.util.regex.Matcher; +import java.util.regex.Pattern; + +public class YamlInterpreter { + private static final Logger LOG = LoggerFactory.getLogger(YamlInterpreter.class); + /** + * Uses stream to write map on remote. + * @param stream OutputStream to remote instance + * @param map (yml) file content + */ + public static void writeToOutputStream(OutputStream stream, Object map) { + try (OutputStreamWriter writer = new OutputStreamWriter(stream, StandardCharsets.UTF_8)) { + if (map instanceof Map) { + writer.write(new Yaml().dumpAsMap(map)); + } else { + writer.write(new Yaml().dumpAs(map, Tag.SEQ, DumperOptions.FlowStyle.BLOCK)); + } + } catch (IOException e) { + LOG.error("Could not successfully write to remote."); + e.printStackTrace(); + } + } + + /** + * Parses yaml file from remote into map. + * @param stream InputStream + * @return map of Yaml syntax + */ + public static Map readFromInputStream(InputStream stream) { + StringBuilder yamlContent = new StringBuilder(); + try (Reader reader = new BufferedReader(new InputStreamReader(stream))) { + int c; + while ((c = reader.read()) != -1) { + yamlContent.append((char) c); + } + } catch (IOException e) { + LOG.error("Could not successfully read from remote."); + e.printStackTrace(); + } + Yaml yaml = new Yaml(); + return yaml.load(yamlContent.toString()); + } + + /** + * Checks with RegEx if ip could be valid address. + * @param file name of file to be checked + * @return true, if file contains valid ipv4-address + */ + public static boolean isIPAddressFile(String file) { + String ip = file.replace(".yml", ""); + final String IPV4_REGEX = + "^(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\." + + "(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\." + + "(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\." + + "(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$"; + final Pattern IPV4_PATTERN = Pattern.compile(IPV4_REGEX); + Matcher matcher = IPV4_PATTERN.matcher(ip); + return matcher.matches(); + } +} diff --git a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/util/ConfigurationFile.java b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/util/ConfigurationFile.java deleted file mode 100644 index 5ddf6db7e..000000000 --- a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/util/ConfigurationFile.java +++ /dev/null @@ -1,94 +0,0 @@ -package de.unibi.cebitec.bibigrid.core.util; - -import de.unibi.cebitec.bibigrid.core.model.Configuration; -import de.unibi.cebitec.bibigrid.core.model.exceptions.ConfigurationException; -import org.apache.commons.cli.CommandLine; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import org.yaml.snakeyaml.Yaml; -import org.yaml.snakeyaml.error.YAMLException; - -import java.io.FileInputStream; -import java.io.FileNotFoundException; -import java.nio.file.Files; -import java.nio.file.Path; -import java.nio.file.Paths; -import java.util.Map; - -import static de.unibi.cebitec.bibigrid.core.util.VerboseOutputFilter.V; - -/** - * @author mfriedrichs(at)techfak.uni-bielefeld.de - */ -public final class ConfigurationFile { - private static final Logger LOG = LoggerFactory.getLogger(ConfigurationFile.class); - private static final String DEFAULT_DIRNAME = System.getProperty("user.home"); - private static final String DEFAULT_FILENAME = ".bibigrid.yml"; - private static final String PROPERTIES_FILEPATH_PARAMETER = "o"; - - private Path propertiesFilePath; - private boolean isAlternativeFilepath; - private String propertiesMode; - - public ConfigurationFile(CommandLine commandLine) { - Path defaultPropertiesFilePath = Paths.get(DEFAULT_DIRNAME, DEFAULT_FILENAME); - if (commandLine.hasOption(PROPERTIES_FILEPATH_PARAMETER)) { - String path = commandLine.getOptionValue(PROPERTIES_FILEPATH_PARAMETER); - Path newPath = Paths.get(path); - if (Files.isReadable(newPath)) { - propertiesFilePath = newPath; - isAlternativeFilepath = true; - LOG.info("Using alternative config file: '{}'.", propertiesFilePath.toString()); - } else { - LOG.error("Alternative config ({}) file is not readable. Falling back to default: '{}'", newPath.toString(), defaultPropertiesFilePath.toString()); - } - } - if (propertiesFilePath == null) { - propertiesFilePath = defaultPropertiesFilePath; - LOG.info("Using default configuration file ({}).", defaultPropertiesFilePath); - } - if (Files.exists(propertiesFilePath)) { - LOG.info(V, "Reading options from properties file at '{}'.", propertiesFilePath); - try { - // In order to load the yaml file directly into the provider Configuration we have to peek the mode - Map yamlMap = new Yaml().load(new FileInputStream(propertiesFilePath.toFile())); - propertiesMode = yamlMap.getOrDefault("mode", null); - } catch (FileNotFoundException e) { - LOG.error("Failed to load mode parameter from properties file."); - } - } else { - LOG.info("No properties file found at '{}'. Using command line parameters only.", - propertiesFilePath); - } - } - - public boolean isAlternativeFilepath() { - return isAlternativeFilepath; - } - - public Path getPropertiesFilePath() { - return propertiesFilePath; - } - - public String getPropertiesMode() { - return propertiesMode; - } - - public Configuration loadConfiguration(Class configurationClass) - throws ConfigurationException { - if (Files.exists(propertiesFilePath)) { - try { - return new Yaml().loadAs(new FileInputStream(propertiesFilePath.toFile()), configurationClass); - } catch (FileNotFoundException e) { - throw new ConfigurationException("Failed to load properties file.", e); - } catch (YAMLException e) { - throw new ConfigurationException("Failed to parse configuration file. "+e.getMessage(), e); - } - } - try { - return configurationClass.getConstructor().newInstance(); - } catch (Exception e) { - throw new ConfigurationException("Failed to instantiate empty configuration.", e); - } - } -} diff --git a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/util/ShellScriptCreator.java b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/util/ShellScriptCreator.java index 6ca6fbcac..9d7e03c61 100644 --- a/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/util/ShellScriptCreator.java +++ b/bibigrid-core/src/main/java/de/unibi/cebitec/bibigrid/core/util/ShellScriptCreator.java @@ -142,6 +142,12 @@ public static String getMasterAnsibleExecutionScript(final Configuration config) script.append("ansible workers -i ~/" + AnsibleResources.HOSTS_CONFIG_FILE + " --become -m raw -a \"apt-get update && apt-get --yes install python3\" | sudo tee -a /var/log/ansible.log\n"); + // Test ansible + script.append("echo \"Testing Ansible...\n\";"); + script.append("ansible -i ~/" + AnsibleResources.HOSTS_CONFIG_FILE + " all -m ping | sudo tee -a /var/log/ansible.log \n"); + script.append("if [ $? -eq 0 ]; then echo \"Ansible configuration seems to work properly.\"; else echo\"Ansible hosts not reachable. " + + "There seems to be a misconfiguration.\"; fi\n"); + // Run ansible-galaxy to install ansible-galaxy roles from galaxy, git or url (.tar.gz) if (config.hasCustomAnsibleGalaxyRoles()) { script.append("ansible-galaxy install --roles-path ~/" diff --git a/bibigrid-core/src/main/resources/playbook/roles/additional/kubernetes b/bibigrid-core/src/main/resources/playbook/roles/additional/kubernetes new file mode 160000 index 000000000..287d08305 --- /dev/null +++ b/bibigrid-core/src/main/resources/playbook/roles/additional/kubernetes @@ -0,0 +1 @@ +Subproject commit 287d0830530bcb4640cf10932c9d246afe964add diff --git a/bibigrid-googlecloud/pom.xml b/bibigrid-googlecloud/pom.xml index a676c70ec..7a4d4ecbb 100644 --- a/bibigrid-googlecloud/pom.xml +++ b/bibigrid-googlecloud/pom.xml @@ -5,7 +5,7 @@ bibigrid de.unibi.cebitec.bibigrid - 2.3 + 2.3.1 4.0.0 @@ -16,7 +16,7 @@ de.unibi.cebitec.bibigrid bibigrid-core - 2.3 + 2.3.1 com.google.apis diff --git a/bibigrid-light-rest-4j/pom.xml b/bibigrid-light-rest-4j/pom.xml index ac859f027..35d78746d 100644 --- a/bibigrid-light-rest-4j/pom.xml +++ b/bibigrid-light-rest-4j/pom.xml @@ -241,19 +241,19 @@ de.unibi.cebitec.bibigrid bibigrid-core - 2.3 + 2.3.1 compile de.unibi.cebitec.bibigrid bibigrid-openstack - 2.3 + 2.3.1 compile de.unibi.cebitec.bibigrid bibigrid-main - 2.3 + 2.3.1 compile diff --git a/bibigrid-light-rest-4j/src/main/java/de/unibi/cebitec/bibigrid/light_rest_4j/handler/BibigridCreatePostHandler.java b/bibigrid-light-rest-4j/src/main/java/de/unibi/cebitec/bibigrid/light_rest_4j/handler/BibigridCreatePostHandler.java index ba382dd19..40d52bd18 100644 --- a/bibigrid-light-rest-4j/src/main/java/de/unibi/cebitec/bibigrid/light_rest_4j/handler/BibigridCreatePostHandler.java +++ b/bibigrid-light-rest-4j/src/main/java/de/unibi/cebitec/bibigrid/light_rest_4j/handler/BibigridCreatePostHandler.java @@ -7,6 +7,7 @@ import de.unibi.cebitec.bibigrid.core.intents.TerminateIntent; import de.unibi.cebitec.bibigrid.core.intents.ValidateIntent; import de.unibi.cebitec.bibigrid.core.model.Client; +import de.unibi.cebitec.bibigrid.core.model.Cluster; import de.unibi.cebitec.bibigrid.core.model.Configuration; import de.unibi.cebitec.bibigrid.core.model.ProviderModule; import de.unibi.cebitec.bibigrid.core.model.exceptions.ConfigurationException; @@ -18,6 +19,9 @@ import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import java.util.HashMap; +import java.util.Map; + public class BibigridCreatePostHandler implements LightHttpHandler { private static final Logger LOG = LoggerFactory.getLogger(BibigridValidatePostHandler.class); @@ -60,8 +64,9 @@ private boolean runCreateIntent(ProviderModule module, Configuration config, Cli LOG.error(BibigridCreatePostHandler.KEEP); } else { LOG.error(BibigridCreatePostHandler.ABORT_WITH_INSTANCES_RUNNING); - - TerminateIntent cleanupIntent = module.getTerminateIntent(config); + Map clusterMap = new HashMap<>(); + clusterMap.put(cluster.getCluster().getClusterId(), cluster.getCluster()); + TerminateIntent cleanupIntent = module.getTerminateIntent(config, clusterMap); cleanupIntent.terminate(cluster_id); } @@ -76,7 +81,7 @@ private boolean runCreateIntent(ProviderModule module, Configuration config, Cli } return false; } - cluster_id = cluster.getClusterId(); + cluster_id = cluster.getCluster().getClusterId(); return true; } diff --git a/bibigrid-light-rest-4j/src/main/java/de/unibi/cebitec/bibigrid/light_rest_4j/handler/BibigridInfoIdGetHandler.java b/bibigrid-light-rest-4j/src/main/java/de/unibi/cebitec/bibigrid/light_rest_4j/handler/BibigridInfoIdGetHandler.java index 034c01764..ace5f58de 100644 --- a/bibigrid-light-rest-4j/src/main/java/de/unibi/cebitec/bibigrid/light_rest_4j/handler/BibigridInfoIdGetHandler.java +++ b/bibigrid-light-rest-4j/src/main/java/de/unibi/cebitec/bibigrid/light_rest_4j/handler/BibigridInfoIdGetHandler.java @@ -27,7 +27,7 @@ public void handleRequest(HttpServerExchange exchange) { String clusterId = exchange.getQueryParameters().get("id").getFirst(); Status status = DataBase.getDataBase().status.get(clusterId); if (status == null) { - status = new Status(Status.CODE.Error, "Unknown cluster id" + clusterId + "!"); + status = new Status(Status.CODE.Error, "Unknown cluster id " + clusterId + "!"); } exchange.getResponseHeaders().add(new HttpString("Content-Type"), "application/json"); exchange.setStatusCode(200); diff --git a/bibigrid-light-rest-4j/src/main/java/de/unibi/cebitec/bibigrid/light_rest_4j/handler/BibigridTerminateIdDeleteHandler.java b/bibigrid-light-rest-4j/src/main/java/de/unibi/cebitec/bibigrid/light_rest_4j/handler/BibigridTerminateIdDeleteHandler.java index 3fd6dcf54..ee84e0dc7 100644 --- a/bibigrid-light-rest-4j/src/main/java/de/unibi/cebitec/bibigrid/light_rest_4j/handler/BibigridTerminateIdDeleteHandler.java +++ b/bibigrid-light-rest-4j/src/main/java/de/unibi/cebitec/bibigrid/light_rest_4j/handler/BibigridTerminateIdDeleteHandler.java @@ -3,8 +3,10 @@ import com.fasterxml.jackson.core.JsonProcessingException; import com.fasterxml.jackson.databind.ObjectMapper; import com.networknt.handler.LightHttpHandler; +import de.unibi.cebitec.bibigrid.core.intents.LoadClusterConfigurationIntent; import de.unibi.cebitec.bibigrid.core.intents.TerminateIntent; import de.unibi.cebitec.bibigrid.core.model.Client; +import de.unibi.cebitec.bibigrid.core.model.Cluster; import de.unibi.cebitec.bibigrid.core.model.ProviderModule; import de.unibi.cebitec.bibigrid.openstack.ConfigurationOpenstack; import io.undertow.server.HttpServerExchange; @@ -34,7 +36,15 @@ public void handleRequest(HttpServerExchange exchange) { // set id for termination clusterId = exchange.getQueryParameters().get("id").getFirst(); - TerminateIntent terminateIntent = module.getTerminateIntent(config); + + // Load cluster config from cloud provider -> clusterMap for specific intentModes + LoadClusterConfigurationIntent loadIntent = module.getLoadClusterConfigurationIntent(config); + loadIntent.loadClusterConfiguration(clusterId); + // Load specific cluster to put into clusterMap + Cluster cluster = loadIntent.getCluster(clusterId); + Map clusterMap = new HashMap<>(); + clusterMap.put(clusterId, cluster); + TerminateIntent terminateIntent = module.getTerminateIntent(config, clusterMap); if (terminateIntent.terminate(clusterId)) { try { diff --git a/bibigrid-main/pom.xml b/bibigrid-main/pom.xml index 0d1409b24..406df3f90 100644 --- a/bibigrid-main/pom.xml +++ b/bibigrid-main/pom.xml @@ -5,7 +5,7 @@ bibigrid de.unibi.cebitec.bibigrid - 2.3 + 2.3.1 4.0.0 @@ -21,22 +21,22 @@ de.unibi.cebitec.bibigrid bibigrid-openstack - 2.3 + 2.3.1 de.unibi.cebitec.bibigrid bibigrid-aws - 2.3 + 2.3.1 de.unibi.cebitec.bibigrid bibigrid-azure - 2.3 + 2.3.1 de.unibi.cebitec.bibigrid bibigrid-googlecloud - 2.3 + 2.3.1 @@ -47,7 +47,7 @@ de.unibi.cebitec.bibigrid bibigrid-openstack - 2.3 + 2.3.1 @@ -58,7 +58,7 @@ de.unibi.cebitec.bibigrid bibigrid-aws - 2.3 + 2.3.1 @@ -69,7 +69,7 @@ de.unibi.cebitec.bibigrid bibigrid-azure - 2.3 + 2.3.1 @@ -80,7 +80,7 @@ de.unibi.cebitec.bibigrid bibigrid-googlecloud - 2.3 + 2.3.1 @@ -91,7 +91,7 @@ de.unibi.cebitec.bibigrid bibigrid-core - 2.3 + 2.3.1 commons-cli diff --git a/bibigrid-main/src/main/java/de/unibi/cebitec/bibigrid/CommandLineInterpreter.java b/bibigrid-main/src/main/java/de/unibi/cebitec/bibigrid/CommandLineInterpreter.java new file mode 100644 index 000000000..eb89bb775 --- /dev/null +++ b/bibigrid-main/src/main/java/de/unibi/cebitec/bibigrid/CommandLineInterpreter.java @@ -0,0 +1,137 @@ +package de.unibi.cebitec.bibigrid; + +import de.unibi.cebitec.bibigrid.core.Constant; +import de.unibi.cebitec.bibigrid.core.model.IntentMode; +import org.apache.commons.cli.*; + +import static de.unibi.cebitec.bibigrid.core.intents.LoadClusterConfigurationIntent.LOG; +import static de.unibi.cebitec.bibigrid.core.model.IntentMode.*; +import static de.unibi.cebitec.bibigrid.core.model.IntentMode.LIST; + +/** + * Parses commandline and contains options. + * + * @author tdilger - t.dilger(at)uni-bielefeld.de + */ +public class CommandLineInterpreter { + private CommandLine cl; + private Options cmdLineOptions; + private OptionGroup intentOptions; + private IntentMode intentMode; + + CommandLineInterpreter(String[] parameter) { + CommandLineParser clParser = new DefaultParser(); + addOptions(); + try { + cl = clParser.parse(cmdLineOptions, parameter); + + intentMode = IntentMode.fromString(intentOptions.getSelected()); + } catch (ParseException pe) { + LOG.error("Error while parsing the commandline arguments: {}", pe.getMessage()); + LOG.error(Constant.ABORT_WITH_NOTHING_STARTED); + } + } + + /** + * Adds terminal behaviour and intent options. + */ + private void addOptions() { + intentOptions = IntentOptions.getIntentOptions(); + cmdLineOptions = new Options() + .addOption(new Option("v", "verbose", false,"More verbose output")) + .addOption(new Option("o","config",true,"Path to JSON configuration file")) + .addOption(new Option("d","debug",false,"Don't shut down cluster in the case of a configuration error.")) + .addOption(new Option("m","mode",true,"One of "+String.join(",", Provider.getInstance().getProviderNames()))) + .addOptionGroup(intentOptions); + } + + public static CommandLineInterpreter parseCommandLine(String[] parameter) { + return new CommandLineInterpreter(parameter); + } + + public boolean isMode(String mode) { + return cl.hasOption(mode); + } + + public Options getCmdLineOptions() { + return cmdLineOptions; + } + + public IntentMode getIntentMode() { + return intentMode; + } + + public String getOptionValue(String option) { + return cl.getOptionValue(option); + } + + public String[] getOptionValues(String option) { + return cl.getOptionValues(option); + } +} + +/** + * Additional intent options Version. + * Contains Help, Validate, Create, Scale Up, Scale Down, Terminate, IDE and List + */ +class IntentOptions extends OptionGroup { + private static final String CID = "cluster-id"; + private static final String SCALE = CID + " worker-Batch count"; + private static final int SCALE_ARGS = 3; // cluster-id, workerBatch, count + private static final boolean REQUIRED = true; + + private IntentOptions() { + super(); + this.setRequired(REQUIRED); + + Option version = new Option(VERSION.getShortParam(), VERSION.getLongParam(), + false, VERSION.getDescription()); + this.addOption(version); + + Option help = new Option(HELP.getShortParam(), HELP.getLongParam(), + false, HELP.getDescription()); + this.addOption(help); + + Option validate = new Option(VALIDATE.getShortParam(), VALIDATE.getLongParam(), + false, VALIDATE.getDescription()); + this.addOption(validate); + + Option create = new Option(CREATE.getShortParam(), CREATE.getLongParam(), + false, CREATE.getDescription()); + this.addOption(create); + + Option terminate = new Option(TERMINATE.getShortParam(), TERMINATE.getLongParam(), + true, TERMINATE.getDescription()); + terminate.setArgs(Option.UNLIMITED_VALUES); + terminate.setArgName(CID); + this.addOption(terminate); + + Option upscale = new Option(SCALE_UP.getShortParam(), SCALE_UP.getLongParam(), + true, SCALE_UP.getDescription()); + upscale.setArgs(SCALE_ARGS); + upscale.setArgName(SCALE); + this.addOption(upscale); + + Option downscale = new Option(SCALE_DOWN.getShortParam(), SCALE_DOWN.getLongParam(), + true, SCALE_DOWN.getDescription()); + downscale.setArgs(SCALE_ARGS); + downscale.setArgName(SCALE); + this.addOption(downscale); + + Option ide = new Option(IDE.getShortParam(), IDE.getLongParam(), + true, IDE.getDescription()); + ide.setArgName(CID); + this.addOption(ide); + + Option list = new Option(LIST.getShortParam(), LIST.getLongParam(), + true, LIST.getDescription()); + list.setOptionalArg(true); + list.setArgName(CID); + this.addOption(list); + } + + static OptionGroup getIntentOptions() { + return new IntentOptions(); + } +} + diff --git a/bibigrid-main/src/main/java/de/unibi/cebitec/bibigrid/StartUp.java b/bibigrid-main/src/main/java/de/unibi/cebitec/bibigrid/StartUp.java index 7600551bc..f9ddc3cd3 100644 --- a/bibigrid-main/src/main/java/de/unibi/cebitec/bibigrid/StartUp.java +++ b/bibigrid-main/src/main/java/de/unibi/cebitec/bibigrid/StartUp.java @@ -36,122 +36,62 @@ public class StartUp { "afterwards."; private static final String KEEP = "Keeping the partly configured cluster for debug purposes. Please remember to shut it down afterwards."; - private static final String CID = "cluster-id"; - private static final String SCALE = CID+" worker-Batch count"; - private static final int SCALE_ARGS = 3; // cluster-id, workerBatch, count - - private static OptionGroup getCMDLineOptionGroup() { - OptionGroup intentOptions = new OptionGroup(); - intentOptions.setRequired(true); - Option terminate = new Option(TERMINATE.getShortParam(), TERMINATE.getLongParam(), - true, "Terminate running cluster"); - terminate.setArgs(Option.UNLIMITED_VALUES); - terminate.setArgName(CID); - Option upscale = new Option(SCALE_UP.getShortParam(), SCALE_UP.getLongParam(), - true, "Adds a given amount of workers of a specific instance type"); - upscale.setArgs(SCALE_ARGS); - upscale.setArgName(SCALE); - Option downscale = new Option(SCALE_DOWN.getShortParam(), SCALE_DOWN.getLongParam(), - true, "Terminates a given amount of workers of a specific instance type"); - downscale.setArgs(SCALE_ARGS); - downscale.setArgName(SCALE); - Option ide = new Option(IDE.getShortParam(), IDE.getLongParam(), - true, "Start a Web IDE"); - ide.setArgName(CID); - Option list = new Option(LIST.getShortParam(), LIST.getLongParam(), - true, "List running clusters"); - list.setOptionalArg(true); - list.setArgName(CID); - intentOptions - .addOption(new Option(VERSION.getShortParam(), VERSION.getLongParam(), - false, "Version")) - .addOption(new Option(HELP.getShortParam(), HELP.getLongParam(), - false, "Help")) - .addOption(new Option(CREATE.getShortParam(), CREATE.getLongParam(), - false, "Create cluster")) - .addOption(list) - .addOption(new Option(VALIDATE.getShortParam(), VALIDATE.getLongParam(), - false, "Validate the configuration file")) - .addOption(terminate) - .addOption(upscale) - .addOption(downscale) - .addOption(ide); - return intentOptions; - } - public static void main(String[] args) { - CommandLineParser cli = new DefaultParser(); - OptionGroup intentOptions = getCMDLineOptionGroup(); - Options cmdLineOptions = new Options() - .addOption(new Option("h","help",false,"Get some online help")) - .addOption(new Option("v","verbose", false,"More verbose output")) - .addOption(new Option("o","config",true,"Path to JSON configuration file")) - .addOption(new Option("d","debug",false,"Don't shut down cluster in the case of a configuration error.")) - .addOption(new Option("m","mode",true,"One of "+String.join(",",Provider.getInstance().getProviderNames()))) - .addOptionGroup(intentOptions); - - - try { - CommandLine cl = cli.parse(cmdLineOptions, args); + CommandLineInterpreter cli = CommandLineInterpreter.parseCommandLine(args); + IntentMode intentMode = cli.getIntentMode(); - IntentMode intentMode = IntentMode.fromString(intentOptions.getSelected()); + // Only version info print necessary + if (intentMode == VERSION) { + printVersionInfo(); + return; + } + // Just to get information faster + // TODO verbose mode print table of different machines? - // Only version info print necessary - if (intentMode == VERSION) { - printVersionInfo(); - return; - } - // Just to get information faster - if (intentMode == HELP) { - // printInstanceTypeHelp(module, client, config); - printHelp(cl, cmdLineOptions); - return; - } + if (intentMode == HELP) { + printHelp(cli.getCmdLineOptions()); + return; + } - // Options - if (cl.hasOption("verbose")) { - VerboseOutputFilter.SHOW_VERBOSE = true; - LOG.info("Enable verbose mode for more detailed output."); - } + // Options + if (cli.isMode("verbose")) { + VerboseOutputFilter.SHOW_VERBOSE = true; + LOG.info("Enable verbose mode for more detailed output."); + } - if (cl.hasOption("debug")) { - Configuration.DEBUG = true; - LOG.info("Enable debug mode to keep cluster in case of a configuration error."); - } + if (cli.isMode("debug")) { + Configuration.DEBUG = true; + LOG.info("Enable debug mode to keep cluster in case of a configuration error."); + } - String providerMode = cl.getOptionValue("mode"); - String configurationFile = cl.getOptionValue("config"); + String providerMode = cli.getOptionValue("mode"); + String configurationFile = cli.getOptionValue("config"); - ProviderModule module = loadProviderModule(providerMode); + ProviderModule module = loadProviderModule(providerMode); - try { - // Provider specific configuration and validator - Configuration config = module.getConfiguration(configurationFile); - Validator validator = module.getValidator(config, module); + try { + // Provider specific configuration and validator + // TODO add another module -> loadConfiguration with ssh .. from config file / source, but else loaded from remote + Configuration config = module.getConfiguration(configurationFile); + Validator validator = module.getValidator(config, module); - // Map of IntentMode and clusterIds - Map clOptions = new HashMap<>(); - for (IntentMode im : IntentMode.values()) { - String[] parameters = cl.getOptionValues(im.getShortParam()); - if (parameters == null) { - parameters = new String[] {cl.getOptionValue(im.getShortParam())}; - } - clOptions.put(im, parameters); + // Map of IntentMode and clusterIds + Map clOptions = new HashMap<>(); + for (IntentMode im : IntentMode.values()) { + String[] parameters = cli.getOptionValues(im.getShortParam()); + if (parameters == null) { + parameters = new String[] {cli.getOptionValue(im.getShortParam())}; } + clOptions.put(im, parameters); + } - // TODO validates by configuration file - // Only a valid option for creation and validation parameter - if (validator.validateProviderParameters()) { - runIntent(module, validator, clOptions, intentMode, config); - } else { - LOG.error(Constant.ABORT_WITH_NOTHING_STARTED); - } - } catch (ConfigurationException e) { - LOG.error(e.getMessage()); + if (validator.validateProviderParameters()) { + runIntent(module, validator, clOptions, intentMode, config); + } else { LOG.error(Constant.ABORT_WITH_NOTHING_STARTED); } - } catch (ParseException pe) { - LOG.error("Error while parsing the commandline arguments: {}", pe.getMessage()); + } catch (ConfigurationException e) { + LOG.error(e.getMessage()); LOG.error(Constant.ABORT_WITH_NOTHING_STARTED); } } @@ -163,7 +103,7 @@ public static void main(String[] args) { */ private static ProviderModule loadProviderModule(String providerMode) { ProviderModule module; - String [] availableProviderModes = Provider.getInstance().getProviderNames(); + String[] availableProviderModes = Provider.getInstance().getProviderNames(); if (availableProviderModes.length == 1) { LOG.info("Use {} provider.", availableProviderModes[0]); module = Provider.getInstance().getProviderModule(availableProviderModes[0]); @@ -186,7 +126,12 @@ private static ProviderModule loadProviderModule(String providerMode) { * @param intentMode Current IntentMode * @param config Configuration */ - private static void runIntent(ProviderModule module, Validator validator, Map clOptions, IntentMode intentMode, Configuration config) { + private static void runIntent( + ProviderModule module, + Validator validator, + Map clOptions, + IntentMode intentMode, + Configuration config) { try { module.createClient(config); } catch (ClientConnectionFailedException e) { @@ -195,55 +140,66 @@ private static void runIntent(ProviderModule module, Validator validator, Map clusterMap = loadIntent.getClusterMap(); - if (clusterMap.isEmpty()) { + // In order to validate the native instance types, we need a client. + // So validating configuration is deferred after client connection is established. + // Config validation in validate and create intentMode only + if (intentMode == VALIDATE || intentMode == CREATE) { + try { + if (!validator.validateConfiguration()) { + LOG.error(ABORT_WITH_NOTHING_STARTED); + return; + } + } catch (Exception e){ + LOG.error(e.getMessage()); + if (Configuration.DEBUG) { + e.printStackTrace(); + } return; } - ListIntent listIntent = module.getListIntent(clusterMap); - if (clusterId == null) { - LOG.info(listIntent.toString()); + if (module.getValidateIntent(config).validate()) { + if (intentMode == VALIDATE) LOG.info(I, "You can now start your cluster."); + else { + // intentMode CREATE - Start cluster creation + CreateCluster cluster = module.getCreateIntent(config, null); + runCreateIntent(module, config, cluster); + } } else { - LOG.info(listIntent.toDetailString(clusterId)); + LOG.error("There were one or more errors. Please adjust your configuration."); } return; } + // Usually parameters equals clusterId(s) or null + String[] parameters = clOptions.get(intentMode); + + // If parameter given, parameter[0] is clusterId + String clusterId = parameters.length == 0 ? null : parameters[0]; + + // Load cluster config from cloud provider -> clusterMap for specific intentModes + LoadClusterConfigurationIntent loadIntent = module.getLoadClusterConfigurationIntent(config); + loadIntent.loadClusterConfiguration(clusterId); + Map clusterMap = loadIntent.getClusterMap(); + if (clusterMap == null || clusterMap.isEmpty()) { + // if there is no cluster, none action possible + return; + } + switch (intentMode) { - case VALIDATE: - if (validator.configurationInvalid()) { - break; - } - if (module.getValidateIntent(config).validate()) { - LOG.info(I, "You can now start your cluster."); + case LIST: + ListIntent listIntent = module.getListIntent(clusterMap); + if (clusterId == null) { + LOG.info(listIntent.toString()); } else { - LOG.error("There were one or more errors. Please adjust your configuration."); - } - break; - case CREATE: - if (validator.configurationInvalid()) { - break; - } - if (module.getValidateIntent(config).validate()) { - CreateCluster cluster = module.getCreateIntent(config, clusterId); - runCreateIntent(module, config, cluster); - } else { - LOG.error("There were one or more errors. Please adjust your configuration."); + // TODO list of more than one cluster optional? + LOG.info(listIntent.toDetailString(clusterId)); } break; case TERMINATE: - if (!module.getTerminateIntent(config).terminate(parameters)) { + boolean terminate_success = module.getTerminateIntent(config, clusterMap).terminate(parameters); + if (!terminate_success) { if (parameters.length == 1) { - LOG.error("Could not terminate instances with given parameter {}.", parameters[0]); + LOG.error("Could not terminate instances with given parameter {}.", clusterId); } else { - // LOG.warn("StartUp, given parameters: {}", Arrays.stream(parameters)); StringBuilder error = new StringBuilder("Could not terminate instances with given parameters "); for (int p = 0; p < parameters.length; p++) { String parameter = parameters[p]; @@ -263,12 +219,13 @@ private static void runIntent(ProviderModule module, Validator validator, Map instead.'"); + LOG.error("Wrong usage. Please use '-su ' instead."); return; } CreateCluster createIntent = module.getCreateIntent(config, clusterId); - if (!createIntent.createAdditionalWorkerInstances(workerBatch, count)) { - LOG.error("Could not create worker instances with specified batch."); + boolean create_success = createIntent.createWorkerInstances(workerBatch, count); + if (!create_success) { + LOG.error("Could not create {} worker instances with specified batch {}.", count, workerBatch); return; } break; @@ -277,18 +234,20 @@ private static void runIntent(ProviderModule module, Validator validator, Map instead.'"); + LOG.error("Wrong usage. Please use '-sd ' instead."); return; } - module.getTerminateIntent(config) - .terminateInstances(clusterId, workerBatch, count); + module.getTerminateIntent(config, clusterMap).terminateInstances(clusterId, workerBatch, count); break; case IDE: try { // Load private key file config.getClusterKeyPair().setName(CreateCluster.PREFIX + clusterId); config.getClusterKeyPair().load(); - new IdeIntent(module, clusterId, config).start(); + Cluster cluster = clusterMap.get(clusterId); + if (loadIntent.loadIdeConfiguration(cluster.getPublicIp())) { + new IdeIntent(cluster, config).start(); + } } catch (IOException e) { LOG.error("Exception occurred loading private key. {}",e.getMessage()); if (Configuration.DEBUG) { @@ -303,7 +262,6 @@ private static void runIntent(ProviderModule module, Validator validator, Map clusterMap = new HashMap<>(); + clusterMap.put(cluster.getCluster().getClusterId(), cluster.getCluster()); + module.getTerminateIntent(config, clusterMap).terminate(cluster.getCluster()); } return false; } @@ -349,14 +308,12 @@ private static boolean runCreateIntent(ProviderModule module, Configuration conf * Prints out version of BiBiGrid with date of build. */ private static void printVersionInfo() { - Map m = getVersionInfo(); - System.out.println(String.format("v%s (Build: %s)", - m.get("version"), - m.get("build"))); + Map m = getVersionInfo(); + LOG.info(String.format("v%s (Build: %s)", m.get("version"), m.get("build"))); } - private static Map getVersionInfo(){ - Map s = new HashMap(); + private static Map getVersionInfo(){ + Map s = new HashMap<>(); try { URL jarUrl = StartUp.class.getProtectionDomain().getCodeSource().getLocation(); String jarPath = URLDecoder.decode(jarUrl.getFile(), "UTF-8"); @@ -372,11 +329,10 @@ private static Map getVersionInfo(){ /** * Prints out terminal help. - * @param commandLine given cl input arguments * @param cmdLineOptions options [-ch -c -v -o ...] */ - private static void printHelp(CommandLine commandLine, Options cmdLineOptions) { - Map map = getVersionInfo(); + private static void printHelp(Options cmdLineOptions) { + Map map = getVersionInfo(); HelpFormatter help = new HelpFormatter(); String footer = "\nDocumentation at https://github.com/BiBiServ/bibigrid/docs\n"; footer += "Loaded provider modules: " + String.join(", ", Provider.getInstance().getProviderNames()); @@ -384,10 +340,9 @@ private static void printHelp(CommandLine commandLine, Options cmdLineOptions) { String modes = Arrays.stream(IntentMode.values()).map(m -> "--" + m.getLongParam()).collect(Collectors.joining("|")); System.out.println("BiBiGrid is a tool for an easy cluster setup inside a cloud environment.\n"); help.printHelp(100, - "java -jar bibigrid-"+String.join(", ", Provider.getInstance().getProviderNames())+"-"+map.get("version")+".jar", - "", - cmdLineOptions, - footer); + "java -jar bibigrid-" + + String.join(", ", Provider.getInstance().getProviderNames()) + + "-" + map.get("version")+".jar","", cmdLineOptions, footer); System.out.println('\n'); } diff --git a/bibigrid-main/tests.log b/bibigrid-main/tests.log new file mode 100644 index 000000000..46277740f --- /dev/null +++ b/bibigrid-main/tests.log @@ -0,0 +1,48 @@ +140 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +154 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +146 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +152 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +131 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +141 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +127 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +117 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +136 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +145 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +137 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +137 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +139 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +149 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +131 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +119 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +133 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +142 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +148 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +169 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +166 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +158 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +137 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +167 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +194 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +164 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +163 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +150 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +169 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +149 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +137 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +175 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +167 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +176 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +151 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +168 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +176 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +151 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +158 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +168 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +159 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +148 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +150 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +173 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +148 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +143 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +150 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +162 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack diff --git a/bibigrid-openstack/pom.xml b/bibigrid-openstack/pom.xml index eb1cac98b..7d94155ec 100644 --- a/bibigrid-openstack/pom.xml +++ b/bibigrid-openstack/pom.xml @@ -5,7 +5,7 @@ bibigrid de.unibi.cebitec.bibigrid - 2.3 + 2.3.1 4.0.0 @@ -16,7 +16,7 @@ de.unibi.cebitec.bibigrid bibigrid-core - 2.3 + 2.3.1 @@ -34,7 +34,7 @@ com.google.guava guava - 28.0-jre + 29.0-jre diff --git a/bibigrid-openstack/src/main/java/de/unibi/cebitec/bibigrid/openstack/ClientOpenstack.java b/bibigrid-openstack/src/main/java/de/unibi/cebitec/bibigrid/openstack/ClientOpenstack.java index b9ee3cee4..fee58ed3d 100644 --- a/bibigrid-openstack/src/main/java/de/unibi/cebitec/bibigrid/openstack/ClientOpenstack.java +++ b/bibigrid-openstack/src/main/java/de/unibi/cebitec/bibigrid/openstack/ClientOpenstack.java @@ -5,6 +5,7 @@ import org.openstack4j.api.OSClient; import org.openstack4j.api.compute.ServerGroupService; import org.openstack4j.api.exceptions.AuthenticationException; +import org.openstack4j.api.exceptions.ClientResponseException; import org.openstack4j.model.common.Identifier; import org.openstack4j.model.compute.Image; import org.openstack4j.model.storage.block.Volume; @@ -68,7 +69,19 @@ public void authenticate() throws ClientConnectionFailedException { OpenStackCredentials credentials = config.getOpenstackCredentials(); try { OSFactory.enableHttpLoggingFilter(config.isDebugRequests()); - internalClient = buildOSClientV3(credentials); + /* openstack4j can't follow "redirect" responses, therefore + we try to redirect to identity version 3 manually */ + try { + internalClient = buildOSClientV3(credentials); + } catch (ClientResponseException e) { + if (!credentials.getEndpoint().endsWith("/v3")){ + credentials.setEndpoint(credentials.getEndpoint()+"/v3"); + LOG.warn("AUTH endpoint seems not to point to identity api v3. Try again with"+ credentials.getEndpoint()); + internalClient = buildOSClientV3(credentials); + } else { + throw new ClientResponseException(e.getMessage(),e.getStatus()); + } + } // select region if (config.getRegion() != null && !config.getRegion().equals(credentials.getRegion())) { LOG.warn("General region option '{}' overwrites Openstack credentials configuration '{}'!",config.getRegion(),credentials.getRegion()); diff --git a/bibigrid-openstack/src/main/java/de/unibi/cebitec/bibigrid/openstack/CreateClusterEnvironmentOpenstack.java b/bibigrid-openstack/src/main/java/de/unibi/cebitec/bibigrid/openstack/CreateClusterEnvironmentOpenstack.java index 8186a2fa3..f7e7d6cbd 100644 --- a/bibigrid-openstack/src/main/java/de/unibi/cebitec/bibigrid/openstack/CreateClusterEnvironmentOpenstack.java +++ b/bibigrid-openstack/src/main/java/de/unibi/cebitec/bibigrid/openstack/CreateClusterEnvironmentOpenstack.java @@ -106,7 +106,7 @@ public CreateClusterEnvironmentOpenstack createSubnet() throws ConfigurationExce } // create a new network network = osc.networking().network().create(Builders.network() - .name(NETWORK_PREFIX + cluster.getClusterId()) + .name(NETWORK_PREFIX + cluster.getCluster().getClusterId()) .adminStateUp(true) .build()); cfg.setNetwork(network.getName()); @@ -129,7 +129,7 @@ public CreateClusterEnvironmentOpenstack createSubnet() throws ConfigurationExce } // now we can create a new subnet subnet = osc.networking().subnet().create(Builders.subnet() - .name(SUBNET_PREFIX + cluster.getClusterId()) + .name(SUBNET_PREFIX + cluster.getCluster().getClusterId()) .network(network) .ipVersion(IPVersionType.V4) .enableDHCP(true) @@ -168,8 +168,8 @@ public CreateClusterEnvironmentOpenstack createSecurityGroup() throws Configurat } try { ComputeSecurityGroupService csgs = cluster.getClient().compute().securityGroups(); - sge = csgs.create(SECURITY_GROUP_PREFIX + cluster.getClusterId(), - "Security group for cluster: " + cluster.getClusterId()); + sge = csgs.create(SECURITY_GROUP_PREFIX + cluster.getCluster().getClusterId(), + "Security group for cluster: " + cluster.getCluster().getClusterId()); // allow ssh access (TCP:22) from everywhere csgs.createRule(getPortBuilder(sge.getId(), IPProtocol.TCP, 22, 22).cidr("0.0.0.0/0").build()); // no restriction within the security group diff --git a/bibigrid-openstack/src/main/java/de/unibi/cebitec/bibigrid/openstack/CreateClusterOpenstack.java b/bibigrid-openstack/src/main/java/de/unibi/cebitec/bibigrid/openstack/CreateClusterOpenstack.java index b961df2e0..c75f31dd8 100644 --- a/bibigrid-openstack/src/main/java/de/unibi/cebitec/bibigrid/openstack/CreateClusterOpenstack.java +++ b/bibigrid-openstack/src/main/java/de/unibi/cebitec/bibigrid/openstack/CreateClusterOpenstack.java @@ -48,7 +48,7 @@ protected List resolveMountSources(List metadata = new HashMap<>(); metadata.put(Instance.TAG_NAME, masterNameTag); - metadata.put(Instance.TAG_BIBIGRID_ID, clusterId); + metadata.put(Instance.TAG_BIBIGRID_ID, cluster.getClusterId()); metadata.put(Instance.TAG_USER, config.getUser()); InstanceTypeOpenstack masterSpec = (InstanceTypeOpenstack) config.getMasterInstance().getProviderType(); ServerCreateBuilder scb; @@ -221,7 +221,7 @@ protected List launchClusterWorkerInstances( try { final Map metadata = new HashMap<>(); metadata.put(Instance.TAG_NAME, workerNameTag); - metadata.put(Instance.TAG_BIBIGRID_ID, clusterId); + metadata.put(Instance.TAG_BIBIGRID_ID, cluster.getClusterId()); metadata.put(Instance.TAG_USER, config.getUser()); int workerBatch = batchIndex + 1; metadata.put(Instance.TAG_BATCH, String.valueOf(workerBatch)); @@ -265,7 +265,6 @@ protected List launchClusterWorkerInstances( @Override protected List launchAdditionalClusterWorkerInstances( - Cluster cluster, int batchIndex, int workerIndex, Configuration.WorkerInstanceConfiguration instanceConfiguration, String workerNameTag) { @@ -273,13 +272,13 @@ protected List launchAdditionalClusterWorkerInstances( try { final Map metadata = new HashMap<>(); metadata.put(Instance.TAG_NAME, workerNameTag); - metadata.put(Instance.TAG_BIBIGRID_ID, clusterId); + metadata.put(Instance.TAG_BIBIGRID_ID, cluster.getClusterId()); metadata.put(Instance.TAG_USER, config.getUser()); metadata.put(Instance.TAG_BATCH, String.valueOf(batchIndex)); InstanceTypeOpenstack workerSpec = (InstanceTypeOpenstack) instanceConfiguration.getProviderType(); for (int i = workerIndex; i < workerIndex + instanceConfiguration.getCount(); i++) { metadata.put(Instance.TAG_INDEX, String.valueOf(workerIndex)); - ServerCreateBuilder scb = loadServerConfiguration(cluster, batchIndex, i, instanceConfiguration) + ServerCreateBuilder scb = loadServerConfiguration(batchIndex, i, instanceConfiguration) .addMetadata(metadata) .configDrive(workerSpec.getConfigDrive() != 0) .userData(ShellScriptCreator.getUserData(config, true)); @@ -304,13 +303,11 @@ protected List launchAdditionalClusterWorkerInstances( /** * Loads server config from OSClient interface. - * @param cluster specific cluster configuration * @param batchIndex index of worker batch * @param workerIndex index of worker in worker batch * @return build up server */ private ServerCreateBuilder loadServerConfiguration( - Cluster cluster, int batchIndex, int workerIndex, Configuration.WorkerInstanceConfiguration instanceConfiguration) throws NotYetSupportedException { diff --git a/bibigrid-openstack/src/main/java/de/unibi/cebitec/bibigrid/openstack/ProviderModuleOpenstack.java b/bibigrid-openstack/src/main/java/de/unibi/cebitec/bibigrid/openstack/ProviderModuleOpenstack.java index fa377b93e..ac20ce56f 100644 --- a/bibigrid-openstack/src/main/java/de/unibi/cebitec/bibigrid/openstack/ProviderModuleOpenstack.java +++ b/bibigrid-openstack/src/main/java/de/unibi/cebitec/bibigrid/openstack/ProviderModuleOpenstack.java @@ -46,8 +46,8 @@ public ListIntent getListIntent(Map clusterMap) { } @Override - public TerminateIntent getTerminateIntent(Configuration config) { - return new TerminateIntentOpenstack(this, client, config); + public TerminateIntent getTerminateIntent(Configuration config, Map clusterMap) { + return new TerminateIntentOpenstack(this, client, config, clusterMap); } @Override diff --git a/bibigrid-openstack/src/main/java/de/unibi/cebitec/bibigrid/openstack/intents/TerminateIntentOpenstack.java b/bibigrid-openstack/src/main/java/de/unibi/cebitec/bibigrid/openstack/intents/TerminateIntentOpenstack.java index ef3e7f92a..1ebff10f7 100644 --- a/bibigrid-openstack/src/main/java/de/unibi/cebitec/bibigrid/openstack/intents/TerminateIntentOpenstack.java +++ b/bibigrid-openstack/src/main/java/de/unibi/cebitec/bibigrid/openstack/intents/TerminateIntentOpenstack.java @@ -17,6 +17,8 @@ import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import java.util.Map; + /** * Implements TerminateIntent for Openstack. * @@ -27,8 +29,11 @@ public class TerminateIntentOpenstack extends TerminateIntent { private static final Logger LOG = LoggerFactory.getLogger(TerminateIntentOpenstack.class); private final OSClient os; - public TerminateIntentOpenstack(ProviderModule providerModule, Client client, Configuration config) { - super(providerModule, client, config); + public TerminateIntentOpenstack(ProviderModule providerModule, + Client client, + Configuration config, + Map clusterMap) { + super(providerModule, client, config, clusterMap); os = ((ClientOpenstack) client).getInternal(); } diff --git a/pom.xml b/pom.xml index c7c88394c..264c44d51 100644 --- a/pom.xml +++ b/pom.xml @@ -5,7 +5,7 @@ de.unibi.cebitec.bibigrid bibigrid - 2.3 + 2.3.1 default diff --git a/tests.log b/tests.log new file mode 100644 index 000000000..ccc6623ee --- /dev/null +++ b/tests.log @@ -0,0 +1,393 @@ +141 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +160 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +163 [main] INFO de.unibi.cebitec.bibigrid.StartUp - Use openstack provider. +166 [main] INFO d.u.c.b.core.model.Configuration - Using default configuration file (/home/tim/.bibigrid/configuration.yml). +217 [main] INFO d.u.c.b.core.model.Configuration - SSH public key file found. (/home/tim/.ssh/id_rsa.pub) +217 [main] INFO d.u.c.b.core.model.Configuration - Region set. (Bielefeld) +217 [main] INFO d.u.c.b.core.model.Configuration - Availability zone set. (default) +218 [main] INFO d.u.c.b.core.model.Configuration - Master instances set: +[type=de.NBI tiny, image=3df3af5c-83ce-491e-a8a2-da3345d04b6b] +219 [main] INFO d.u.c.b.core.model.Configuration - Worker instances set: +[type=de.NBI tiny, image=Ubuntu 16.04 LTS (2020-01-21), count=2] +[type=de.NBI small, image=Ubuntu 16.04 LTS (2020-01-21), count=1] +220 [main] INFO d.u.c.b.core.model.Configuration - Additional open ports set: 0.0.0.0/0:80 (TCP) +220 [main] INFO d.u.c.b.core.model.Configuration - NFS support enabled. +221 [main] INFO d.u.c.b.core.model.Configuration - Theia support disabled. +223 [main] INFO d.u.cebitec.bibigrid.core.Validator - Found valid credentials file (/home/tim/.bibigrid/credentials.yml). +223 [main] INFO d.u.cebitec.bibigrid.core.Validator - Set OpenStack Credentials ... +229 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:5000/v3/ -> /auth/tokens +427 [main] DEBUG org.jboss.logging - Logging Provider: org.jboss.logging.Slf4jLoggerProvider +428 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientWriterInterceptorContext, Method : proceed +428 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyWriter: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +428 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyWriter: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +1248 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +1248 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +1248 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +1292 [main] INFO d.u.c.b.openstack.ClientOpenstack - Openstack connection established. +1327 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /servers/detail +2765 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +2765 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +2765 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +2791 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Load Cluster Configurations ... +2791 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for all clusters ... + +2791 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for cluster with id 7cenl89btqxwddz ... +2791 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for instance bibigrid-worker-2-1-7cenl89btqxwddz ... +2791 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /flavors/df0ebd94-a4ab-46a1-ab3f-71c50c742349 +2964 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +2964 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +2964 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +2967 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /flavors/detail +3191 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +3191 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +3192 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +3196 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /images/e4ff922e-7681-411c-aa9b-6784390a904e +3649 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +3649 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +3649 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +3655 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Configuration for instance bibigrid-worker-2-1-7cenl89btqxwddz loaded successfully. +3655 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for instance bibigrid-worker-1-2-7cenl89btqxwddz ... +3655 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /flavors/7a27d96c-db45-4fe8-b9ff-29e1b4bd695e +3811 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +3811 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +3811 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +3812 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /images/e4ff922e-7681-411c-aa9b-6784390a904e +4344 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +4345 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +4345 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +4345 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Configuration for instance bibigrid-worker-1-2-7cenl89btqxwddz loaded successfully. +4345 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for instance bibigrid-worker-1-1-7cenl89btqxwddz ... +4346 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /flavors/7a27d96c-db45-4fe8-b9ff-29e1b4bd695e +4501 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +4501 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +4502 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +4502 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /images/e4ff922e-7681-411c-aa9b-6784390a904e +4961 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +4961 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +4962 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +4962 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Configuration for instance bibigrid-worker-1-1-7cenl89btqxwddz loaded successfully. +4962 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for instance bibigrid-master-7cenl89btqxwddz ... +4962 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /flavors/7a27d96c-db45-4fe8-b9ff-29e1b4bd695e +5127 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +5127 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +5127 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +5127 [main] INFO d.u.c.b.core.model.Configuration - Availability zone set. (default) +5127 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /images/e4ff922e-7681-411c-aa9b-6784390a904e +5545 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +5545 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +5545 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +5546 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Configuration for instance bibigrid-master-7cenl89btqxwddz loaded successfully. +5546 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Initialize cluster with id 7cenl89btqxwddz ... +5551 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Cluster with id 7cenl89btqxwddz initialized successfully. + +5551 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for cluster with id q6miiaur9svfpyi ... +5551 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for instance bibigrid-worker-1-3-q6miiaur9svfpyi ... +5551 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /flavors/0549db89-697a-4829-b45a-b7a79a5f2dbd +5722 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +5722 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +5722 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +5723 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /images/e4ff922e-7681-411c-aa9b-6784390a904e +6280 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +6280 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +6281 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +6281 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Configuration for instance bibigrid-worker-1-3-q6miiaur9svfpyi loaded successfully. +6281 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for instance bibigrid-worker-1-2-q6miiaur9svfpyi ... +6281 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /flavors/0549db89-697a-4829-b45a-b7a79a5f2dbd +6447 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +6447 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +6447 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +6448 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /images/e4ff922e-7681-411c-aa9b-6784390a904e +7289 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +7290 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +7290 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +7290 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Configuration for instance bibigrid-worker-1-2-q6miiaur9svfpyi loaded successfully. +7290 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for instance bibigrid-worker-1-1-q6miiaur9svfpyi ... +7291 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /flavors/0549db89-697a-4829-b45a-b7a79a5f2dbd +7450 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +7451 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +7451 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +7451 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /images/e4ff922e-7681-411c-aa9b-6784390a904e +7944 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +7944 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +7944 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +7945 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Configuration for instance bibigrid-worker-1-1-q6miiaur9svfpyi loaded successfully. +7945 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for instance bibigrid-master-q6miiaur9svfpyi ... +7945 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /flavors/1ed1fc00-3352-4ffd-a5de-0a2bf0d117a7 +8121 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +8121 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +8121 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +8122 [main] INFO d.u.c.b.core.model.Configuration - Availability zone set. (default) +8122 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /images/e4ff922e-7681-411c-aa9b-6784390a904e +8705 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +8705 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +8705 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +8706 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Configuration for instance bibigrid-master-q6miiaur9svfpyi loaded successfully. +8706 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Initialize cluster with id q6miiaur9svfpyi ... +8706 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Cluster with id q6miiaur9svfpyi initialized successfully. + +8706 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for cluster with id vs75zjltogluzpb ... +8706 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for instance bibigrid-worker-2-1-vs75zjltogluzpb ... +8706 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /flavors/df0ebd94-a4ab-46a1-ab3f-71c50c742349 +8865 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +8865 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +8865 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +8866 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /images/e4ff922e-7681-411c-aa9b-6784390a904e +9060 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +9060 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +9061 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +9061 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Configuration for instance bibigrid-worker-2-1-vs75zjltogluzpb loaded successfully. +9061 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for instance bibigrid-worker-1-2-vs75zjltogluzpb ... +9061 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /flavors/7a27d96c-db45-4fe8-b9ff-29e1b4bd695e +9217 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +9217 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +9217 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +9217 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /images/e4ff922e-7681-411c-aa9b-6784390a904e +9383 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +9383 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +9383 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +9384 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Configuration for instance bibigrid-worker-1-2-vs75zjltogluzpb loaded successfully. +9384 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for instance bibigrid-worker-1-1-vs75zjltogluzpb ... +9384 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /flavors/7a27d96c-db45-4fe8-b9ff-29e1b4bd695e +9537 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +9537 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +9537 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +9537 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /images/e4ff922e-7681-411c-aa9b-6784390a904e +9998 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +9998 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +9998 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +9999 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Configuration for instance bibigrid-worker-1-1-vs75zjltogluzpb loaded successfully. +9999 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for instance bibigrid-master-vs75zjltogluzpb ... +9999 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /flavors/7a27d96c-db45-4fe8-b9ff-29e1b4bd695e +10168 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +10168 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +10168 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +10168 [main] INFO d.u.c.b.core.model.Configuration - Availability zone set. (default) +10169 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /images/e4ff922e-7681-411c-aa9b-6784390a904e +10351 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +10352 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +10352 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +10352 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Configuration for instance bibigrid-master-vs75zjltogluzpb loaded successfully. +10352 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Initialize cluster with id vs75zjltogluzpb ... +10352 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Cluster with id vs75zjltogluzpb initialized successfully. + +10352 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for cluster with id xrqkuitvyvkqyju ... +10352 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for instance bibigrid-worker-2-1-xrqkuitvyvkqyju ... +10353 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /flavors/df0ebd94-a4ab-46a1-ab3f-71c50c742349 +10506 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +10506 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +10507 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +10507 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /images/e4ff922e-7681-411c-aa9b-6784390a904e +10696 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +10696 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +10696 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +10696 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Configuration for instance bibigrid-worker-2-1-xrqkuitvyvkqyju loaded successfully. +10696 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for instance bibigrid-worker-1-2-xrqkuitvyvkqyju ... +10697 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /flavors/7a27d96c-db45-4fe8-b9ff-29e1b4bd695e +10863 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +10864 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +10864 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +10864 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /images/e4ff922e-7681-411c-aa9b-6784390a904e +11322 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +11322 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +11322 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +11323 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Configuration for instance bibigrid-worker-1-2-xrqkuitvyvkqyju loaded successfully. +11323 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for instance bibigrid-worker-1-1-xrqkuitvyvkqyju ... +11323 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /flavors/7a27d96c-db45-4fe8-b9ff-29e1b4bd695e +11481 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +11481 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +11482 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +11482 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /images/e4ff922e-7681-411c-aa9b-6784390a904e +11839 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +11839 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +11839 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +11840 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Configuration for instance bibigrid-worker-1-1-xrqkuitvyvkqyju loaded successfully. +11840 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for instance bibigrid-master-xrqkuitvyvkqyju ... +11840 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /flavors/7a27d96c-db45-4fe8-b9ff-29e1b4bd695e +11996 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +11997 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +11997 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +11997 [main] INFO d.u.c.b.core.model.Configuration - Availability zone set. (default) +11997 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /images/e4ff922e-7681-411c-aa9b-6784390a904e +12179 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +12179 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +12179 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +12180 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Configuration for instance bibigrid-master-xrqkuitvyvkqyju loaded successfully. +12180 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Initialize cluster with id xrqkuitvyvkqyju ... +12180 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Cluster with id xrqkuitvyvkqyju initialized successfully. + +12180 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Cluster Configuration loaded successfully. +12180 [main] INFO d.u.c.b.core.intents.ListIntent - Listing Cluster Configurations: + +12181 [main] INFO de.unibi.cebitec.bibigrid.StartUp - + cluster-id | user | launch date | key name | public-ip | private-ip | # inst | group-id | subnet-id | network-id +-------------------------------------------------------------------------------------------------------------------------------------------------------------------------- + q6miiaur9svfpyi | awalende | 02/10/20 13:40:13 | bibigridq6mii… | 129.70.51.12 | 192.168.20.82 | 4 | bibigridsg-q6miia… | - | - + 7cenl89btqxwddz | jkrueger | 02/10/20 12:24:46 | bibigrid7cenl… | 129.70.51.152 | 192.168.20.126 | 4 | bibigridsg-7cenl8… | - | - + xrqkuitvyvkqyju | tim | 02/10/20 09:56:53 | bibigridxrqku… | 129.70.51.20 | 192.168.20.61 | 4 | bibigridsg-xrqkui… | - | - + vs75zjltogluzpb | tim | 02/10/20 11:41:34 | bibigridvs75z… | 129.70.51.41 | 192.168.20.152 | 4 | bibigridsg-vs75zj… | - | - + +140 [main] INFO de.unibi.cebitec.bibigrid.Provider - Registered provider module: openstack +141 [main] INFO de.unibi.cebitec.bibigrid.StartUp - Use openstack provider. +142 [main] INFO d.u.c.b.core.model.Configuration - Using default configuration file (/home/tim/.bibigrid/configuration.yml). +187 [main] INFO d.u.c.b.core.model.Configuration - SSH public key file found. (/home/tim/.ssh/id_rsa.pub) +188 [main] INFO d.u.c.b.core.model.Configuration - Region set. (Bielefeld) +188 [main] INFO d.u.c.b.core.model.Configuration - Availability zone set. (default) +189 [main] INFO d.u.c.b.core.model.Configuration - Master instances set: +[type=de.NBI tiny, image=3df3af5c-83ce-491e-a8a2-da3345d04b6b] +189 [main] INFO d.u.c.b.core.model.Configuration - Worker instances set: +[type=de.NBI tiny, image=Ubuntu 16.04 LTS (2020-01-21), count=2] +[type=de.NBI small, image=Ubuntu 16.04 LTS (2020-01-21), count=1] +190 [main] INFO d.u.c.b.core.model.Configuration - Additional open ports set: 0.0.0.0/0:80 (TCP) +191 [main] INFO d.u.c.b.core.model.Configuration - NFS support enabled. +191 [main] INFO d.u.c.b.core.model.Configuration - Theia support disabled. +193 [main] INFO d.u.cebitec.bibigrid.core.Validator - Found valid credentials file (/home/tim/.bibigrid/credentials.yml). +193 [main] INFO d.u.cebitec.bibigrid.core.Validator - Set OpenStack Credentials ... +199 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:5000/v3/ -> /auth/tokens +394 [main] DEBUG org.jboss.logging - Logging Provider: org.jboss.logging.Slf4jLoggerProvider +394 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientWriterInterceptorContext, Method : proceed +394 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyWriter: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +394 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyWriter: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +1164 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +1164 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +1164 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +1207 [main] INFO d.u.c.b.openstack.ClientOpenstack - Openstack connection established. +1248 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /flavors/detail +1774 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +1774 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +1775 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +1789 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /servers/detail +3077 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +3077 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +3077 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +3096 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Load Cluster Configurations ... +3096 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for cluster with id xrqkuitvyvkqyju ... +3096 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for instance bibigrid-worker-2-1-xrqkuitvyvkqyju ... +3096 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /flavors/df0ebd94-a4ab-46a1-ab3f-71c50c742349 +3255 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +3255 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +3256 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +3260 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /images/e4ff922e-7681-411c-aa9b-6784390a904e +3759 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +3760 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +3760 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +3765 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Configuration for instance bibigrid-worker-2-1-xrqkuitvyvkqyju loaded successfully. +3765 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for instance bibigrid-worker-1-2-xrqkuitvyvkqyju ... +3766 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /flavors/7a27d96c-db45-4fe8-b9ff-29e1b4bd695e +3920 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +3920 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +3921 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +3921 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /images/e4ff922e-7681-411c-aa9b-6784390a904e +4482 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +4482 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +4482 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +4483 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Configuration for instance bibigrid-worker-1-2-xrqkuitvyvkqyju loaded successfully. +4483 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for instance bibigrid-worker-1-1-xrqkuitvyvkqyju ... +4483 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /flavors/7a27d96c-db45-4fe8-b9ff-29e1b4bd695e +4639 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +4639 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +4639 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +4640 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /images/e4ff922e-7681-411c-aa9b-6784390a904e +5117 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +5117 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +5117 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +5118 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Configuration for instance bibigrid-worker-1-1-xrqkuitvyvkqyju loaded successfully. +5118 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for instance bibigrid-master-xrqkuitvyvkqyju ... +5118 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /flavors/7a27d96c-db45-4fe8-b9ff-29e1b4bd695e +5265 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +5266 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +5266 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +5266 [main] INFO d.u.c.b.core.model.Configuration - Availability zone set. (default) +5266 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /images/e4ff922e-7681-411c-aa9b-6784390a904e +5784 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +5784 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +5784 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +5785 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Configuration for instance bibigrid-master-xrqkuitvyvkqyju loaded successfully. +5785 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Initialize cluster with id xrqkuitvyvkqyju ... +5790 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Cluster with id xrqkuitvyvkqyju initialized successfully. + +5790 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Cluster Configuration loaded successfully. +5790 [main] INFO d.u.c.b.c.intents.TerminateIntent - Terminate given parameter xrqkuitvyvkqyju +5790 [main] INFO d.u.c.b.c.intents.TerminateIntent - Terminating cluster with ID 'xrqkuitvyvkqyju' ... +5791 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /servers/6ea218be-344e-4c43-91f7-728259d2dc05 +6201 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /servers/2f57258e-41ad-43df-9ff6-f05326fae12b +6496 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /servers/3bbba287-8962-4cab-bd56-3ab06a157300 +6767 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /servers/eef1f114-49a6-406c-b0f2-e45893772248 +8039 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /os-security-groups +8810 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +8811 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +8811 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +8819 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /os-security-groups/d59a6abb-2a32-481f-bbb0-aaf65809c374 +9158 [main] INFO d.u.c.b.o.TerminateIntentOpenstack - Security group 'bibigridsg-xrqkuitvyvkqyju' deleted. +9158 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /os-keypairs/bibigridxrqkuitvyvkqyju +9327 [main] INFO d.u.c.b.o.TerminateIntentOpenstack - Keypair 'bibigridxrqkuitvyvkqyju' deleted! +9327 [main] INFO d.u.c.b.c.intents.TerminateIntent - Private key /home/tim/.bibigrid/keys/bibigridxrqkuitvyvkqyju deleted. +9328 [main] INFO d.u.c.b.c.intents.TerminateIntent - Public key /home/tim/.bibigrid/keys/bibigridxrqkuitvyvkqyju.pub deleted. +9328 [main] INFO d.u.c.b.c.intents.TerminateIntent - Cluster 'xrqkuitvyvkqyju' terminated! +9328 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /servers/detail +10177 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +10177 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +10177 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +10183 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Load Cluster Configurations ... +10183 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for cluster with id vs75zjltogluzpb ... +10183 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for instance bibigrid-worker-2-1-vs75zjltogluzpb ... +10183 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /flavors/df0ebd94-a4ab-46a1-ab3f-71c50c742349 +10340 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +10341 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +10341 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +10341 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /images/e4ff922e-7681-411c-aa9b-6784390a904e +10827 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +10827 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +10827 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +10828 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Configuration for instance bibigrid-worker-2-1-vs75zjltogluzpb loaded successfully. +10828 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for instance bibigrid-worker-1-2-vs75zjltogluzpb ... +10828 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /flavors/7a27d96c-db45-4fe8-b9ff-29e1b4bd695e +10986 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +10987 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +10987 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +10987 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /images/e4ff922e-7681-411c-aa9b-6784390a904e +11180 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +11180 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +11180 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +11180 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Configuration for instance bibigrid-worker-1-2-vs75zjltogluzpb loaded successfully. +11180 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for instance bibigrid-worker-1-1-vs75zjltogluzpb ... +11181 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /flavors/7a27d96c-db45-4fe8-b9ff-29e1b4bd695e +11342 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +11342 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +11342 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +11342 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /images/e4ff922e-7681-411c-aa9b-6784390a904e +11529 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +11529 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +11529 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +11530 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Configuration for instance bibigrid-worker-1-1-vs75zjltogluzpb loaded successfully. +11530 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Loading Configuration for instance bibigrid-master-vs75zjltogluzpb ... +11530 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /flavors/7a27d96c-db45-4fe8-b9ff-29e1b4bd695e +11674 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +11674 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +11674 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +11675 [main] INFO d.u.c.b.core.model.Configuration - Availability zone set. (default) +11675 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /images/e4ff922e-7681-411c-aa9b-6784390a904e +12124 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +12125 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +12125 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +12125 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Configuration for instance bibigrid-master-vs75zjltogluzpb loaded successfully. +12125 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Initialize cluster with id vs75zjltogluzpb ... +12125 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Cluster with id vs75zjltogluzpb initialized successfully. + +12125 [main] INFO d.u.c.b.c.i.LoadClusterConfigurationIntent - Cluster Configuration loaded successfully. +12125 [main] INFO d.u.c.b.c.intents.TerminateIntent - Terminate given parameter vs75zjltogluzpb +12125 [main] INFO d.u.c.b.c.intents.TerminateIntent - Terminating cluster with ID 'vs75zjltogluzpb' ... +12126 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /servers/431774ed-715f-4110-97cf-8edb51777eab +12402 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /servers/fac4f084-410f-41b7-af7c-8b027ce3c827 +12683 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /servers/22f36ac2-c96a-4ccf-bef3-0d4de0bd9faa +12997 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /servers/b02457b1-c6f4-44e0-a154-dd8d293db920 +14284 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /os-security-groups +15409 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - Interceptor Context: org.jboss.resteasy.core.interception.jaxrs.ClientReaderInterceptorContext, Method : proceed +15409 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: org.jboss.resteasy.spi.ResteasyProviderFactory$SortedKey +15409 [main] DEBUG o.j.resteasy.resteasy_jaxrs.i18n - MessageBodyReader: com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider +15410 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /os-security-groups/e00e5f50-49ac-4fd8-815f-3d2acbac168f +15781 [main] INFO d.u.c.b.o.TerminateIntentOpenstack - Security group 'bibigridsg-vs75zjltogluzpb' deleted. +15781 [main] DEBUG o.o.c.t.internal.HttpExecutor - Executing Request: https://openstack.cebitec.uni-bielefeld.de:8774/v2.1/3727bc448db74d748e72a03c5cdbcd72 -> /os-keypairs/bibigridvs75zjltogluzpb +15957 [main] INFO d.u.c.b.o.TerminateIntentOpenstack - Keypair 'bibigridvs75zjltogluzpb' deleted! +15957 [main] INFO d.u.c.b.c.intents.TerminateIntent - Private key /home/tim/.bibigrid/keys/bibigridvs75zjltogluzpb deleted. +15957 [main] INFO d.u.c.b.c.intents.TerminateIntent - Public key /home/tim/.bibigrid/keys/bibigridvs75zjltogluzpb.pub deleted. +15957 [main] INFO d.u.c.b.c.intents.TerminateIntent - Cluster 'vs75zjltogluzpb' terminated!