Perl modules for the instantiation of, configuration of and communication with multiple VMs in an Openstack environment.
Install some Perl libraries:
sudo apt-get install liblocal-lib-perl libio-all-lwp-perl
sudo cpan install Parallel::ForkManager File::Flock JSON Text::ASCIITable
Make sure that Parallel::ForkManager is at least version 0.7.6.
Do a "git clone" of this repository or simply wget the two modules:
wget https://github.com/wgerlach/BulkVM/raw/master/ManageBulkInstances.pm
wget https://github.com/wgerlach/BulkVM/raw/master/SubmitVM.pm
If you want to use AWE, you might want to download this script:
wget https://github.com/wtangiit/vmScriptAWE/raw/master/vmAWE.pl
chmod +x vmAWE.pl
Configure your ~/.bashrc and create ~/.bulkvm as explained below.
Please defined the following environment variables, or modify ManageBulkInstances.pm accordingly:
OS_USERNAME=
OS_PASSWORD=
OS_AUTH_URL=
OS_TENANT_NAME= or OS_TENANT_ID=
Afterwards, do not forget to update the current shell:
. ~/.bashrc
When you start VMs, you can specify some default options that are used when you create new VMs, e.g. default ssh key_name, or default image you want.
~/.bulkvm
key_name=xyz
image_name=Ubuntu Precise 12.04 (Preferred Image)
Options provided at command line have higher priority of course. Note that your private key file should have the same name as the key_name in Openstack and should be located in your ~/.ssh directory. In this example it would be ~/.ssh/xyz or ~/.ssh/xyz.pem .
Test configuration, e.g. with vmAWE.pl:
vmAWE.pl --info
This modules containes functions to start multiple instances and attach volumes, if needed. It uses the OpenStack JSON-based API.
This module contains functions to communicate with the VMs via ssh and scp within a threaded environment. For each VM there is one thread that is responsible for the communication. Most important functionality is the execution of commands on the remote VMs. Long running jobs can be executed within a screen environment, which allows to detach from the VM and periodically check if the job is still running. A nice and fancy feature is the execution of small perl functions directly on the remote VM, without the need to create a perl script and scp'ing to the VM.
This example is based on vmAWE.pl which is now maintained by Wei and can be found here:
https://github.com/wtangiit/vmScriptAWE
The first four option groups are generic, they are inherited from the ManageBulkInstances.pm module. Only the last two option groups (AWE actions and AWE options) are AWE-specifc options that are defined in the vmAWE.pl script.
options of vmAWE.pl:
Nova actions:
--create=i create i new instances from snapshot/image
--delete use with --group,ipfile or iplist
--info list all instances, volumes, flavors...
--listgroup=s list all instances in this group (must be owner)
--savegroup=s save group in ipfile
VM actions:
--sshtest try to ssh all instances
Create options:
--flavor_name=s optional, use with --create
--image=s image ID, use with --create
--image_name=s image name, use with action --create
--key_name=s required, key_name as in Openstack
--groupname=s optional, name of the new group
--nogroupcheck optional, use this to add VMs to existing group
--onlygroupname optional, instance names all equal groupname
--owner=s optional, metadata information on VM, default os_username
--noownercheck optional, disables owner check
--disksize=i optional, in GB, creates, attaches and mounts volume
--wantip optional, external IP, only with count=1
--user-data=s optional, pass user data file to new instances
--saveIpToFile optional, saves list of IPs in file
Specify existing VMs for actions and deletion:
--group=s use VMs with this groupname (metadata-field)
--instance=s use single VMs with this instance name
--iplist=s@ list of ips, comma separated, use with --sshkey
AWE actions (independent, can be combinded):
--addscripts=s list of scripts, comma separated
--awecfg=s see --serverurl and --awegroup
--update installs or updates AWE client
--startawe
--stopawe
--restartawe
--command=s pass a command to all VMs, e.g. "df -h"
--example example for executing perl subroutine remotely
AWE options:
--serverurl=s optional, use with action --awecfg
--awegroup=s optional, use with actions --awecfg
Show tenant resources:
vmAWE.pl --info
Start instance from default image:
vmAWE.pl --create 5 --flavor_name idp.100 --groupname mygroup
List group:
vmAWE.pl --listgroup mygroup
Do something with group (script-specific, in this example the AWE client will be started on all VMs in the group "mygroup")
vmAWE.pl --group mygroup --startawe
Delete group:
vmAWE.pl --delete --group mygroup
Complicated AWE specific example (everything in one step: creates VMs, installs AWE clients, configures them, start clients):
vmAWE.pl --create 2 --key_name dmnewmagellanpub --awecfg awe.cfg --groupname MY_UNIQUE_NAME --awegroup MY_UNIQUE_NAME --update --startawe 2>&1 | tee vmawe.log