Cloud-Control version 1.0

  * Rundeck ready!
  * See CHANGELOG for more information.

CHANGELOG

@@ -0,0 +1,52 @@
+CHANGELOG
+=========
+
+1.0: 2014-10-01
+----------------
+New feature:
+- Upload "ready to use" Rundeck Jobs automatically.
+
+Updates:
+- Ditched RVM, using ruby package instead.
+- Rename "rundeck_scripts" directory to "lib".
+- The setup script will now create /srv/cloud-control and generate/copy all the files and directories needed.
+- Moved all the Ruby script to actual Rundeck Jobs.
+- Added "data_folder" variable to the setup script.
+- Removed unneeded HOME variable from ENV file.
+- Removed unneeded curb gem from Gemfile.
+- get_images.rb is now using the curb gem instead of a wget system call.
+- RUNDECK_SCRIPTS variable got renamed to LIB.
+- Generate Rundeck Jobs XMLs based on user environment.
+- build-essential added to packages list.
+- Add rundeck user to libvirts and kvm Unix groups.
+- The setup script will now fetch the current Trusty 14.04 LTS version from: https://cloud-images.ubuntu.com
+- Added .gitignore file.
+- Added CHANGELOG file.
+
+0.3: 2014-09-28
+----------------
+New features:
+- PostgreSQL backend support.
+- New vnc_port field added to backend DBs.
+
+Updates:
+- bzr package no needed anymore.
+- cloud-utils package added.
+- Removed CLOUD_UTILS variable.
+- source.include? "origin" to "img".
+
+Fixed:
+- generate_xml "Checking if there is at least one guest on the system". Logic fixed.
+
+0.2: 2014-09-25
+----------------
+New feature:
+- MySQL backend support.
+
+0.1: 2014-09-03
+----------------
+- First Cloud-Control commit!
+
+2014-08-21
+----------
+- Initial commit

Gemfile

@@ -1,5 +1,3 @@
 source 'https://rubygems.org'
 
-gem 'rubyzip'
-gem 'curb'
 gem 'colorize'

README.md

@@ -1,48 +1,54 @@
 Description
-===========
-
-Cloud-Control is a project using KVM and Rundeck that lets you create/start/shutdown/destroy virtual machines.
-You can choose to start a new virtual machine with an ISO or an Ubuntu Cloud image.
+---------------
 
+Cloud-Control is a Rundeck/KVM project that lets you create/start/shutdown/destroy virtual machines. You can choose to start a new virtual machine with an ISO or an Ubuntu Cloud image.
 
-Assumptions
+Requirements
 ===========
 
+
+Rundeck should be installed.
+
 KVM should already be working.
 
-kvm-ok should return:
-INFO: /dev/kvm exists
-KVM acceleration can be used
+    ubuntu@cbuisson:~$ kvm-ok
+    INFO: /dev/kvm exists
+    KVM acceleration can be used
 
-Rundeck should be working as well.
+Installation
+========
 
+You will need to execute the `setup` script to install all the required packages and gems.
+Cloud-Control is using RVM.
 
-Requirements
-============
+    ./setup
 
 Environment
------------
+-----------------
+
+Cloud-Control has been developed for Ubuntu Trusty 14.04 LTS.
 
 You will need to edit the "setup" file and add:
-  -A backend type (file OR MySQL)
-  -Start IP (i.e 192.168.0.1)
-  -End IP (i.e 192.168.0.100)
-  -Gateway IP (i.e 192.168.0.254)
+
+ - A backend type (flat file, MySQL or PostgreSQL)
+ - Start IP (i.e 192.168.0.1)
+ - End IP (i.e 192.168.0.100)
+ - Gateway IP (i.e 192.168.0.254)
 
 Cloud-Control will assign static IPs to the KVM guests.
 You need to specify an IP range for the guests and a gateway to route out.
 
 Templates
----------
+--------------
 
-SSH-KEY: You can add your public key to TEMPLATE-user-data.erb in the templates directory.
+ - SSH-KEY: You can add your public key to TEMPLATE-user-data.erb in the templates directory.
+ - netmask: Cloud-Control is setup to work on with Class C IPs, therefore the netwask is hard coded to: 255.255.255.0
 
-When an Ubuntu Cloud image is used to launch a new instance, it will get a static IP.
-ISO's on the other hand, will get a DHCP ip.
+When an Ubuntu Cloud image is used to launch a new instance, the vm will get a static IP.
 
+ISO's on the other hand, will get a DHCP ip.
 
-Setup
-=====
+Assumptions
+-----------------
 
-You will need to execute the "setup" script to install all the required packages and gems.
-Cloud-Control is using RVM.
+ - VMs will reach the internet trough the hypervisor via `br0`, edit the KVM guests' XML template to reflect your environment.

generate_scripts.rb

@@ -5,39 +5,53 @@
 start_ip=ARGV[1]
 end_ip=ARGV[2]
 gateway_ip=ARGV[3]
-mysql_password=ARGV[4]
+data_folder=ARGV[4]
+mysql_password=ARGV[5]
 pwd=`/bin/pwd`
 
-scripts=["delete_guest", "generate_static_ip", "generate_xml"]
-
 #Generate ENV file
 template = ERB.new(File.read("templates/ENV.erb"))
 xml_content = template.result(binding)
-File.open("ENV", "w") do |file|
+File.open("#{data_folder}/ENV", "w") do |file|
+  file.puts xml_content
+end
+
+#Generate generate_static_ip.rb file
+template = ERB.new(File.read("templates/generate_static_ip.erb"))
+xml_content = template.result(binding)
+File.open("#{data_folder}/lib/generate_static_ip.rb", "w") do |file|
+  file.puts xml_content
+end
+
+#Generate Rundeck XML
+template = ERB.new(File.read("templates/rundeck_jobs.xml.erb"))
+xml_content = template.result(binding)
+File.open("#{data_folder}/rundeck_jobs.xml", "w") do |file|
+  file.puts xml_content
+end
+
+system("sudo su rundeck -c 'rd-project -p cloud-control -a create'")
+system("sudo su rundeck -c 'rd-jobs load -r -f #{data_folder}/rundeck_jobs.xml -p cloud-control'")
+
+#Generate get_images.rb file
+template = ERB.new(File.read("templates/get_images.erb"))
+xml_content = template.result(binding)
+File.open("#{data_folder}/get_images.rb", "w") do |file|
   file.puts xml_content
 end
 
 #Generate user-data template file
 var = "<%= ip %>"
 template = ERB.new(File.read("templates/TEMPLATE-user-data.erb"))
 xml_content = template.result(binding)
-File.open("templates/TEMPLATE-user-data.erb", "w") do |file|
+File.open("#{data_folder}/templates/TEMPLATE-user-data.erb", "w") do |file|
   file.puts xml_content
 end
 
-guest=""
-eval File.read("ENV")
-#Generate files from ERB
-scripts.each do |s|
-  template = ERB.new(File.read("templates/"+s+".erb"))
-  xml_content = template.result(binding)
-  File.open("rundeck_scripts/"+s+".rb", "w") do |file|
-    file.puts xml_content
-  end
-  system("chmod 775 \"rundeck_scripts/\"#{s}\".rb\"")
-end
+system("sudo cp templates/TEMPLATE.xml.erb #{data_folder}/templates/")
+system("sudo chown -R rundeck. #{data_folder}")
 
-puts "\nAll the scripts were generated!\n Backend: #{backend}\n Ip Range: #{start_ip} to #{end_ip}"
+puts "\nAll the scripts were generated!\n Cloud-Control folder location: #{data_folder}\n Backend: #{backend}\n Ip Range: #{start_ip} to #{end_ip}"
 
 if backend == "mysql"
   begin

rundeck_scripts/def.rb → lib/def.rb

@@ -2,7 +2,7 @@
 
 def generate_value (value_name, search, separator, array_name)
 array_name=[]
-Dir.glob("#{KVM_DIR}/*/*.xml").each do |files|
+Dir.glob("#{GUESTS_DIR}/*/*.xml").each do |files|
   File.open(files, "r").each_line do |line|
     if line.include?(value_name)
       line.split(' ').each do |split|

setup

@@ -4,6 +4,7 @@ backend=""
 start_ip=""
 end_ip=""
 gateway_ip=""
+data_folder="/srv/cloud-control"
 mysql_password="cloudcontrol"
 
 if [ -z "$backend" ]; then echo -e "Please set a backend type for the VMs' data\n\n  Choose between: file, mysql OR postgres" && exit 1; fi
@@ -12,28 +13,28 @@ if [ -z "$gateway_ip" ]; then echo -e "Please set a gateway IP!\n" && exit 1; fi
 
 ##################
 
-RUBY_VERSION=2.1.1
-
-echo "Get RVM"
-curl -sSL https://get.rvm.io | bash
-source ~/.rvm/scripts/rvm
-type rvm | head -n 1
-echo "System and RVM requirements"
+echo "System requirements"
 sudo apt-get update
-sudo apt-get -y install libcurl3 libcurl3-gnutls libcurl4-openssl-dev qemu-utils cloud-utils
-rvm requirements
-echo "Installing: Ruby ${RUBY_VERSION}"
-rvm install ruby-$RUBY_VERSION
-rvm use ruby-$RUBY_VERSION
+sudo apt-get -y install build-essential ruby1.9 qemu-utils cloud-utils
 echo "Installing: bundler"
-gem install bundler --no-ri --no-rdoc
+sudo gem install bundler --no-ri --no-rdoc
 bundle install
 echo "Creating kvm_guests and sources folders"
-mkdir -p kvm_guests sources/{iso,cloud_images}
+sudo mkdir -p $data_folder/{kvm_guests,lib,templates,sources/{iso,cloud_images}}
+sudo chown -R rundeck. $data_folder
+sudo cp lib/* $data_folder/lib
+echo "Add rundeck user to libvirtd and kvm groups"
+sudo adduser rundeck libvirtd && sudo adduser rundeck kvm
+echo "Restart libvirtd..."
+sudo service libvirt-bin restart
+echo "Restart Rundeck...."
+sudo service rundeckd restart
+echo "Giving 20 seconds to Rundeck to fully restart, hang tight..."
+sleep 20
 
 if [[ $backend == "file" ]]; then
-  mkdir interfaces
-  touch interfaces/ip_address_pool.txt
+  sudo mkdir $data_folder/interfaces
+  sudo touch $data_folder/interfaces/ip_address_pool.txt
 elif [[ $backend == "mysql" ]]; then
   echo "mysql-server-5.5 mysql-server/root_password password $mysql_password
   mysql-server-5.5 mysql-server/root_password seen true
@@ -42,10 +43,10 @@ elif [[ $backend == "mysql" ]]; then
   " | sudo debconf-set-selections
   export DEBIAN_FRONTEND=noninteractive
   sudo apt-get install -q -y mysql-server mysql-client libmysqlclient-dev
-  gem install mysql2
+  sudo gem install mysql2
 elif [[ $backend == "postgres" ]]; then
   sudo apt-get -y install postgresql libpq-dev
-  gem install pg
+  sudo gem install pg
   sudo su - postgres -c "createuser pguser -s"
   echo -e "local all postgres peer\nlocal all pguser trust\nlocal all all peer\nhost all all 127.0.0.1/32 md5" | sudo tee /etc/postgresql/9.3/main/pg_hba.conf
   sudo service postgresql restart
@@ -55,4 +56,12 @@ else
 fi
 
 ruby=`which ruby`
-$ruby generate_scripts.rb $backend $start_ip $end_ip $gateway_ip $mysql_password
+sudo $ruby generate_scripts.rb $backend $start_ip $end_ip $gateway_ip $data_folder $mysql_password
+
+sudo $ruby $data_folder/get_images.rb https://cloud-images.ubuntu.com/trusty/current/trusty-server-cloudimg-amd64-disk1.img
+
+rundeck_url=`sudo cat /etc/rundeck/framework.properties |grep framework.server.url |awk '{print $3}'`
+echo -e "\n#############"
+echo "Cloud-Control installation completed!"
+echo "Go to: ${rundeck_url}/project/cloud-control/jobs"
+echo "######"

templates/ENV.erb

@@ -2,32 +2,31 @@ require 'rubygems'
 require 'erb'
 require 'securerandom'
 require 'colorize'
-require 'curb'
 require 'ipaddr'
 require 'fileutils'
 require 'tempfile'
 require 'fileutils'
 <% if backend == "mysql" %>require 'mysql2'<% elsif backend == "postgres" %>require 'pg'<% end %>
 
 #User environment
-HOME="<%= pwd.chomp %>"
 <% if backend == "mysql" %>MYSQL_PASSWORD="<%= mysql_password %>"<% end %>
 
 #KVM guest
-KVM_DIR=HOME+"/kvm_guests"
-GUEST=KVM_DIR+"/"+guest
+DATA_DIR="<%= data_folder %>"
+GUESTS_DIR=DATA_DIR+"/kvm_guests"
+GUEST=GUESTS_DIR+"/"+guest
 XML=GUEST+"/"+guest+".xml"
 SNAPSHOTS=GUEST+"/snapshots"
 USER_DATA=GUEST+"/user-data"
 KVM_GUEST_IMAGE=GUEST+"/image"
 
 #Misc variables
-<% if backend == "file" %>IP_FILE=HOME+"/interfaces/ip_address_pool.txt"<% end %>
-RUNDECK_SCRIPTS=HOME+"/rundeck_scripts"
-SOURCES=HOME+"/sources"
+<% if backend == "file" %>IP_FILE=DATA_DIR+"/interfaces/ip_address_pool.txt"<% end %>
+LIB=DATA_DIR+"/lib"
+SOURCES=DATA_DIR+"/sources"
 ISO=SOURCES+"/iso"
 CLOUD_IMAGES=SOURCES+"/cloud_images"
-TEMPLATES=HOME+"/templates"
+TEMPLATES=DATA_DIR+"/templates"
 
 #IP range
 START_IP="<%= start_ip %>"