Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial swag at VirtualBox support #5

Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
157 changes: 113 additions & 44 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,54 +1,68 @@
# vm-automation
## Introduction
The vm-automation repo was created to simplify interactions with virtual
machines. Specifically, this was built to support automated testing by
simplifying interaction with VMs. Currently, it supports VMWare
Workstation through the vmrun.exe command-line application and ESXi
through encapsulation of pyvmomi functions. My testing has used python
machines. Specifically, this was built to support automated testing by
simplifying interaction with VMs. Currently, it supports VMware
Workstation through the vmrun.exe command-line application and ESXi
through encapsulation of pyVmomi functions, as well as VirtualBox through
encapsulations of pyvbox functions. My testing has used python
2.7.

#### Note:
#### Note:
* This is a dependency for an in-development internal R7 tool. Some functions
might be modified in the near term with regard to parameters and return values
might be modified in the near term with regard to parameters and return values
to support that internal project.
* VMWare workstation support may be a bit behnd ESXi
* Currently, I have logging to the screen turned on to help with
* VMware Workstation support may be a bit behind ESXi
* Currently, I have logging to the screen turned on to help with
debugging. vm-automation logs to a file passed into the intialization
function for the server. The default name is `defaultLogfile.log`
### Why bother to encapsulate pyvmomi and vmrun.exe?

### Why bother to encapsulate pyVmomi and vmrun.exe?
I'm no big fan of re-inventing the wheel, and I conceed that is a bit of
what I did here, but I did it for some very good reasons:
* Using this library allows me to seamlessly manage VMware workstation
VMs and VMWare ESXi VMs because each server type has a class, and I
overloaded the management functions to work with both classes.<BR>
* Pyvmomi is not particularly simple. If you do not believe me, see
the `uploadFileToGuest` function. It contains the pyvmomi calls to
* Using this library allows me to seamlessly manage VMware Workstation
VMs and VMware ESXi VMs because each server type has a class, and I
overloaded the management functions to work with both classes.
* pyVmomi is not particularly simple. If you do not believe me, see
the `uploadFileToGuest` function. It contains the pyVmomi calls to
upload a file to a guest OS. It's about 40 lines. Even worse is the
code to get a list of VM snapshots, which requires a recurive search.
Vm-automation encapsulates the pyvmomi code and adds error handling as
Vm-automation encapsulates the pyVmomi code and adds error handling as
well.

### Can I use it as a stand-alone solution?
Certainly. I created the library as a separate entity specifically
because I planned to reuse this library to support other projects.
because I planned to reuse this library to support other projects.
Right now, it has all the functions that I need to run automated payload
testing, but I'm all for adding on to support more projects.

### How do I use it?
* If you don't have python (2.7), crawl out from under the rock and install
it.
* Install [pyvmomi](https://pypi.python.org/pypi/pyvmomi) on your machine:
`pip install --upgrade pyvmomi`
* VMware Workstation and ESXi
* Install [pyVmomi](https://pypi.python.org/pypi/pyvmomi) on your machine:
`pip install --upgrade pyvmomi`
* Oracle VirtualBox
* Install the VirtualBox SDK (available from the usual [VirtualBox download page](https://www.virtualbox.org/wiki/Downloads))
* there are some [useful guides](http://www.cesareriva.com/how-to-install-the-virtualbox-sdk/) out there to get the SDK ZIP archive installed correctly
* Install [pyvbox](https://pypi.python.org/pypi/pyvbox) on your machine:
`pip install --upgrade pyvbox`
* Git this repo:
`git clone [email protected]:rapid7/vm-automation.git`

### What Hypervisors are supported?
Right now, just VMWare Workstation and ESXi (vSphere). I was sad to find out that the API calls this repo uses are only available on the paid version of ESXi, but we all gotta' eat, so I can't be too mad. Hopefully, in the future, we can get support up and running for something like Virtualbox.
### Which hypervisors are supported?
The following hypervisors are currently supported:

* VMware Workstation
* VMware ESXi (vSphere)
* *NOTE* the API calls this repo uses are only available on the paid version of ESXi, but we all gotta' eat, so I can't be too mad
* Oracle VirtualBox
* *NOTE* in addition to having VirtualBox installed, you'll need to also install the VirtualBox SDK, see note above

### How can I get started?
The fastest way to get started is to instantiate a server (it can be
either VMWare ESXi or Workstation). I need to add some documentation for Workstation, because I've been focused on ESXi so far.
The fastest way to get started is to instantiate a server (it can be either VMware ESXi or Workstation). I need to add some documentation for Workstation, because I've been focused on ESXi so far.

#### ESXi
```
tmoose@ubuntu:~/rapid7/vm-automation$ python
>>> import vm_automation
Expand All @@ -58,7 +72,7 @@ True
```
OK, yeah; I probably should not have made the port number a string....
Anyway, that `connect` got us a connection to the server which is stored
in the class and reused when we need to talk to the server. It also
in the class and reused when we need to talk to the server. It also
registered the connection for removal when our process exits.
Let's query the type of Server:
```
Expand All @@ -75,7 +89,7 @@ Now, we can print it:
```
>>> for vm in myserver.vmList:
... print vm.vmName
...
...
[APT] Windows XP Pro
[APT] Windows 7 x64
[APT] Ubuntu 16x64
Expand All @@ -84,16 +98,16 @@ Now, we can print it:
[APT] Windows 10x64 Pro
```
These VMs are of a custom vmObject type defined in the esxiVm.py file
As they are a class, you just get an object which is kind of a pain to
As they are a class, you just get an object which is kind of a pain to
reference. It might be easier to create a dictionary with the names and
objects as pairs:
```
>>> vmDic = {}
>>> for vm in myserver.vmList:
... vmDic[vm.vmName] = vm
...
...
```
Now, we can play with a given vm more easily:
Now, we can play with a given VM more easily:
```
>>> print vmDic['[APT] Windows 10x64 Pro'].isPoweredOn()
False
Expand All @@ -107,16 +121,16 @@ True
serverlog:[2017-04-04 15:32:50.859867] POWERING OFF [APT] Windows 10x64 Pro
serverlog:[2017-04-04 15:32:55.887729] DONE
True
>>>
>>>
```
More advanced features require you to authenticate with the VM itself:
```
>>> vmDic['[APT] Windows 10x64 Pro'].setUsername('username')
>>> vmDic['[APT] Windows 10x64 Pro'].setPassword('password')
```
I suggest you use the function `waitForVmsToBoot` before calling any
interactive VM functions. VMware tools gets in odd states during the
boot process, and that function will wait for VMWare tools to stabilize
interactive VM functions. VMware tools gets in odd states during the
boot process, and that function will wait for VMware tools to stabilize
and be ready to handle requests:
```
>>> myserver.waitForVmsToBoot([vmDic['[APT] Windows 10x64 Pro']])
Expand All @@ -131,7 +145,7 @@ Let's grab a process list:
True
>>> for proc in vmDic['[APT] Windows 10x64 Pro'].procList:
... print proc
...
...
0 [System Process] [System Process]
4 System System NT AUTHORITY\SYSTEM
264 smss.exe smss.exe NT AUTHORITY\SYSTEM
Expand Down Expand Up @@ -177,7 +191,7 @@ creating, and deleting snapshots:
serverlog:[2017-04-04 16:06:16.775557] FINDING SNAPSHOTS FOR [APT] Windows 10x64 Pro
>>> for snapshot in vmDic['[APT] Windows 10x64 Pro'].snapshotList:
... print snapshot
...
...
((vim.vm.SnapshotTree) {
dynamicType = <unset>,
dynamicProperty = (vmodl.DynamicProperty) [],
Expand Down Expand Up @@ -225,17 +239,17 @@ serverlog:[2017-04-04 16:06:16.775557] FINDING SNAPSHOTS FOR [APT] Windows 10x64
replaySupported = false
}, 'TURNED_OFF/TESTING_BASE')
```
I admit that was less than helpful because the snapshots are stored as
pyvmomi snapshot objects, but you can get the snapshot names:
I admit that was less than helpful because the snapshots are stored as
pyVmomi snapshot objects, but you can get the snapshot names:
```
>>> for snapshot in vmDic['[APT] Windows 10x64 Pro'].snapshotList:
... print snapshot[0].name
...
...
TURNED_OFF
TESTING_BASE
```
Even then, it is not much of an issue, as most of the snapshot functions
use snapshot names rather than the pyvmomi class variables.
use snapshot names rather than the pyVmomi class variables.
For example, let's look at creating a snapshot. (Turning off the VM
first makes it faster). There are lots of optional parameters for these
functions, but I assumed most common use-cases as the default values.
Expand All @@ -250,7 +264,7 @@ serverlog:[2017-04-04 16:17:02.679320] TAKING SNAPSHOT new_snapshot ON [APT] Win
serverlog:[2017-04-04 16:18:04.798396] FINDING SNAPSHOTS FOR [APT] Windows 10x64 Pro
>>> for snapshot in vmDic['[APT] Windows 10x64 Pro'].snapshotList:
... print snapshot[0].name
...
...
TURNED_OFF
TESTING_BASE
new_snapshot
Expand All @@ -266,14 +280,12 @@ True
serverlog:[2017-04-04 16:18:50.773661] FINDING SNAPSHOTS FOR [APT] Windows 10x64 Pro
>>> for snapshot in vmDic['[APT] Windows 10x64 Pro'].snapshotList:
... print snapshot[0].name
...
...
TURNED_OFF
TESTING_BASE
>>>
>>>
```



As a first-use, I implemented this library to support payload testing,
so the following methods are supported now:
server class:
Expand Down Expand Up @@ -315,12 +327,69 @@ In time, they may get moved:
* `setTestVm`
* `takeTempSnapshot`

#### VirtualBox
```
>>> import vm_automation
>>> myserver = vm_automation.virtualboxServer()
>>> myserver.enumerateVms()
>>> myserver.vmList
[<vm_automation.virtualboxVm.virtualboxVm instance at 0x1108417e8>, <vm_automation.virtualboxVm.virtualboxVm instance at 0x10f0f1dd0>, <vm_automation.virtualboxVm.virtualboxVm instance at 0x10f0f1b00>, <vm_automation.virtualboxVm.virtualboxVm instance at 0x10f0f1e18>, <vm_automation.virtualboxVm.virtualboxVm instance at 0x11083c998>, <vm_automation.virtualboxVm.virtualboxVm instance at 0x11083cd40>, <vm_automation.virtualboxVm.virtualboxVm instance at 0x11083cfc8>, <vm_automation.virtualboxVm.virtualboxVm instance at 0x11083ca70>, <vm_automation.virtualboxVm.virtualboxVm instance at 0x11083ca28>, <vm_automation.virtualboxVm.virtualboxVm instance at 0x11083c5f0>, <vm_automation.virtualboxVm.virtualboxVm instance at 0x11083c2d8>, <vm_automation.virtualboxVm.virtualboxVm instance at 0x11083c3b0>, <vm_automation.virtualboxVm.virtualboxVm instance at 0x11083c908>, <vm_automation.virtualboxVm.virtualboxVm instance at 0x11083c9e0>, <vm_automation.virtualboxVm.virtualboxVm instance at 0x11083cbd8>, <vm_automation.virtualboxVm.virtualboxVm instance at 0x11085f098>, <vm_automation.virtualboxVm.virtualboxVm instance at 0x11085f0e0>, <vm_automation.virtualboxVm.virtualboxVm instance at 0x11085f170>]
>>> for vm in myserver.vmList:
... print(vm.vmName)
...
Ubuntu 16.04 amd64
Windows 8.1 x86_64
Windows 7 Ultimate N SP1 (x86_64)
Windows XP Pro SP3 (x86)
Ubuntu 16.04 amd64 2
Ubuntu 16.04 amd64 3
Windows 7 Ultimate (x86)
Windows 2008 x64 SP1
Windows Server 2012 x64
Windows 10 Enterprise (x64)
Ubuntu 14.04 Desktop (amd64)
Ubuntu 16.04 amd64
Ubuntu 14.04 amd64 2
Ubuntu 14.04 amd64 3
Windows Server 2003
Windows Server 2003 2
Windows 7 Professional x64
kali 2016.2 amd64
>>> myserver.vmList[17].vmName
'kali 2016.2 amd64'
>>> myserver.vmList[17].isPoweredOn()
False
>>> myserver.vmList[17].isPoweredOff()
True
>>> myserver.vmList[17].powerOn()
serverlog:[2017-09-26 10:23:51.720436] POWERING ON kali 2016.2 amd64
>>> myserver.vmList[17].isPoweredOn()
False
>>> myserver.vmList[17].isPoweredOn()
True
>>> myserver.vmList[17].isPoweredOn()
True
>>> myserver.vmList[17].isPoweredOn()
True
>>> myserver.vmList[17].vmUsername = "root"
>>> myserver.vmList[17].vmPassword = "toor"
>>> sout, serr = myserver.vmList[17].runAuthenticatedVmCommand(["/bin/uname", "-a"])
>>> print sout
Linux kali-pbarry 4.12.0-kali2-amd64 #1 SMP Debian 4.12.13-1kali2 (2017-10-03) x86_64 GNU/Linux

>>> myserver.vmList[17].powerOff()
serverlog:[2017-09-26 10:24:04.007438] POWERING OFF kali 2016.2 amd64
>>> myserver.vmList[17].isPoweredOn()
False
>>> myserver.vmList[17].isPoweredOff()
True
```

### This is kind of cool; how can I help?
There are several ways to contribute:
* If there's something you'd like to be able to do with a virtual machine that's not supported, go for it!
* If there's a hypervisor you'd like supported, please feel free to start a library to support it; just please make sure you match the function names! I'd love to add support for virtualbox because it is free, but I do not know when I will get time, and I have not looked to how hard it would be.
* If there's a hypervisor you'd like supported, please feel free to start a library to support it; just please make sure you match the function names! I'd love to add support for VirtualBox because it is free, but I do not know when I will get time, and I have not looked to how hard it would be.
* If you want to add Unit testing to make sure that the functions are supported correctly across classes, you would be doing God's work.
* If you want to go back and catch the VMWare workstation library up to the ESXi library, that would help a lot, and it would just be making sure function names and return values are the same.
* If you want to go back and catch the VMware Workstation library up to the ESXi library, that would help a lot, and it would just be making sure function names and return values are the same.
* If you run through the examples above and one is wrong, correct it for quick contributing karma!


3 changes: 2 additions & 1 deletion setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
packages=['vm_automation'],
install_requires=[
'pyVmomi',
'pyvbox',
],
classifiers=[
'Development Status :: 4 - Beta',
Expand All @@ -22,5 +23,5 @@
'Programming Language :: Python :: 2.7',
],
platforms=['Windows', 'Linux', 'Solaris', 'Mac OS-X', 'Unix'],
keywords='vsphere vmware esx',
keywords='vsphere vmware esx virtualbox',
zip_safe=True)
1 change: 1 addition & 0 deletions vm_automation/__init__.py
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
from .esxiVm import esxiServer, esxiVm
from .workstationVm import workstationServer, workstationVm
from .virtualboxVm import virtualboxServer, virtualboxVm
Loading