Improvements to workers' deployment documentation

worker/deploy/README.md

@@ -1,47 +1,153 @@
-# Worker Node deployment
+# Worker Node Deployment
 
-Arrebol tasks are processed by the Workers. In a typical deployment, multiple Workers are deployed together in a single virtual machine, a `Worker Node`. As each task run in an isolated docker container, the Docker engine and other dependencies must have be installed and configured in the Worker Node. Below, we detail the configuration procedure.
+This document provides instructions for deploying worker nodes. It was designed to facilitate the deployment of worker nodes over multiple machines.
 
-## Requeriments
+## **Overview**
 
-Before the configuration and installation of Worker Node dependencies, each Worker Node virtual machine should be configured to be reached via SSH (using a rsa key pair). Also, the [Ansible](https://www.ansible.com/) automation tool should be installed in the deploy `coordination host`. This host is a machine that commands that deploy. To this end, the `coordination host` should be able to access the Worker Node through SSH.
+### _Worker Node_
 
-To install Ansible run these commands:
+**Worker Node** is the host where workers live. You need to deploy at least one worker node for Arrebol.
+
+### _Coordination Host_
+
+The **coordination host** is a machine from which scripts are executed to deploy worker nodes. The coordination host must have access to worker nodes via **SSH (using a rsa key pair)** to perform its function.
+
+![Worker Deployment](../../imgs/wd.png)
+
+## **Infrastructure**
+
+You will need at least two hosts: **a coordination host** and **a worker node**. At the end of the deployment, the coordination host is no longer needed.
+
+### _Worker Node_
+
+ #### Requirements
+
+ The requirements of the Worker Node depend a lot on the workload of the Jobs you intend to submit to Arrebol. Below is a minimum standard recommendation:
+
+ - vCPU: 2
+ - RAM: 2GB
+ - FREE DISK SPACE: 10GB
+ - OS: Ubuntu 16 or higher
+
+ #### Security Group
+
+ * Custom TCP Rule to allow ingress in the port 5555 (Docker API)
+ * Custom TCP Rule to allow ingress in the port 22 (SSH)
+
+### _Coordination Host_
+
+ #### Requirements 
+ - vCPU: 1
+ - RAM: 1GB
+ - FREE DISK SPACE: 10GB
+ - OS: Ubuntu 16 or higher
+
+ #### Security Group
+
+ * Custom TCP Rule to allow egress in the port 22 (SSH)
+
+## **Setup**
+
+### _1.Create RSA Key Pair_
+
+`Note: If you already have a key pair configured between the coordination host and the worker nodes go to Install dependencies.`
+
+Log into the coordination host, and then use command-line SSH to generate a key pair using the RSA algorithm.
+To generate RSA keys, on the command line, enter:
+
+```bash
+mkdir -p ~/.ssh
+cd ~/.ssh
+ssh-keygen -t rsa -N "" -f coordination_host_key
+chmod 600 coordination_host_key
+```
+If the commands were successful we will have two new files in the **~/.ssh** directory: 
+* **coordination_host_key**: The private key. It authenticates the coordination host. Ansible uses this key to access worker nodes.
+* **coordination_host_key.pub**: The public key. It authorizes the coordination host to access the worker node. Its content must be in the authorized keys of each worker node.
+
+To coordination host have authorization to access worker node via SSH the public key (**coordination_host_key.pub**) must be copied in an **authorized keys file** from worker node.
+
+Then, log into the worker node machine and create a copy of the public key as you see fit. The commands below write the public key into **authorized_keys** of worker node considering that it is in the home directory.
 ```bash
+mkdir -p ~/.ssh
+touch ~/.ssh/authorized_keys
+cat ~/coordination_host_key.pub >> ~/.ssh/authorized_keys
+```
+
+
+
+### _2.Install dependencies_
+
+Log in the coordination host and run the following commands to install dependencies.
+
+```
 sudo apt update
 sudo apt install software-properties-common
 sudo apt-add-repository --yes --update ppa:ansible/ansible
-sudo apt install -y ansible
+sudo apt install -y ansible git grep sed 
 ```
 
-`Note: The deployment requires versions newer than 14 for ubuntu.`
+### _3.Download the repository_
 
-## Configuration
+Log in the coordination host and run the below commands to download the arrebol repository.
 
-The `hosts.conf` configuration file should be edited to declare the Worker Node. See below how to edit it.
+```bash
+git clone -b feature/remote-worker https://github.com/ufcg-lsd/arrebol.git
+```
+
+### _4.Fill configuration file_
+
+Go to the directory _worker/deploy_ inside _arrebol_ directory.
 
-### Example
+```bash
+cd arrebol/worker/deploy
 ```
-worker_ip_1=10.30.1.36
-worker_ip_2=10.30.1.37
 
-remote_user=ubuntu
+Then, edit the `hosts.conf` file present in this directory, as follows.\
+ **All fields of the files must be filled**. See below how to edit it. 
 
-ansible_ssh_private_key_file=/home/admin/.ssh/priv_key
+#### Hosts configuration
 
-# Default
-ansible_files_path=./ansible-playbook
+File: host.conf
 ```
+# For each worker node write a field started by `worker_ip`
+# In this example, there will be 2 worker nodes.
+
+worker_ip_1=
+worker_ip_2=
 
-## Install
+remote_user=
+
+ansible_ssh_private_key_file=
+```
+
+| Field | Description |
+|:---------------------------------:|----------------|
+| **worker_ip** | The **worker node** addresses. For each ip create a field beginning with **_worker_ip_**. |
+| **remote_user** | The user name that should be used to access the **worker nodes** via ssh. |
+| **ansible_ssh_private_key_file** | The path to the **coordination host private key** that will be used to access worker nodes via ssh |
+
+Considering that this user name is **ubuntu**, your worker nodes are **10.30.1.1** and **10.30.1.2**, and the private key are in the **~/.ssh/coordination_host**. The content of the **host.conf** file would be:
+
+```
+worker_ip_1=10.30.1.1
+worker_ip_2=10.30.1.2
+
+remote_user=ubuntu
+
+ansible_ssh_private_key_file=~/.ssh/coordination_host
+```
+
+### _5.Install_
 
 After configuring the `hosts.conf` file, execute the `install.sh` script in the coordination host to setup the Worker Node.
 
  ```
  sudo bash install.sh
  ```
 
-## Check 
+## **Check**
+
 
 To verify whether the worker deployment was successful, one can submit below sample requests to the docker api.
 
@@ -50,12 +156,12 @@ To verify whether the worker deployment was successful, one can submit below sam
 `Note 2: If you use a cloud provider you may need to add a firewall rule.`
 
 ---
-Request
+### Request
 ```bash
 curl http://<worker-address>:5555/version
 ```
 
-Expected
+### Expected
 ```json
 {
  "Platform": {

worker/deploy/hosts.conf

@@ -1,9 +1,9 @@
+# For each worker node write a field started by `worker_ip`
 # worker_ip[anything]
 worker_ip_1=
 
+# The user name that should be used to access the worker nodes via ssh.
 remote_user=
 
+# The path to the private key that will be used to access worker nodes via ssh
 ansible_ssh_private_key_file=
-
-# Default
-ansible_files_path=./ansible-playbook

worker/deploy/install.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-readonly ANSIBLE_FILES_PATH_PATTERN="ansible_files_path"
+readonly ANSIBLE_HOSTS_FILE="./ansible-playbook/hosts"
 readonly PRIVATE_KEY_FILE_PATH_PATTERN="ansible_ssh_private_key_file"
 readonly DEPLOYED_WORKER_IP_PATTERN="worker_ip"
 readonly DEPLOY_WORKER_YML_FILE="deploy-worker.yml"
@@ -15,9 +15,6 @@ if [ -z "$MY_PATH" ] ; then
 fi
 
 HOSTS_CONF_FILE="${MY_PATH}/hosts.conf"
-
-ANSIBLE_FILES_PATH=$(grep "${ANSIBLE_FILES_PATH_PATTERN}" "${HOSTS_CONF_FILE}" | awk -F "=" '{print $2}')
-ANSIBLE_HOSTS_FILE="${ANSIBLE_FILES_PATH}/hosts"
 PRIVATE_KEY_FILE_PATH=$(grep "${PRIVATE_KEY_FILE_PATH_PATTERN}" "${HOSTS_CONF_FILE}" | awk -F "=" '{print $2}')
 
 # Clears the worker machine addresses