Skip to content

Commit

Permalink
Merge branch 'staging'
Browse files Browse the repository at this point in the history
  • Loading branch information
vktrrdk committed Sep 2, 2024
2 parents 3f6ba43 + abb169b commit 5bb2aa5
Show file tree
Hide file tree
Showing 55 changed files with 118 additions and 133 deletions.
2 changes: 1 addition & 1 deletion wiki/FAQ.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Make sure that you use the correct SSH key when accessing the machine.
If you have changed your SSH key since creating the machine, it will not be automatically changed on the machines already running. The connection to these VMs still requires the key that was set at the time of creation.
The private key is always used to connect to the machine.

You can find more about SSH keys [here](./simple_vm/keypairs.md#ssh-keys-and-sharing-access).
You can find more about SSH keys [here](./simple_vm/keypairs.md#ssh-keys-and-how-to-share-access).

Maintenance may be performed at the site where your machines are running. Occasionally, malfunctions also occur during regular service.

Expand Down
8 changes: 2 additions & 6 deletions wiki/Tutorials/Mosh/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,13 +29,9 @@ sudo apt install mosh
```

#### Install Mosh on your Virtual Machine
You have two ways of installing Mosh on your virtual machine:

1. By having it installed on vm creation, [see](../../simple_vm/Instance/create_instance.md#mosh-udp-ports).

2. Installing it manually:
Y
For this Tutorial we chose Ubuntu 18.04 LTS as the image running on our virtual machine.
First, connect to your VM. Information on how to connect can be cound on the [instance overview](../../simple_vm/Instance/instance_overview.md#8-how-to-connect) and on the [instance detail page](../../simple_vm/Instance/instance_detail.md#general-information)
First, connect to your VM. Information on how to connect can be cound on the [instance overview](../../simple_vm/Instance/instance_overview.md#further-details-and-actions) and on the [instance detail page](../../simple_vm/Instance/instance_detail.md#general-information)
In our case we need to use ssh:
```
ssh -i /path/to/your/ssh/private/key [email protected] -p 30024
Expand Down
44 changes: 21 additions & 23 deletions wiki/simple_vm/Cluster/cluster_detail.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,42 +9,40 @@ The detail page of your cluster offers more information about your cluster and i
The general information part shows you:

- The name of your cluster.
- The general status of your cluster. The status mainly depends on the status of your master node.
- The general status of your cluster.
- Who created your cluster and when.
- Which project your cluster belongs to.
- How you can connect to your cluster. The information differs depending on your configuration.
- The image and flavor of your master node.
- How you can connect to your cluster. The information differs depending on your configuration. It shows how to connect to the master node.

Further, you can take the following actions:

- Shutting your cluster off or restarting it, when it is shutdown
- Scaling-Up and Scaling-Down of the cluster as described for the [Cluster Overview](./cluster_overview.md) page. Same hods for the password generation
- Rename your cluster
- Delete your cluster by clicking `Delete Cluster`.

## Master node

![master_node_information](../img/cluster_detail/master_node.png)

The master node part shows you:

- The name of your master node.
- Who created your master node and when.
- Which project your master node belongs to.
- The Elixir ID of the node creator.
- The OpenStack ID of your master node

You can't delete your master node.

## Worker nodes

![worker_nodes_information](../img/cluster_detail/worker_nodes.png)

The worker nodes part shows you:

- The worker batches.
- The worker node names.
The worker nodes part shows you a listing of the worker nodes by batches.
In detail one can see:
- The relation of active workers to total workers for a batch
- The worker node names
- Who created the worker node and when.
- Which project your worker node belongs to.
- The Elixir ID of the node creator.
- The OpenStack ID of each worker node.
- The connection information for the worker nodes

Further, you can take actions as for each instance on the [instance-detail page](../Instance/instance_detail.md).


## Additional information

Further, you can take the following actions on each worker node:
Similar to the functions delivered on the instance detail page for single virtual machines one call also get information on the following aspects:

- Delete a worker node by clicking `Delete instance`.
- Volumes
- Installed Conda packages and corresponding logs
- User Management
- Cluster Event logs
21 changes: 7 additions & 14 deletions wiki/simple_vm/Cluster/cluster_overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,21 +21,19 @@ You can set how many clusters you want to see on a page and scroll through the p

![cluster_card](../img/cluster_overview/card_overview.png)

See the name, the status, the project, and the creator of the cluster.<br>
Click on the name to get to the [Detail page](cluster_detail.md).
## How to connect
See the name, the status, the project, and the creator of the cluster.
Also one gets shown the batches and the number of active machines per batch.
Click on the name or the corresponding icon to get to the [Detail page](cluster_detail.md).
### How to connect

![how_to_connect](../img/cluster_overview/how_to_connect.png)

Click `How to connect`.
A list with connection information opens depending on your cluster configuration.<br>
A list with connection information is shown below, depending on your cluster configuration.

???+ info "SSH"
The SSH command tells you how to connect to your master node.

## Scale-Up your cluster

To expand your cluster and add more workers, click `Scale-up`.
To expand your cluster and add more workers, click `Scale Cluster up`.

![scale-up](../img/cluster_overview/scale_up_modal.png)

Expand Down Expand Up @@ -111,9 +109,7 @@ recognizes the removed worker nodes.**
### Configure your cluster

Your have to reconfigure your master node to recognize the removed worker nodes.
A modal opens after removing the worker nodes, which shows you the necessary steps.

![successful_down_scaling](../img/cluster_overview/scale_down_successful.png)
A modal opens after removing the worker nodes, which shows you the necessary steps. The operation is similar to the upscaling-mechanism.

Connect to your master instance with SSH.<br>
Download the scaling script:
Expand Down Expand Up @@ -152,12 +148,9 @@ sudo /etc/init.d/slurmctld restart

To stop your cluster, click `Stop`. A modal opens, asking you to confirm your action.

![stop_cluster_modal](../img/cluster_overview/stop_modal.png)

## Delete your cluster

To delete your cluster, click `Delete`. A modal opens, asking you to confirm your action.

![delete_cluster_modal](../img/cluster_overview/delete_modal.png)

If you have volumes attached to a node of your cluster, they get detached but not deleted.
2 changes: 1 addition & 1 deletion wiki/simple_vm/Cluster/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ The instances are connected to each other and can thus exchange data.
<br>
In SimpleVM, you can add worker nodes to your cluster, called **up scaling**.
You can remove worker nodes from your cluster, called **down scaling**.
See the instructions [here](./cluster_overview.md#3-scale-up).
See the instructions [here](./cluster_overview.md#scale-up-your-cluster).

## Shared directories

Expand Down
9 changes: 9 additions & 0 deletions wiki/simple_vm/Cluster/new_cluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ A message informs you if you can't start a cluster. This can happen if:
- Ask an administrator of your project to start a cluster for you.
- Ask an administrator of your project to change the appropriate setting.

Basically, the handling of starting a new cluster is similar to that of a [single instance](../Instance/create_instance.md).

## Resource overview

![project_resource_overview](../img/new_cluster/resource_overview.png)
Expand Down Expand Up @@ -78,6 +80,9 @@ Each worker batch has its own number.
You can have as many batches as you have resources and flavors available.<br>
Click the batches tab to switch between batches.

Below one can see the new resource usage which results in the chosen Batch configurations.
To save a certain batch configuration and add another batch, one has to click on the "Save batch"-button that is next to the input for the number of workers of a certain flavor type.

### Select flavor and count for a worker batch

Select the flavor for a worker batch and set the count of wanted workers.
Expand All @@ -94,6 +99,10 @@ Select the flavor and worker count for this batch.

Click `Remove Worker Batch` to remove the selected worker batch from your cluster configuration.

## Grant access for members

Like for the start of a [single instance](../Instance/create_instance.md) one can also grant access to the machine for other members of the project the cluster gets started in.

## Start your cluster

![start_cluster](../img/new_cluster/start_cluster.png)
Expand Down
39 changes: 23 additions & 16 deletions wiki/simple_vm/Instance/create_instance.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,21 +23,21 @@ The available options are listed and explained below.

![create-instance-part1](../img/create_instance/create-instance-part1.png)

### 1. Selected project
### Selected project

This select shows the selected project for which you want to start a new virtual machine and only shows when
you belong to more than one project. In case you are only member of a single project, the corresponding project is selected automatically.

### 2. Used and allocated resources
### Used and allocated resources

This area shows the resources used in your project.

### 3. Name your virtual machine
### Name your virtual machine

Name your virtual machine here or generate a random name.
After starting your virtual machine, a unique ID appends to the name.

### 4. Flavor selection
### Flavor selection

Choose the flavor of your virtual machine.
Click on a tab to switch between flavor types or use the filter to search by name.
Expand All @@ -54,7 +54,7 @@ Therefore, you should use ephemeral storage for temporary data that often change
(e.g. cache, buffers, or session data) or data often replicated across your environment.
If you need to persist data from an ephemeral, create a backup on a volume.
See the [Best practices for data backup](../backup.md) wiki page for more information.<br>
Use [Volumes](#8-volumes) for data that must persist.
Use [Volumes](#volumes) for data that must persist.

???+ danger "Backup important data from an ephemeral"
Ephemeral storage is a fleeting storage.
Expand All @@ -64,42 +64,45 @@ Use [Volumes](#8-volumes) for data that must persist.
![create-instance-part2](../img/create_instance/create-instance-part2.png)


### 5. Image selection
### Image selection

Choose the image your virtual machine starts with.
An image includes the operating system and tool packages installed on your vm.<br>
You may choose between base images provided by de.NBI, pre-build images containing a Research Environment
provided by de.NBI, or one of your snapshots.
Click on a tab to switch between them or use the filter to search by name.<br>
For more information about images and snapshots, see [Images and Snapshots](../snapshots.md).
.
![create-instance-part3](../img/create_instance/create-instance-part3.png)

### 6. Research Environments

#### Research Environments

In case you have chosen an base image, you are able to choose Research Environment to be installed on your machine manually.
It will take longer than choosing an image with an preinstalled Research Environment.
Find more information on Browser-based Research Environments [here](../customization.md#research-environments).

![create-instance-part4](../img/create_instance/create-instance-part4.png)
### Additional settings

### 7. Conda tools
#### Conda tools

You may choose conda, bioconda, and anaconda tools, which install on your machine at launch.
To add a tool, you may filter by name, and click the green plus button.<br>
For more information, see the [customization wiki page](../customization.md#conda).

![create-instance-part5](../img/create_instance/create-instance-part5.png)
![conda-tools](../img/create_instance/conda_tools.png)



### 8. Volumes
#### Volumes

Create, attach, and mount a new volume or attach and mount an already existing volume to the machine.
When creating a new volume you can choose the name, the path it gets mounted to and a volume size in GB.
If you want to know more about volumes, see the [Volumes](../volumes.md) wiki page.
You will see a selection of volumes which are queued for creation and existing volumes you have chosen for attachment below.

![add_new_volume](../img/create_instance/new_instance_vol_new.png)
![add_existing_volume](../img/create_instance/new_instance_vol_ex.png)

### 9. Grant access for project members
#### Grant access for project members

Grant members of your project SSH access to your virtual machine.<br>
You can't grant access to members without an SSH key stored in the portal.
Expand All @@ -110,15 +113,19 @@ Each granted member can access your virtual machine with their respective privat
All users connected to your vm have the same permissions and don't have separate home directories.<br>
Only the person who initially started the machine can stop, restart, or delete it.

### 10. Summary and Start
![grant-access](../img/create_instance/add_users_to_vm.png)


### Summary and Start

At the end of the Create-Instance form you get an overview of all selections made by you.
When every necessary settings are given, you are able to initiate the start of the virtual machine by clicking on "Start instance".
You are otherwise informed of this by corresponding messages in the summary.
After a short time, the page redirects you to the [Instance Overview](./instance_overview.md) page.

![summary](../img/create_instance/new_instance_summary.png )

## Restrictions on the choice of resources
## Restrictions on the choice of resources

Certain resources, such as GPU flavors or high-memory flavors, are subject to certain restrictions.
This is due to the scarcity of these resources.
Expand Down
13 changes: 12 additions & 1 deletion wiki/simple_vm/Instance/instance_detail.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,9 +78,12 @@ You are able to copy the link, view the logs of the research environment setup a
Deleting the backend of the research environment will make the research environment unaccessible.
When problems with the access occur, a renewal of the backend might fix the problem.

## User Management
The user management enables you to grant and revoke access to the research environment.
To grant access, a user has to be a member of the project.

[users](../img/instance_detail/user_management.png )


???+ warning "Concurrent sessions"
This doesn't automatically enable concurrent sessions, i.e., your session terminates
Expand All @@ -93,4 +96,12 @@ To grant access, a user has to be a member of the project.
![conda](../img/instance_detail/detail_conda.png)

The conda tab shows which conda packages got installed on the machines during startup.
Below a log of the installation can be viewed and downloaded as a `PDF` or `.txt`-file
Below a log of the installation can be viewed and downloaded as a `PDF` or `.txt`-file


## Event logs

It is possible to display all actions that have changed the status of the machine in tabular form.
The event log shows actions performed by users, such as starting and stopping machines, as well as system actions (for example, when the data center shuts down the machines for maintenance work). A date and the person or system responsible for the action is specified.

![events](../img/instance_detail/details_events.png)
20 changes: 1 addition & 19 deletions wiki/simple_vm/customization.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,25 +24,7 @@ Filter by name to find the tools you need.<br>
Click on the green plus sign to add the tool or click the red minus sign to remove the tool from your
selection of tools.<br>

![conda_table_with_selections](./img/create_instance/new_instance_conda.png)

After choosing your tools and packages, you need to read and confirm the proceeding installation process.

![conda_confirmation_needed](./img/create_instance/new_instance_okay.png)

### Installation process

??? info "Minoconda3"
Miniconda3 will install, which is a lightweight package manager.
We decided against the full Anaconda set of tools due to its size with over 1500 packages.<br>
If you wish to install Anaconda, you will have to choose it from the list of packages.

We create a temporary private and public key pair to access your virtual machine at launch, and to
install your selection with [Ansible](https://docs.ansible.com/ansible/latest/index.html).
At the end of the installation process we remove the temporary key from your vm and copy your public key onto it,
regardless of whether the installation of your selection succeeds or fails.<br>
You may download the installation logs as a PDF or TXT file. See [Instance Detail](./Instance/instance_detail.md#ansible-logs)
for more information.
![conda_table_with_selections](./img/create_instance/conda_tools.png)

### Environment

Expand Down
Binary file modified wiki/simple_vm/img/cluster_detail/general.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified wiki/simple_vm/img/cluster_detail/worker_nodes.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified wiki/simple_vm/img/cluster_overview/card_overview.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed wiki/simple_vm/img/cluster_overview/delete_modal.png
Binary file not shown.
Binary file modified wiki/simple_vm/img/cluster_overview/pagination.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified wiki/simple_vm/img/cluster_overview/password_modal.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified wiki/simple_vm/img/cluster_overview/scale_down_modal.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified wiki/simple_vm/img/cluster_overview/scale_successful.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified wiki/simple_vm/img/cluster_overview/scale_up_modal.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed wiki/simple_vm/img/cluster_overview/stop_modal.png
Binary file not shown.
Binary file modified wiki/simple_vm/img/create_instance/add_users_to_vm.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified wiki/simple_vm/img/create_instance/create-instance-part1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified wiki/simple_vm/img/create_instance/create-instance-part2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified wiki/simple_vm/img/create_instance/new_instance_vol_ex.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified wiki/simple_vm/img/create_instance/new_instance_vol_new.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified wiki/simple_vm/img/instance_detail/detail_conda.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified wiki/simple_vm/img/instance_detail/detail_general.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified wiki/simple_vm/img/instance_detail/detail_ports.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified wiki/simple_vm/img/instance_detail/detail_resenv.png
Binary file modified wiki/simple_vm/img/instance_detail/detail_volume.png
Binary file modified wiki/simple_vm/img/instance_detail/user_management.png
Binary file modified wiki/simple_vm/img/new_cluster/master_flavor.png
Binary file modified wiki/simple_vm/img/new_cluster/master_image.png
Binary file modified wiki/simple_vm/img/new_cluster/name.png
Binary file modified wiki/simple_vm/img/new_cluster/resource_overview.png
Binary file modified wiki/simple_vm/img/new_cluster/start_cluster.png
Binary file modified wiki/simple_vm/img/new_cluster/worker_flavor.png
Binary file modified wiki/simple_vm/img/project_overview_pub_key.png
Binary file modified wiki/simple_vm/img/snapshots/create_snapshot.png
Binary file modified wiki/simple_vm/img/snapshots/overview.png
Binary file modified wiki/simple_vm/img/snapshots/startsnap.png
Binary file removed wiki/simple_vm/img/volumes/create_and_attach.png
Diff not rendered.
Binary file removed wiki/simple_vm/img/volumes/extend_volume.png
Diff not rendered.
Binary file modified wiki/simple_vm/img/volumes/general.png
Loading

0 comments on commit 5bb2aa5

Please sign in to comment.