Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

423 rest api prototype #437

Merged
merged 60 commits into from
Sep 21, 2023
Merged
Changes from 1 commit
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
75c3b33
added exact versions for openstsacksdk and python-openstackclient (#413)
XaverStiensmeier May 25, 2023
769eb10
Keep master updated (#401)
XaverStiensmeier May 30, 2023
a660624
Added option to generate cluster_id before create process
XaverStiensmeier Aug 7, 2023
6da2ae3
Added rest api prototype
XaverStiensmeier Aug 7, 2023
3af6c65
reworked naming convention and added terminate command. Added basic r…
XaverStiensmeier Aug 8, 2023
bb9713c
Converter global LOG to class attribute self.log to enable different …
XaverStiensmeier Aug 9, 2023
60c7ca1
Reverted logging to global logging because using redirect might be mo…
XaverStiensmeier Aug 9, 2023
f4b3a5d
Using contextlib to redirect prints
XaverStiensmeier Aug 9, 2023
a4a547e
Started rewriting prints to logging and make logging not global and t…
XaverStiensmeier Aug 10, 2023
e26d1f7
Fixed list_clusters needing log now.
XaverStiensmeier Aug 10, 2023
70000b4
updated terminate.py and occurences to local logging.
XaverStiensmeier Aug 10, 2023
00f8f7b
changed logging to local for ansible configurator
XaverStiensmeier Aug 10, 2023
34169b4
unfinished: started localizing logging in logging_path_handler.py
XaverStiensmeier Aug 10, 2023
c88e6ad
updating ssh_handler.py now logging locally (and affected modules)
XaverStiensmeier Aug 11, 2023
1703c38
updating ssh_handler.py now logging locally (and affected modules)
XaverStiensmeier Aug 11, 2023
2feca2f
improved variable names
XaverStiensmeier Aug 11, 2023
62073f1
updated provider_handler.py to local logging
XaverStiensmeier Aug 11, 2023
5ec389f
changed global logging to local logging
XaverStiensmeier Aug 11, 2023
23b7bff
changed global logging to local logging
XaverStiensmeier Aug 11, 2023
e7bde51
Fixed many small logging mistakes and changed validation logging to l…
XaverStiensmeier Aug 11, 2023
471a155
Fixed formatting
XaverStiensmeier Aug 11, 2023
6717d04
Cleaned startup.py
XaverStiensmeier Aug 11, 2023
7bc1f83
Fixed logging error and made use of logging for all commands
XaverStiensmeier Aug 11, 2023
7ed1bda
Added cpu based worker selection
XaverStiensmeier Aug 16, 2023
a2f7d9a
Added new logging option 42 for "PRINT"
XaverStiensmeier Aug 16, 2023
8372bb1
Improved logger and added an explanation implementation
XaverStiensmeier Aug 16, 2023
bceac0e
Changed info to post and contains list now instead of single element
XaverStiensmeier Aug 17, 2023
7a589a4
Switched to main method.
XaverStiensmeier Aug 17, 2023
e713525
fixed many small things regarding log, added gateway mode for ssh_han…
XaverStiensmeier Aug 18, 2023
b9ff1aa
Enabled multiple subnets for when network is given. Not fully operati…
XaverStiensmeier Aug 21, 2023
6b3f22e
Fixed crash causing bug when using network instead of subnet
XaverStiensmeier Aug 22, 2023
667edcf
Removed unnecessary debug warning
XaverStiensmeier Aug 22, 2023
3fe12b9
made print nicer
XaverStiensmeier Aug 22, 2023
107642e
further fixed using network instead of subnet
XaverStiensmeier Aug 22, 2023
fc8474f
fixed issues regarding port calculation and gateway_ip
XaverStiensmeier Aug 23, 2023
b136f71
Added check wether a cluster is running
XaverStiensmeier Aug 23, 2023
878ec5a
removed prints
XaverStiensmeier Aug 23, 2023
96298be
removed prints
XaverStiensmeier Aug 23, 2023
0173865
Added comments for docs
XaverStiensmeier Aug 24, 2023
2c83017
Added pydantic base models
XaverStiensmeier Aug 24, 2023
6b71c1a
Capitalized names
XaverStiensmeier Aug 24, 2023
80a4db9
added option to terminate with assume_true
XaverStiensmeier Aug 25, 2023
e57ddf4
removed as docs fulfills this purpose now
XaverStiensmeier Aug 25, 2023
2a355e1
added option to not upload Credentials
XaverStiensmeier Aug 25, 2023
7b8e477
fixed minor bug causing bibigrid not finding private keys.
XaverStiensmeier Aug 29, 2023
a3b48b3
removed print
XaverStiensmeier Aug 29, 2023
87a5295
fixed name not being capitalized (ansible)
XaverStiensmeier Aug 29, 2023
793ff5b
fixed old linting error
XaverStiensmeier Aug 29, 2023
29a04d5
fixed old linting error
XaverStiensmeier Aug 29, 2023
a6a3b44
implemented gateway with portFunction using sympy
XaverStiensmeier Sep 18, 2023
4a458ca
using gateway automatically deactivates public ip usage now.
XaverStiensmeier Sep 18, 2023
9a2fee3
merged dev into 423-rest-api-prototype
XaverStiensmeier Sep 19, 2023
2b7d358
updated documentation
XaverStiensmeier Sep 20, 2023
5e5b151
update is now able to use gateway if given.
XaverStiensmeier Sep 20, 2023
ca9c82c
ide is now able to use gateway if given.
XaverStiensmeier Sep 20, 2023
26e48c5
new version correctly integrated
XaverStiensmeier Sep 20, 2023
1d5858d
removed unnecessary add to stdout (already standard)
XaverStiensmeier Sep 21, 2023
0bf89e0
removed unnecessary add to stdout (already standard) from startup_res…
XaverStiensmeier Sep 21, 2023
65c3ca2
if regex is found, check will succeed now.
XaverStiensmeier Sep 21, 2023
ea051d9
fixed ssh not using gateway
XaverStiensmeier Sep 21, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
fixed ssh not using gateway
XaverStiensmeier committed Sep 21, 2023
commit ea051d9a3f50b68e96858751be4746988b454e0d
12 changes: 11 additions & 1 deletion bibigrid/core/actions/create.py
Original file line number Diff line number Diff line change
@@ -9,6 +9,7 @@
from functools import partial

import paramiko
import sympy
import yaml

from bibigrid.core.actions import terminate
@@ -431,8 +432,17 @@ def log_cluster_start_info(self):
Detailed cluster info: How to log detailed info about the created cluster
:return:
"""
gateway = self.configurations[0].get("gateway")
ssh_ip = self.master_ip
port = None
if gateway:
octets = {f'oct{enum + 1}': int(elem) for enum, elem in enumerate(self.master_ip.split("."))}
port = int(sympy.sympify(gateway["portFunction"]).subs(dict(octets)))
ssh_ip = gateway["ip"]
self.log.log(42, f"Cluster {self.cluster_id} with master {self.master_ip} up and running!")
self.log.log(42, f"SSH: ssh -i '{KEY_FOLDER}{self.key_name}' {self.ssh_user}@{self.master_ip}")
self.log.log(42,
f"SSH: ssh -i '{KEY_FOLDER}{self.key_name}' {self.ssh_user}@{ssh_ip}"
f"{f' -p {port}' if gateway else ''}")
self.log.log(42, f"Terminate cluster: ./bibigrid.sh -i '{self.config_path}' -t -cid {self.cluster_id}")
self.log.log(42, f"Detailed cluster info: ./bibigrid.sh -i '{self.config_path}' -l -cid {self.cluster_id}")
if self.configurations[0].get("ide"):
4 changes: 2 additions & 2 deletions bibigrid/core/utility/handler/ssh_handler.py
Original file line number Diff line number Diff line change
@@ -108,9 +108,9 @@ def is_active(client, floating_ip_address, private_key, username, log, gateway,
if gateway:
log.info(f"Using SSH Gateway {gateway.get('ip')}")
octets = {f'oct{enum+1}': int(elem) for enum, elem in enumerate(floating_ip_address.split("."))}
port = sympy.sympify(gateway["portFunction"]).subs(dict(octets))
port = int(sympy.sympify(gateway["portFunction"]).subs(dict(octets)))
client.connect(hostname=gateway.get("ip") or floating_ip_address, username=username,
pkey=private_key, timeout=7, auth_timeout=5, port=int(port))
pkey=private_key, timeout=7, auth_timeout=5, port=port)
establishing_connection = False
log.info(f"Successfully connected to {floating_ip_address}")
except paramiko.ssh_exception.NoValidConnectionsError as exc:
Loading