Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SSH fails to connect after launchpad apply #96

Open
th3geekguy opened this issue Apr 3, 2024 · 8 comments
Open

SSH fails to connect after launchpad apply #96

th3geekguy opened this issue Apr 3, 2024 · 8 comments

Comments

@th3geekguy
Copy link

I receive the following errors and am unable to ever connect via SSH (~/.mirantis-launchpad/cluster/[cluster-name]/apply.log):

time="03 Apr 24 14:36 UTC" level=debug msg="preparing phase 'Check For Upgrades'"
time="03 Apr 24 14:36 UTC" level=info msg="\x1b[32m==> Running phase: Check For Upgrades\x1b[0m"
time="03 Apr 24 14:36 UTC" level=debug msg="phase 'Check For Upgrades' took 0s"
time="03 Apr 24 14:36 UTC" level=debug msg="preparing phase 'Open Remote Connection'"
time="03 Apr 24 14:36 UTC" level=info msg="\x1b[32m==> Running phase: Open Remote Connection\x1b[0m"
time="03 Apr 24 14:36 UTC" level=debug msg="[ssh] [sanitized]:22: using default known_hosts file ~/.ssh/known_hosts"
time="03 Apr 24 14:36 UTC" level=debug msg="[ssh] [sanitized]:22: failed to get ssh agent client: connect ssh agent: SSH_AUTH_SOCK is not set"
time="03 Apr 24 14:36 UTC" level=debug msg="[ssh] [sanitized]:22: checking identity file /terraTrain/key-pair"
time="03 Apr 24 14:36 UTC" level=debug msg="[ssh] [sanitized]:22: using known_hosts file from ssh config /terraTrain/.ssh/known_hosts"
time="03 Apr 24 14:36 UTC" level=debug msg="[ssh] [sanitized]:22: failed to get ssh agent client: connect ssh agent: SSH_AUTH_SOCK is not set"
time="03 Apr 24 14:36 UTC" level=debug msg="[ssh] [sanitized]:22: checking identity file /terraTrain/key-pair"
time="03 Apr 24 14:36 UTC" level=debug msg="[ssh] [sanitized]:22: using known_hosts file from ssh config /terraTrain/.ssh/known_hosts"
time="03 Apr 24 14:36 UTC" level=debug msg="[ssh] [sanitized]:22: failed to get ssh agent client: connect ssh agent: SSH_AUTH_SOCK is not set"
time="03 Apr 24 14:36 UTC" level=debug msg="[ssh] [sanitized]:22: checking identity file /terraTrain/key-pair"
time="03 Apr 24 14:36 UTC" level=debug msg="[ssh] [sanitized]:22: using an unencrypted private key from /terraTrain/key-pair"
time="03 Apr 24 14:36 UTC" level=debug msg="[ssh] [sanitized]:22: using an unencrypted private key from /terraTrain/key-pair"
time="03 Apr 24 14:36 UTC" level=debug msg="[ssh] [sanitized]:22: using an unencrypted private key from /terraTrain/key-pair"
time="03 Apr 24 14:36 UTC" level=debug msg="[ssh] [sanitized]:22: using known_hosts file from ssh config /terraTrain/.ssh/known_hosts"
time="03 Apr 24 14:36 UTC" level=debug msg="[ssh] [sanitized]:22: failed to get ssh agent client: connect ssh agent: SSH_AUTH_SOCK is not set"
time="03 Apr 24 14:36 UTC" level=debug msg="[ssh] [sanitized]:22: using cached auth method for /terraTrain/key-pair"
time="03 Apr 24 14:36 UTC" level=debug msg="[ssh] [sanitized]:22: using known_hosts file from ssh config /terraTrain/.ssh/known_hosts"
time="03 Apr 24 14:36 UTC" level=debug msg="[ssh] [sanitized]:22: failed to get ssh agent client: connect ssh agent: SSH_AUTH_SOCK is not set"
time="03 Apr 24 14:36 UTC" level=debug msg="[ssh] [sanitized]:22: using cached auth method for /terraTrain/key-pair"
time="03 Apr 24 14:36 UTC" level=debug msg="[ssh] [sanitized]:22: using known_hosts file from ssh config /terraTrain/.ssh/known_hosts"
time="03 Apr 24 14:36 UTC" level=debug msg="[ssh] [sanitized]:22: failed to get ssh agent client: connect ssh agent: SSH_AUTH_SOCK is not set"
time="03 Apr 24 14:36 UTC" level=debug msg="[ssh] [sanitized]:22: using cached auth method for /terraTrain/key-pair"
time="03 Apr 24 14:36 UTC" level=debug msg="[ssh] [sanitized]:22: using known_hosts file from ssh config /terraTrain/.ssh/known_hosts"
time="03 Apr 24 14:36 UTC" level=debug msg="[ssh] [sanitized]:22: failed to get ssh agent client: connect ssh agent: SSH_AUTH_SOCK is not set"
time="03 Apr 24 14:36 UTC" level=debug msg="[ssh] [sanitized]:22: using cached auth method for /terraTrain/key-pair"
time="03 Apr 24 14:38 UTC" level=debug msg="[SSH] [sanitized]: failed to connect: ssh dial: dial tcp [sanitized]:22: connect: connection timed out"
time="03 Apr 24 14:38 UTC" level=debug msg="[SSH] [sanitized]: failed to connect: ssh dial: dial tcp [sanitized]:22: connect: connection timed out"
time="03 Apr 24 14:38 UTC" level=error msg="[SSH] [sanitized]: attempt 1 of 60.. failed to connect: not connected: client connect: ssh dial: dial tcp [sanitized]:22: connect: connection timed out"
time="03 Apr 24 14:38 UTC" level=debug msg="[SSH] [sanitized]: failed to connect: ssh dial: dial tcp [sanitized]:22: connect: connection timed out"
time="03 Apr 24 14:38 UTC" level=debug msg="[SSH] [sanitized]: failed to connect: ssh dial: dial tcp [sanitized]:22: connect: connection timed out"
time="03 Apr 24 14:38 UTC" level=error msg="[SSH] [sanitized]: attempt 1 of 60.. failed to connect: not connected: client connect: ssh dial: dial tcp [sanitized]:22: connect: connection timed out"
time="03 Apr 24 14:38 UTC" level=error msg="[SSH] [sanitized]: attempt 1 of 60.. failed to connect: not connected: client connect: ssh dial: dial tcp [sanitized]:22: connect: connection timed out"
time="03 Apr 24 14:38 UTC" level=error msg="[SSH] [sanitized]: attempt 1 of 60.. failed to connect: not connected: client connect: ssh dial: dial tcp [sanitized]:22: connect: connection timed out"
time="03 Apr 24 14:38 UTC" level=debug msg="[SSH] [sanitized]: using known_hosts file from ssh config /terraTrain/.ssh/known_hosts"
time="03 Apr 24 14:38 UTC" level=debug msg="[SSH] [sanitized]: failed to get ssh agent client: connect ssh agent: SSH_AUTH_SOCK is not set"
time="03 Apr 24 14:38 UTC" level=debug msg="[SSH] [sanitized]: using cached auth method for /terraTrain/key-pair"
time="03 Apr 24 14:38 UTC" level=debug msg="[SSH] [sanitized]: using known_hosts file from ssh config /terraTrain/.ssh/known_hosts"
time="03 Apr 24 14:38 UTC" level=debug msg="[SSH] [sanitized]: failed to get ssh agent client: connect ssh agent: SSH_AUTH_SOCK is not set"
time="03 Apr 24 14:38 UTC" level=debug msg="[SSH] [sanitized]: using cached auth method for /terraTrain/key-pair"
time="03 Apr 24 14:38 UTC" level=debug msg="[SSH] [sanitized]: using known_hosts file from ssh config /terraTrain/.ssh/known_hosts"
time="03 Apr 24 14:38 UTC" level=debug msg="[SSH] [sanitized]: failed to get ssh agent client: connect ssh agent: SSH_AUTH_SOCK is not set"
time="03 Apr 24 14:38 UTC" level=debug msg="[SSH] [sanitized]: using cached auth method for /terraTrain/key-pair"
time="03 Apr 24 14:38 UTC" level=debug msg="[SSH] [sanitized]: using known_hosts file from ssh config /terraTrain/.ssh/known_hosts"
time="03 Apr 24 14:38 UTC" level=debug msg="[SSH] [sanitized]: failed to get ssh agent client: connect ssh agent: SSH_AUTH_SOCK is not set"
time="03 Apr 24 14:38 UTC" level=debug msg="[SSH] [sanitized]: using cached auth method for /terraTrain/key-pair"

Instances are up in Openstack and I am able to connect with key pair on my local machine -- but I should also be able to connect using my TerraTrain lab and configuration setup by launchpad 🤷

What am I missing? Thanks.
@mirantisbard
Copy link

mirantisbard commented Apr 3, 2024 via email

@th3geekguy
Copy link
Author

Ok, I checked and it looks fine (I'm also running as root within the container):

[root]-[e345a1b4870b]-[~]-[20:22-Wed Apr 03]
$ ll /terraTrain/.ssh/
total 12
drwx------ 2 root root 4096 Apr  3 20:19 ./
drwxr-xr-x 1 root root 4096 Apr  3 20:19 ../
-rw------- 1 root root    0 Apr  3 15:17 known_hosts

@th3geekguy
Copy link
Author

known_hosts is empty -- but I'm not sure that would block making a connection

@mirantisbard
Copy link

mirantisbard commented Apr 4, 2024 via email

@th3geekguy
Copy link
Author

No change:

$ ssh -i key-pair ubuntu@[lab_ip] -vvv
OpenSSH_7.6p1 Ubuntu-4ubuntu0.7, OpenSSL 1.0.2n  7 Dec 2017
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: resolving "lab_ip" port 22
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to lab_ip [lab_ip] port 22.
debug1: connect to address lab_ip port 22: Connection timed out
ssh: connect to host lab_ip port 22: Connection timed out

@mirantisbard
Copy link

mirantisbard commented Apr 4, 2024 via email

@james-nesbitt
Copy link
Collaborator

can you confirm something for me: are you running in an environment with ssh-agent running, with at least one key added, but not your ssh key? Rig has a known bug in this case caused by k0sproject/rig@0c0e311.

@kke
Copy link
Contributor

kke commented May 6, 2024

The issue title is SSH fails to connect after launchpad apply but the issue text says I receive the following errors and am unable to ever connect via SSH.

I'm not sure if I understood correctly that you have never been able to connect using ssh from that host at all? The title would suggest that launchpad broke something, but reading between the lines makes me think it never worked?

Maybe your client was blacklisted because of multiple failed connection attempts or perhaps there is some firewall. Both launchpad and ssh give connection timed out which should not result from problems with known_hosts file, agent or keys.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@kke @james-nesbitt @th3geekguy @mirantisbard