Use of ProxyCommand causes ssh session to never continue prompting for input #2253

phr34k · 2024-07-14T05:00:58Z

Prerequisites

Write a descriptive title.
Make sure you are able to repro it on the latest version
Search the existing issues.

Steps to reproduce

I've expierencing an issue with using ProxyCommand. I think I should be use proxy command to automate commands like setup tcp tunnels and then use ssh to connect to tcp tunnel, however when I use proxy command the ssh never continues to prompts for commands.

I have configured two windows computer (A and B) to both run sshd using the 9.5.0.p1-beta software, same versions. winget install Microsoft.OpenSSH.Beta
Machine (A) has sshd configured with public key authentication, running ssh localhost successfully logs me into an ssh session.
Machine (B) has normal ssh configured with private key, so that it uses public key authentication and can successfully login to machine a.
I've setup port forwarding on the router of machine a, so I can use ssh server@ipv6address with the public address to login across networks

Because of nat conditions a public address isn't always available, or rather said the port forwardings aren't always available, so want to use tcp tunnel to sshd to bypass these restrictions. The basic idea is that thirdparty software like ngrok and frp can be used create a public named end-point for the (tcp-)tunnel to sshd.

I want to use proxy command in ssh config to automate some of the commands involved by writing a simple script, that prepares the tunnel and then uses ssh to connect to the tunnel, there are myriad of examples of using proxy command directly calling ssh so I believe this should be rather standard use-case.

However at this point, the problems with begins. Because ssh also supports tunneling, I'll reduce the complexity and explain only using ssh to illustrate the differences of the commands side by side.:

Use ssh -N -L 127.0.0.1:2222:127.0.0.1:22 server@ipv6address to create the tunnel, any connection on port 2222 will be forwarded to sshd on the remote host.
ssh [email protected] -p 2222, using this command the ssh session works like expected
ssh -o "ProxyCommand=ssh -v -o User=server 127.0.0.1 -p 2222" 127.0.0.1 using this command the ssh session hangs indefinitely

Judging about what happens on command terminal, it seems like it's actually reading ssh output and scanning it as 'banner' https://github.com/PowerShell/openssh-portable/blob/661803c9ec4d7dee6574eb6ff0c85b2b7006edb1/kex.c#L1609

Expected behavior

# command should prompt for command input,  the same way as `ssh [email protected] -p 2222
PS > ssh -o "ProxyCommand=ssh -v -o User=server 127.0.0.1 -p 2222" 127.0.0.1

Microsoft Windows [Version 10.0.19045.4651]
(c) Microsoft Corporation. All rights reserved.

server@DESKTOP-IVJK79E C:\Users\server>

Actual behavior

# never prompt for command input,  the like `ssh [email protected] -p 2222` does do, seems to interpret inner-ssh output as banner
PS > ssh -v -o "ProxyCommand=ssh -v -o User=server 127.0.0.1 -p 2222" 127.0.0.1 

OpenSSH_for_Windows_9.5p1, LibreSSL 3.8.2
debug1: Reading configuration data $UserProfile/.ssh/config
debug1: Reading configuration data $UserProfile/.ssh/config-citra
debug1: Executing proxy command: exec ssh -q -v -o User=server 127.0.0.1 -p 2222
debug1: identity file $UserProfile/.ssh/id_rsa type 0
debug1: identity file $UserProfile/.ssh/id_rsa-cert type -1
debug1: identity file $UserProfile/.ssh/id_ecdsa type -1
debug1: identity file $UserProfile/.ssh/id_ecdsa-cert type -1
debug1: identity file $UserProfile/.ssh/id_ecdsa_sk type -1
debug1: identity file $UserProfile/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file $UserProfile/.ssh/id_ed25519 type -1
debug1: identity file $UserProfile/.ssh/id_ed25519-cert type -1
debug1: identity file $UserProfile/.ssh/id_ed25519_sk type -1
debug1: identity file $UserProfile/.ssh/id_ed25519_sk-cert type -1
debug1: identity file $UserProfile/.ssh/id_xmss type -1
debug1: identity file $UserProfile/.ssh/id_xmss-cert type -1
debug1: identity file $UserProfile/.ssh/id_dsa type -1
debug1: identity file $UserProfile/.ssh/id_dsa-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_for_Windows_9.5
OpenSSH_for_Windows_9.5p1, LibreSSL 3.8.2
debug1: Reading configuration data $UserProfile/.ssh/config
debug1: Reading configuration data $UserProfile/.ssh/config-citra
Pseudo-terminal will not be allocated because stdin is not a terminal.
debug1: Connecting to 127.0.0.1 [127.0.0.1] port 2222.
debug1: Connection established.
debug1: identity file $UserProfile/.ssh/id_rsa type 0
debug1: identity file $UserProfile/.ssh/id_rsa-cert type -1
debug1: identity file $UserProfile/.ssh/id_ecdsa type -1
debug1: identity file $UserProfile/.ssh/id_ecdsa-cert type -1
debug1: identity file $UserProfile/.ssh/id_ecdsa_sk type -1
debug1: identity file $UserProfile/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file $UserProfile/.ssh/id_ed25519 type -1
debug1: identity file $UserProfile/.ssh/id_ed25519-cert type -1
debug1: identity file $UserProfile/.ssh/id_ed25519_sk type -1
debug1: identity file $UserProfile/.ssh/id_ed25519_sk-cert type -1
debug1: identity file $UserProfile/.ssh/id_xmss type -1
debug1: identity file $UserProfile/.ssh/id_xmss-cert type -1
debug1: identity file $UserProfile/.ssh/id_dsa type -1
debug1: identity file $UserProfile/.ssh/id_dsa-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_for_Windows_9.5
debug1: Remote protocol version 2.0, remote software version OpenSSH_for_Windows_9.5
debug1: compat_banner: match: OpenSSH_for_Windows_9.5 pat OpenSSH* compat 0x04000000
debug1: Authenticating to 127.0.0.1:2222 as 'server'
debug1: load_hostkeys: fopen $UserProfile/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen __PROGRAMDATA__\\ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen __PROGRAMDATA__\\ssh/ssh_known_hosts2: No such file or directory
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ssh-ed25519 SHA256:7FnSP7rQxDSHXEbDj5LrFoY5Xl50EoysnR2qCxXoKHs
debug1: load_hostkeys: fopen $UserProfile/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen __PROGRAMDATA__\\ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen __PROGRAMDATA__\\ssh/ssh_known_hosts2: No such file or directory
debug1: Host '[127.0.0.1]:2222' is known and matches the ED25519 host key.
debug1: Found key in $UserProfile/.ssh/known_hosts:92
debug1: ssh_packet_send2_wrapped: resetting send seqnr 3
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: ssh_packet_read_poll2: resetting read seqnr 3
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: get_agent_identities: ssh_fetch_identitylist: agent contains no identities
debug1: Will attempt key: $UserProfile/.ssh/id_rsa RSA SHA256:wYxuNzUb36c65V4Dn4cjOUsdrJwIvgBogxE2lXmQNck
debug1: Will attempt key: $UserProfile/.ssh/id_ecdsa
debug1: Will attempt key: $UserProfile/.ssh/id_ecdsa_sk
debug1: Will attempt key: $UserProfile/.ssh/id_ed25519
debug1: Will attempt key: $UserProfile/.ssh/id_ed25519_sk
debug1: Will attempt key: $UserProfile/.ssh/id_xmss
debug1: Will attempt key: $UserProfile/.ssh/id_dsa
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com,webauthn-sk-ecdsa-sha2-nistp256@openssh.com,ssh-dss,ssh-rsa,rsa-sha2-256,rsa-sha2-512>
debug1: kex_ext_info_check_ver: publickey-hostbound@openssh.com=<0>
debug1: kex_ext_info_check_ver: ping@openssh.com=<0>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Next authentication method: publickey
debug1: Offering public key: $UserProfile/.ssh/id_rsa RSA SHA256:wYxuNzUb36c65V4Dn4cjOUsdrJwIvgBogxE2lXmQNck
debug1: Server accepts key: $UserProfile/.ssh/id_rsa RSA SHA256:wYxuNzUb36c65V4Dn4cjOUsdrJwIvgBogxE2lXmQNck
Authenticated to 127.0.0.1 ([127.0.0.1]:2222) using "publickey".
debug1: channel 0: new session [client-session] (inactive timeout: 0)
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: pledge: filesystem
debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
debug1: client_input_hostkeys: searching $UserProfile/.ssh/known_hosts for [127.0.0.1]:2222 / (none)
debug1: client_input_hostkeys: searching $UserProfile/.ssh/known_hosts2 for [127.0.0.1]:2222 / (none)
debug1: client_input_hostkeys: hostkeys file $UserProfile/.ssh/known_hosts2 does not exist
debug1: client_input_hostkeys: host key found matching a different name/address, skipping UserKnownHostsFile update
debug1: pledge: fork
debug1: kex_exchange_identification: banner line 0: Microsoft Windows [Version 10.0.19045.4651]
debug1: kex_exchange_identification: banner line 1: (c) Microsoft Corporation. All rights reserved.
debug1: kex_exchange_identification: banner line 2:
sshd re-exec requires execution with an absolute path
debug1: kex_exchange_identification: banner line 3: server@DESKTOP-IVJK79E C:\\Users\\server>SSH-2.0-OpenSSH_for_Windows_9.5
debug1: kex_exchange_identification: banner line 4:

Error details

No response

Environment data

PS > $PSVersionTable

Name                           Value
----                           -----
PSVersion                      5.1.22621.3880
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.22621.3880
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1


PS > $PSVersionTable

Name                           Value
----                           -----
PSVersion                      5.1.19041.4648
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.19041.4648
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

Version

This problem seems to happen for both ssh bundled in git, and the one installed from this repository.

SSH-2.0-OpenSSH_for_Windows_9.5 <--> SSH-2.0-OpenSSH_for_Windows_9.5
SSH-2.0-OpenSSH_9.7 <--> SSH-2.0-OpenSSH_for_Windows_9.5

Visuals

Here's a small video clip of what happens https://github.com/user-attachments/assets/117a2ce0-e21a-4b57-94bb-200208869be4.

Extra information

To give some extra context, in my specific scenario, I am specifically intrested in proxy command because due to non-ssh based tcp tunnels, other features like ProxyJump for jump-servers aren't relevant to my use-case.

rustdesk-ssh.ps1

param ($RustdeskId, $IdentityFile, $User, $LocalPort, $RemotePort, $RemoteHost)
write-host "${RustdeskId} through ssh on ${User}@localhost:${LocalPort} <---> $User@${RemoteHost}:${RemotePort}"
C:\Program` Files\RustDesk\rustdesk.exe --port-forward $RustdeskId $LocalPort $RemoteHost $RemotePort
ssh server@127.0.0.1 -p $LocalPort -tt -v

~/.ssh/config

Host XXYYZZWWW
     ProxyCommand powershell D:\Workspaces\liquid\maintenancw\rustdesk-ssh.ps1 -IdentityFile ~/.ssh/id_rsa -User server -LocalPort 2222 -RemotePort 22 -RemoteHost 127.0.0.1 -RustdeskId XXYYZZWWW

In principle this would mean I could use ssh XXYYZZWW and make my life a whole lot easier.

The text was updated successfully, but these errors were encountered:

phr34k · 2024-07-14T05:01:49Z

Related to #2252 and https://bugzilla.mindrot.org/show_bug.cgi?id=3710

phr34k · 2024-07-15T05:58:51Z

It seems that proxy command pipes the actual network traffic over stdout/stdin, and this is actually by design, once using the correct approach everything is working as intended.

phr34k closed this as completed Jul 15, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use of ProxyCommand causes ssh session to never continue prompting for input #2253

Use of ProxyCommand causes ssh session to never continue prompting for input #2253

phr34k commented Jul 14, 2024 •

edited

Loading

phr34k commented Jul 14, 2024 •

edited

Loading

phr34k commented Jul 15, 2024

Use of ProxyCommand causes ssh session to never continue prompting for input #2253

Use of ProxyCommand causes ssh session to never continue prompting for input #2253

Comments

phr34k commented Jul 14, 2024 • edited Loading

Prerequisites

Steps to reproduce

Expected behavior

Actual behavior

Error details

Environment data

Version

Visuals

Extra information

phr34k commented Jul 14, 2024 • edited Loading

phr34k commented Jul 15, 2024

phr34k commented Jul 14, 2024 •

edited

Loading

phr34k commented Jul 14, 2024 •

edited

Loading