Installed Self-Signed Cert: Unable to Parse TLS Packet Header

[Tally]

I'm running connecting to Transmission in a LAN network (technically by VPN) and it works excellently with an insecure connection but attempting a secure connection fails and produces the following error after installing the server's intermediate or chain certificate under the checked "Server uses self-signed certificate" box. I have successfully installed this certificate onto several computers and the Android system, so I believe the certificate itself is not the issue.

The server is Caddy 2.6.2.

Error:
org.equeim.tremotesf.rpc.RpcRequestError$NetworkError: Network error when performing HTTP request

Caused by:
javax.net.ssl.SSLException: Unable to parse TLS packet header

I have a suspicion this is user error, and, if so, I apologize but I couldn't locate a more apt place to seek clarification. Thanks.

[equeim]

What Android version is the app running on?

[Tally]

Android 14 (March 1, 2024 security update, One UI 6.0)

Tremotesf 2.10.1 (F-Droid version)

It does connect to the server successfully if HTTPS is unchecked, and elsewhere the reverse proxy automatically upgrades to HTTPS, so perhaps it's unnecessary.

[equeim]

I just tried to reproduce it with Caddy and running Tremotesf on Android 14. I only needed to make sure that host/ip specified in Caddyfile is the same as one in Tremotesf settings, and load caddy's root certificate. It worked for me.

This error sounds like you are trying to connect to HTTP port instead of HTTPS port while HTTPS checkbox is on. I'm not quite sure how automatic http-to-https redirection should work with Tremotesf, at the very least it doesn't allow to specify self-signed certificate.

I.e. with following Caddyfile:

{
    http_port 8080
    skip_install_trust
    debug
}

192.168.0.12:8181 {
    reverse_proxy localhost:9091
}

It works if I connect to 192.168.0.12:8181 with HTTPS checkbox on and load caddy's root certificate.

Connecting to 192.168.0.12:8080 with HTTPS checkbox on doesn't work because it's a HTTP port and checkbox enforces direct HTTPS connection.

Connecting to 192.168.0.12:8080 with HTTPS checkbox off doesn't work either. It redirects to 8181 port but then fails because with checkbox off Tremotesf doesn't allow to specify self-signed certificate. I suppose that's a bug, since I had only direct HTTP or direct HTTPS connections in mind, without cross-protocol redirects.

[Tally]

Interesting. Thanks for investigating and for your prompt response. I'll work with my configuration further to see if I can resolve it. I have been installing the intermediate key, but all other keys have given "invalid configuration" errors.
FWIW, the relevant block of my Caddyfile is as follows:

devicename, devicename.lan, 192.168.0.123 {
  handle_path /other_service* {
    reverse_proxy localhost:8384
  }
  route /transmission* {
    redir /transmission/ /transmission
    reverse_proxy localhost:9091
  }
  tls internal
}

Connection through browser: