Clustering works on IP but not on domain (with or without SSL)

[cesarandreslopez]

Describe the bug
I'm setting up a cluster with one edge server and one origin server. If I try to get a connection using the demo player to the origin server, it works fine, both using the server's IP (both ws:// and wss://), it works fine.

If I do the same on the edge server, it works fine if I use its IP in ws.

However, it doesn't work when using a domain name pointing to the edge IP, regardless if I use ws or wss.

To Reproduce
Steps to reproduce the behavior:
Server.xml is here: https://pastebin.com/KT2pBKWd
Edge XML is here: https://pastebin.com/0WGpp4uq

On trying to connect to the edge server using the domain:

ome-1  | [2024-04-03 22:07:33.132] I [SPRtcSig-t3333:10] HTTP.Server | http_server.cpp:149  | Client(<ClientSocket: 0x7f4e2f53b810, #25, Connected, TCP, Nonblocking, 172.28.0.7:33348>) is connected on *:3333
ome-1  | [2024-04-03 22:07:33.132] I [SPRtcSig-t3333:10] Signalling | rtc_signalling_server.cpp:316  | New client is connected: <ClientSocket: 0x7f4e2f53b810, #25, Connected, TCP, Nonblocking, 172.28.0.7:33348>
ome-1  | [2024-04-03 22:07:33.448] I [SPRtcSig-t3333:10] AccessController | access_controller.cpp:257  | AdmissionWebhooks queried http://api/stream/admission_request whether client 172.28.0.7:33348 could access ws://vsa.connectedworkers-ksa.com:3333/app/test1?token=123. (Result : Allow Elapsed : 7 ms)
ome-1  | [2024-04-03 22:07:33.761] I [SPRtcSig-t3333:10] Monitor | stream_metrics.cpp:114  | A new session has started playing #default#app/test1 on the WebRTC publisher. WebRTC(1)/Stream total(1)/App total(1)
ome-1  | [2024-04-03 22:07:33.761] I [SPRtcSig-t3333:10] ICE | ice_port.cpp:368  | Added session: 9 (ufrag: PLMI7s:vjWp)
ome-1  | [2024-04-03 22:07:34.124] I [SPICE-t3478:19] ICE | ice_port.cpp:606  | Turn client has connected : <ClientSocket: 0x7f4e28436210, #30, Connected, TCP, Nonblocking, 172.28.0.7:37234>
ome-1  | [2024-04-03 22:07:34.125] I [SPICE-t3478:19] ICE | ice_port.cpp:606  | Turn client has connected : <ClientSocket: 0x7f4e28436c10, #31, Connected, TCP, Nonblocking, 172.28.0.7:37250>
ome-1  | [2024-04-03 22:08:04.499] I [DQICETmout:13] ICE | ice_port.cpp:436  | Removed session(9) from ICEPort | ice_seesions_with_id count(0) ice_sessions_with_ufrag(0) ice_sessions_with_address_pair(0) 
ome-1  | [2024-04-03 22:08:04.499] I [DQICETmout:13] WebRTC Publisher | webrtc_publisher.cpp:650  | IcePort is disconnected. : (#default#app/test1/109) reason(6)
ome-1  | [2024-04-03 22:08:04.499] I [DQICETmout:13] WebRTC Publisher | webrtc_publisher.cpp:552  | Stop command received : #default#app/test1/109
ome-1  | [2024-04-03 22:08:04.499] W [DQICETmout:13] ICE | ice_port.cpp:527  | Agent [Unknow, 9] has expired
ome-1  | [2024-04-03 22:08:04.507] I [DQICETmout:13] AccessController | access_controller.cpp:153  | AdmissionWebhooks queried http://api/stream/admission_request whether client 172.28.0.7:33348 could access ws://vsa.connectedworkers-ksa.com:3333/app/test1?token=123. (Result : Allow Elapsed : 8 ms)
ome-1  | [2024-04-03 22:08:04.508] I [DQICETmout:13] Monitor | stream_metrics.cpp:136  | A session has been stopped playing #default#app/test1 on the WebRTC publisher. Concurrent Viewers[WebRTC(0)/Stream total(0)/App total(0)]
ome-1  | [2024-04-03 22:08:04.508] I [DQICETmout:13] Signalling | rtc_signalling_server.cpp:450  | Client is disconnected: HttpConnection(0x7f4e0f615210) : WebSocket <ClientSocket: 0x7f4e2f53b810, #25, Connected, TCP, Nonblocking, 172.28.0.7:33348> TLS(Disabled) (#default#app / test1, ufrag: local: PLMI7s, remote: vjWp)
ome-1  | [2024-04-03 22:08:04.573] I [SPRtcSig-t3333:10] HTTP.Server | http_server.cpp:216  | Client(<ClientSocket: 0x7f4e2f53b810, #25, Closed, TCP, Nonblocking, 172.28.0.7:33348>) has been disconnected by *:3333
ome-1  | [2024-04-03 22:08:04.917] I [SPICE-t3478:19] ICE | ice_port.cpp:620  | Turn client has disconnected : <ClientSocket: 0x7f4e28436210, #30, Disconnected, TCP, Nonblocking, 172.28.0.7:37234>
ome-1  | [2024-04-03 22:08:04.917] I [SPICE-t3478:19] ICE | ice_port.cpp:620  | Turn client has disconnected : <ClientSocket: 0x7f4e28436c10, #31, Disconnected, TCP, Nonblocking, 172.28.0.7:37250>

Expected behavior
It should work on a domain the same way it works on an IP on an edge server.

Server (please complete the following information):
Docker compose for both edge and origin.

OvenmediaEngijne version 0.16.5

Nginx to serve. Nginx for the origin server is here: https://pastebin.com/qZuDBk5R
Nginx for Edge is here: https://pastebin.com/123WnDSV

Player (please complete the following information):
Player is the regular online player available, using Google Chrome.

Additional context

Assumed that I would be facing issues due to nginx, but if so, not sure why it works on edge using the IP and on origin using both the IP and It's Origin Domain (with ws and was).

Variables in env file for OME are:

for edge:

OME_HOST_IP=0.0.0.0
OME_ORIGIN_PORT=9000
OME_RTMP_PROV_PORT=1935
OME_SRT_PROV_PORT=9999
OME_LLHLS_STREAM_PORT=3333
OME_WEBRTC_SIGNALLING_PORT=3333
OME_WEBRTC_TCP_RELAY_PORT=3478
OME_WEBRTC_CANDIDATE_PORT=10005-10009/udp

DEFAULT_ORIGIN_SERVER=# ORIGIN IP
DEFAULT_ORIGIN_PORT=9000

and for origin:

OME_HOST_IP=0.0.0.0
OME_ORIGIN_PORT=9000
OME_RTMP_PROV_PORT=1935
OME_SRT_PROV_PORT=9999
OME_LLHLS_STREAM_PORT=3333
OME_WEBRTC_SIGNALLING_PORT=3333
OME_WEBRTC_TCP_RELAY_PORT=3478
OME_WEBRTC_CANDIDATE_PORT=10000-10004/udp

I am assigning all needed ports in the docker compose files on both edge and origin as per the documentation.

Again, oddly enough this works well on edge using the IP of the edge machine, but not using the Domain associated to that IP

[bchah]

I'm not sure this is the exact solution to your problem but using the full domain name instead of wildcards (*) can help simplify ICE connectivity. I wonder if removing the wildcards from your config will help...

[getroot]

Since this does not appear to be a bug in OME, I moved it to discussion. If it is identified as a bug during discussion, I will move it back to the issue board.

I don't know how nginx is configured. Is nginx proxying in front of OME Origin and nginx also proxying in front of OME Edge?

I don't know much about nginx, but from a quick glance it looks like you are trying to proxy all OME ports through nginx. I understand that nginx can only proxy HTTP. Nginx will not be able to proxy WebRTC ICE, SRT, etc.

[cesarandreslopez]

For your information, it also works on the nginx proxy of all OME ports after the IP6 change you suggested. Thanks again!

[cesarandreslopez]

@getroot @bchah, thank you for your input. I've moved all ports out of Nginx and directly mounted them on the OME container in docker-compose for both edge and origin. I've had the same results. It works when I use the IP, but it doesn't work on the domain.

Origin works both on IP and domain.

Will continue to investigate further and report here.

[getroot]

Does the domain include an IPv6 address (AAAA record)? If so, you need to enable IPv6 in OME as well, or remove the AAAA record from the domain.

https://airensoft.gitbook.io/ovenmediaengine/configuration/ipv6

[cesarandreslopez]

That makes a lot of sense!

And yes, it worked right away! Thank you @getroot you and this project are awesome.