Get no RTCP packets in the sender

[j-licht]

I've tried to run sender and receiver on two different gentoo systems.
But the sender is always saying: "RTP queue discarded for SSRC"

By commenting out some debugging code in the sender plugin, I think the RTCP receive callback was never fired.
My workaround for now was to don't set the bind-port parameter of the udpsinks and -srcs.
As I see gstreamer sets reuse on the UDP sockets by default, so actually it should work.

[IngJohEricsson]

Hi
Did you get it working ?.
You will get those RTP queue discarded messages when the RTCP feedback is received, either IP addresses are not properly set, or ports are blocked.

My impression is that I needed to set bind-port and port to the same values to get the RTCP packets pass through a NAT and/or FW when using an LTE modem.
The ideal situation would be to have RTP and RTCP on the same port but I have so far not managed to make that work.

[j-licht]

I tried for now without NAT.

I used this for sending:
gst-launch-1.0 rtpbin name=rtpbin rtp-profile=avpf ! videotestsrc is-live=1 ! videoconvert ! video/x-raw,width=190,height=1080,framerate=30/1 ! x264enc name=video key-int-max=50 tune=4 vbv-buf-capacity=200 ! rtph264pay ! gscreamtx media-src=0 ! rtpbin.send_rtp_sink_0 rtpbin.send_rtp_src_0 ! udpsink host=169.254.15.230 port=5000 rtpbin.send_rtcp_src_0 ! udpsink host=169.254.15.230 port=5001 async=0 sync=0 udpsrc port=5001 ! rtpbin.recv_rtcp_sink_0

And this for receiving:
gst-launch-1.0 rtpbin name=rtpbin udpsrc port=5000 ! gscreamrx ! application/x-rtp,media=video,clock-rate=90000,encoding-name=H264 ! rtpbin.recv_rtp_sink_0 rtpbin. ! rtph264depay ! avdec_h264 ! videoconvert ! xvimagesink sync=false async=false udpsrc port=5001 ! rtpbin.recv_rtcp_sink_0 rtpbin.send_rtcp_src_0 ! udpsink port=5001 host=169.254.246.191 sync=false async=false

I dont't see the PTR1 debug output from

scream/code/gscream/gst-gscreamtx/gst-plugin/src/gstgscreamtx.cpp

Line 252 in 041da78

//g_print(" PTR1 %x %x %x\n", filter, filter_ , session);

, so therefore I think I got no RTCP data in the sender.

[IngJohEricsson]

Hi
I tried with the above both with videotestsrc and v4l2src it it seems to work. I ran the test between two Ubuntu 18 Laptops over a local network.

[j-licht]

Yes the both above work as well for me too.

But when I add the bind-port to the udpsink, which transports the rtcp in the sender, like this:
gst-launch-1.0 rtpbin name=rtpbin rtp-profile=avpf ! videotestsrc is-live=1 ! videoconvert ! video/x-raw,width=190,height=1080,framerate=30/1 ! x264enc name=video key-int-max=50 tune=4 vbv-buf-capacity=200 ! rtph264pay ! gscreamtx media-src=0 ! rtpbin.send_rtp_sink_0 rtpbin.send_rtp_src_0 ! udpsink host=169.254.15.230 port=5000 rtpbin.send_rtcp_src_0 ! udpsink host=169.254.15.230 bind-port=5001 port=5001 async=0 sync=0 udpsrc port=5001 ! rtpbin.recv_rtcp_sink_0
then I have the problem.

[IngJohEricsson]

Seems like I don't manage to recreate this problem,.. That said, I actually had problems getting this to work when I tested with a Sierra RV50X LTE modem, but it worked fine with an LTE dongle (the same sim card in both cases). I guess the best way to solve this is make RTP and RTCP mux over the same port, but I have not managed to get this working yet.
/I

[biswaranjanpanigrahi]

Was able to receive the RTCP Feedback packets with below sender and receiver. Running two separate instance on Ubuntu sitting in office env. If it is set to 5001 as suggested RTCP Feedback messages are not recevied

Sender: Here source Port I had updated to 5002

gst-launch-1.0 rtpbin name=rtpbin ! v4l2src device=/dev/video0 ! 'video/x-raw,width=640,height=480,framerate=30/1' ! videoconvert ! x264enc name=video tune=zerolatency ! rtph264pay ! gscreamtx media-src=0 ! rtpbin.send_rtp_sink_0 rtpbin.send_rtp_src_0 ! udpsink host=$RECEIVER_IP port=5000 bind-port=5000 rtpbin.send_rtcp_src_0 ! udpsink host=$RECEIVER_IP bind-port=5001 port=5001 udpsrc port=5002 ! rtpbin.recv_rtcp_sink_0

Receiver: Here Udp Sink port set to 5002
gst-launch-1.0 rtpbin name=rtpbin udpsrc port=5000 ! gscreamrx ! application/x-rtp,media=video,clock-rate=90000,encoding-name=H264 ! rtpbin.recv_rtp_sink_0 rtpbin. ! rtph264depay ! avdec_h264 ! videoconvert ! xvimagesink sync=false async=false udpsrc port=5001 ! rtpbin.recv_rtcp_sink_0 rtpbin.send_rtcp_src_0 ! udpsink port=5002 bind-port=5001 host=$SENDER_IP sync=false async=false

Would like to know how does the congestion window work in this case, size being passed during init e.g. 50000 (is it bytes?), I see the window size does not increase beyond 50000 but decrease even though the bandwidth between the two laptops connected over same Wifi is 12Mbps.

On an AVG the congestion window size remains around 15000 to 25000 (I believe this values are in bytes?)

The Bit rate passed in the register for new stream for x264enc is same as per the existing code set 1000000 (I believe its bps) i.e is around 1Mbps (as per targetBitrate = std::min(maxBitrate, std::max(minBitrate, startBitrate_));)
If I increase the value I see more packet loss as well as garbled video getting played in receiver.

So if we go as per Bit rate, Its sending out 1MB of data but the congestion window stays at very lower value.

Could you please help me to explain more on this algorithm?

[IngJohEricsson]

Hi The main problem is that the RTCP feedback is not received on the sender side. Thus the CWND will not increase, and even worse, no RTP packets will be transmitted because RTCP feedback is need to keep the self-clocking in SCReAM running. I understand that you are on a local network without a firewall. This should make it possible for you to try without the bind-port parameter. Can you try this instead ? Sender: gst-launch-1.0 rtpbin name=rtpbin ! v4l2src device=/dev/video0 ! videoconvert ! video/x-raw,width=1920,height=1080,framerate=30/1 ! x264enc name=video tune=zerolatency ! rtph264pay ! gscreamtx media-src=0 ! rtpbin.send_rtp_sink_0 rtpbin.send_rtp_src_0 ! udpsink host=$RECEIVER_IP port=5000 rtpbin.send_rtcp_src_0 ! udpsink host=$RECEIVER_IP port=5001 udpsrc port=5001 ! rtpbin.recv_rtcp_sink_0 Receiver: gst-launch-1.0 rtpbin name=rtpbin udpsrc port=5000 ! gscreamrx ! application/x-rtp,media=video,clock-rate=90000,encoding-name=H264 ! rtpbin.recv_rtp_sink_0 rtpbin. ! rtph264depay ! avdec_h264 ! videoconvert ! xvimagesink sync=false async=false udpsrc port=5001 ! rtpbin.recv_rtcp_sink_0 rtpbin.send_rtcp_src_0 ! udpsink port=5001 host=$SENDER_IP sync=false async=false Regards Ingemar PS: It should in gstreamer 1.16 be possible to multiplex RTP and RTCP on the same port, using the rtpfunnel element. This should solve the firewall issue, I have however not had the time to try it out. From: biswaranjanpanigrahi <[email protected]> Sent: den 2 maj 2019 19:25 To: EricssonResearch/scream <[email protected]> Cc: Ingemar Johansson S <[email protected]>; Comment <[email protected]> Subject: Re: [EricssonResearch/scream] Get no RTCP packets in the sender (#13) Was able to receive the RTCP Feedback packets with below sender and receiver. Running two separate instance on Ubuntu sitting in office env. If it is set to 5001 as suggested RTCP Feedback messages are not recevied Sender: Here source Port I had updated to 5002 gst-launch-1.0 rtpbin name=rtpbin ! v4l2src device=/dev/video0 ! 'video/x-raw,width=640,height=480,framerate=30/1' ! videoconvert ! x264enc name=video tune=zerolatency ! rtph264pay ! gscreamtx media-src=0 ! rtpbin.send_rtp_sink_0 rtpbin.send_rtp_src_0 ! udpsink host=$RECEIVER_IP port=5000 bind-port=5000 rtpbin.send_rtcp_src_0 ! udpsink host=$RECEIVER_IP bind-port=5001 port=5001 udpsrc port=5002 ! rtpbin.recv_rtcp_sink_0 Receiver: Here Udp Sink port set to 5002 gst-launch-1.0 rtpbin name=rtpbin udpsrc port=5000 ! gscreamrx ! application/x-rtp,media=video,clock-rate=90000,encoding-name=H264 ! rtpbin.recv_rtp_sink_0 rtpbin. ! rtph264depay ! avdec_h264 ! videoconvert ! xvimagesink sync=false async=false udpsrc port=5001 ! rtpbin.recv_rtcp_sink_0 rtpbin.send_rtcp_src_0 ! udpsink port=5002 bind-port=5001 host=$SENDER_IP sync=false async=false Would like to know how does the congestion window work in this case, size being passed during init e.g. 50000 (is it bytes?), I see the window size does not increase beyond 50000 but decrease even though the bandwidth between the two laptops connected over same Wifi is 12Mbps. On an AVG the congestion window size remains around 15000 to 25000 (I believe this values are in bytes?) The Bit rate passed in the register for new stream for x264enc is same as per the existing code set 1000000 (I believe its bps) i.e is around 1Mbps (as per targetBitrate = std::min(maxBitrate, std::max(minBitrate, startBitrate_));) If I increase the value I see more packet loss as well as garbled video getting played in receiver. So if we go as per Bit rate, Its sending out 1MB of data but the congestion window stays at very lower value. Could you please help me to explain more on this algorithm? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <https://protect2.fireeye.com/url?k=8ec108f5-d248a76f-8ec1486e-0cc47ad93e74-ba09f0d53f7a1d60&u=https://github.com/EricssonResearch/scream/issues/13#issuecomment-488759105> , or mute the thread <https://protect2.fireeye.com/url?k=b5b7e1e4-e93e4e7e-b5b7a17f-0cc47ad93e74-93fbfbec5e794634&u=https://github.com/notifications/unsubscribe-auth/ACRZ2GGBXZJ73Y2IOZADINTPTMPVXANCNFSM4HBVGWTQ> .

[biswaranjanpanigrahi]

Thanks for your response, with my changes mentioned earlier as well as with your suggestion (to remove bind port) was able to receive the RTCP FEEDBACK packet.

Log Snippet:

on_receiving_rtcp GST_RTCP_TYPE_RTPFB
RTP begin_seq= 4392 end_seq=4455
ScreamTx::updateCwnd maxBytesInFlight>5000 CWND: 18915

Code Snippet:

filter->screamTx = new ScreamTx(0.8f, 0.9f, 0.1f, false, 1.0f, 2.0f, 50000, 0.0f, 20, false, false);

As you can see the log snippet the cwnd has come down to 18915 where as it was set to 50000.

Question: Is Congestion Window not the total number of packets transmitted? but only count of packets that are not acked?

Could you please let me know how to get the total number of packets sent out? Is it same as the bitrate set? i.e 1Mbps as per the existing code.

Code Snippet:

switch (filter_->media_src) {
case 0:
filter->screamTx->registerNewStream(filter->rtpQueue, ssrc_h, 1.0f, 300e3f, 1000e3f, 15e6f, 5e6f, 0.3f, 0.2f, 0.1f, 0.2f);

Second question: Is when media source is 0 i.e x264enc why is the rate reduced? and set to the encoder in the filter. I would like to increase the bit rate from the sender side how could I achieve that?
And in case of congestion detected does the algorithm also optimize Bit Rate and Resolution? from the sender side.

switch (filter_->media_src) {
case 0: // x264enc
case 3: // vaapih264enc
rate /= 1000;
....

g_object_set(G_OBJECT(filter_->encoder), "bitrate", rate, NULL);
...

Third Question: At the receiver there is no action for the sender report (RTCP) packet from sender?

Best Regards
Biswa

[IngJohEricsson]

Hi Good to hear that removing bind-port was helpful. Answers to your questions and comments, hope that it helps, otherwise please don’t hesitate to ask. The congestion window is the maximum allowed number of bytes in flight, i.e transmitted but not yet acknowledged. There are no dedicated counters for the total number of packets sent out, but it should quite straightforward to add these counters. The intention with the SCReAM congestion control is to adjust the video coder bitrate up and down depending on the link capacity. Because the bitrate is set differently for different encoders (x264, omxh264…) and have different units (bps or kbps) it is necessary to treat each case differently. There is no special receiver handling for RTCP packets received on the receiver side. SCReAM is a completely sender side based algorithm. /Ingemar From: biswaranjanpanigrahi <[email protected]> Sent: den 2 maj 2019 21:50 To: EricssonResearch/scream <[email protected]> Cc: Ingemar Johansson S <[email protected]>; Comment <[email protected]> Subject: Re: [EricssonResearch/scream] Get no RTCP packets in the sender (#13) Thanks for your response, with my changes mentioned earlier as well as with your suggestion (to remove bind port) was able to receive the RTCP FEEDBACK packet. Log Snippet: on_receiving_rtcp GST_RTCP_TYPE_RTPFB RTP begin_seq= 4392 end_seq=4455 ScreamTx::updateCwnd maxBytesInFlight>5000 CWND: 18915 Code Snippet: filter->screamTx = new ScreamTx(0.8f, 0.9f, 0.1f, false, 1.0f, 2.0f, 50000, 0.0f, 20, false, false); As you can see the log snippet the cwnd has come down to 18915 where as it was set to 50000. Question: Is Congestion Window not the total number of packets transmitted? but only count of packets that are not acked? Could you please let me know how to get the total number of packets sent out? Is it same as the bitrate set? i.e 1Mbps as per the existing code. Code Snippet: switch (filter_->media_src) { case 0: filter->screamTx->registerNewStream(filter->rtpQueue, ssrc_h, 1.0f, 300e3f, 1000e3f, 15e6f, 5e6f, 0.3f, 0.2f, 0.1f, 0.2f); Second question: Is when media source is 0 i.e x264enc why is the rate reduced? and set to the encoder in the filter. I would like to increase the bit rate from the sender side how could I achieve that? And in case of congestion detected does the algorithm also optimize Bit Rate and Resolution? from the sender side. switch (filter_->media_src) { case 0: // x264enc case 3: // vaapih264enc rate /= 1000; .... g_object_set(G_OBJECT(filter_->encoder), "bitrate", rate, NULL); ... Third Question: At the receiver there is no action for the sender report (RTCP) packet from sender? Best Regards Biswa — You are receiving this because you commented. Reply to this email directly, view it on GitHub <https://protect2.fireeye.com/url?k=59ecea65-0565316a-59ecaafe-0cc47ad93e1a-f29f26df1073a18d&u=https://github.com/EricssonResearch/scream/issues/13#issuecomment-488807816> , or mute the thread <https://protect2.fireeye.com/url?k=6d3730b9-31beebb6-6d377022-0cc47ad93e1a-df468c4d0301ae5a&u=https://github.com/notifications/unsubscribe-auth/ACRZ2GDPXAECA5DR3URUZJDPTNAU5ANCNFSM4HBVGWTQ> .