Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support sendmmsg for UDP. #12

Open
winlinvip opened this issue Apr 18, 2020 · 2 comments
Open

Support sendmmsg for UDP. #12

winlinvip opened this issue Apr 18, 2020 · 2 comments

Comments

@winlinvip
Copy link
Member

See ossrs/srs#307 (comment)

Linux GSO,可以将多个UDP包延迟分包,提升性能,参考UDP GSO原理及应用

注意:Linux 4.18.0及以上的内核才支持GSO,SRS会自动检测内核版本,下面是在开启GSO后的包,从抓包上看和不开启GSO一样的。如下抓包所示。

rtc-plaintext-linux4-gso-ok.pcapng.zip

rtc-plaintext-multiple-slices-as-one-NALU.pcapng.zip

注意:如果强制在低版本内核上开启GSO,会有未知行为,如下抓包所示。

rtc-plaintext-linux3-gso-invalid.pcapng.zip

SRS新增了包性能分析的API:http://localhost:1985/api/v1/perf

可以使用工具来分析:./scripts/perf_gso.py http://localhost:1985/api/v1/perf

No GSO, Fragment at Source

不开启GSO,在接收包(Source)时将消息分成RTP包,统计数据如下:

image

Note: 可以看到RTP的包数目比RTMP多了1.6倍,如果每个包都要内核处理,性能会有较大影响。

No GSO, Fragment at Connection

不开启GSO,在发送包(Connection)时将消息分成RTP包,统计数据如下:

image

Note: 可以看到和前面差不多,在Source和Connection分包,对于包分布影响不大。

GSO, Fragment at Connection

开启GSO,在发送包(Connection)时将消息分成RTP包,统计数据如下:

image

Note: 可以看到开启GSO后,穿过内核的包数目比RTMP还要少,性能得到提升。实际上GSO并不会减少RTP包数,但能将通过内核的包减少,所以我们认为包数目变少了。

GSO, Larger FU-Payload

之前FU Payload的长度是1200,改成了1300,参考bfc70d64b91e07f4

image

修改后,IP包最大是1356字节,小于1500的MTU。从结果看,RTP包从1.56倍降低到1.49倍,GSO分包没有影响。

GSO, Padding Packets

Audio包比较多,有时候差异不大,比如有三个包:257 256 255,如果能加点padding,那么可以作为一个GSO包发送,参考c95a8517

image

image

image

从数据上看,开启padding(127)后,你将GSO包倍数从0.74降低到0.67,将效能从0.67提升到0.74。

Note:开启padding的负载不高,在千万分之N级别,因为不是每个包都增加127字节,而是只有在包开启padding能GSO时才会加padding。

Note: Padding是RTP标准协议,参考RTP Fixed Header FieldsPadding may be needed by some encryption algorithms with fixed block sizes or for carrying several RTP packets in a lower-layer protocol data

winlinvip added a commit that referenced this issue Apr 18, 2020
winlinvip added a commit that referenced this issue May 3, 2020
winlinvip added a commit that referenced this issue May 3, 2020
@winlinvip
Copy link
Member Author

Remove the sendmmsg because we are refining ST, for #19 and ossrs/srs#2188.

winlinvip referenced this issue in winlinvip/srs Feb 26, 2021
@winlinvip winlinvip reopened this Feb 26, 2021
winlinvip added a commit that referenced this issue Apr 5, 2021
@winlinvip
Copy link
Member Author

目前的进展:sendmmsg和GSO都是能提升UDP IO的性能,但是RTC或UDP服务器,更多的性能损耗不是在IO,而是在内核协议栈,以及(由于是通信业务所以一般都要求)加解密,简单说来:

UDP服务器是一个计算和IO都很密集的业务,而且更侧重计算密集型,因为IO可以直接丢弃UDP包。

因此,对于计算密集型业务,我们首先要做的是能利用多核的能力,也就是多线程或多进程能力( #19 ),在多核改造完成后,再进行IO的优化。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant