Security Update and KCP Performance Update

README.md

@@ -169,6 +169,7 @@ encryption_algorithm=AES-GCM
 | stun_server  | STUN 服务器地址 |否|listen_port 为端口范围模式时不可使用|
 | log_path  | 存放 Log 的目录 |否|不能指向文件本身|
 | kcp_mtu  | 正整数 |否|预设值1440|
+| kcp_conserve  | yes<br>true<br>1<br>no<br>false<br>0 |否|使“快速ACK”稍微缓和|
 | kcp  | manual<br>fast1 - 6<br>regular1 - 5<br> &nbsp; |是|手动设置<br>快速<br>常速<br>(末尾数字：数值越小，速度越快)|
 | kcp_sndwnd  | 正整数 |否|预设值见下表，可以单独覆盖|
 | kcp_rcvwnd  | 正整数 |否|预设值见下表，可以单独覆盖|
@@ -218,13 +219,14 @@ encryption_algorithm=AES-GCM
 |  ----        | :----:     | :----:    | :----:    | :----:     | :----:   |:----: |
 | regular1     | 1024       |   1024    |      1    |   1        |   5      |   1   |
 | regular2     | 1024       |   1024    |      2    |   1        |   5      |   1   |
-| regular3     | 1024       |   1024    |      0    |   1        |   2      |   1   |
-| regular4     | 1024       |   1024    |      0    |   1        |   3      |   1   |
-| regular5     | 1024       |   1024    |      0    |   1        |   0      |   1   |
+| regular3*    | 1024       |   1024    |      1    |   1        |   5      |   1   |
+| regular4*    | 1024       |   1024    |      2    |   1        |   5      |   1   |
+| regular5*    | 1024       |   1024    |      0    |   1        |   3      |   1   |
 
 其中，丢包率越高（高于 10%），kcp_nodelay=1 就比 kcp_nodelay=2 越有优势。在丢包率不特别高的情况下，kcp_nodelay=2 可使延迟抖动更为平滑。
 
-如果想减少流量浪费、不介意延迟稍微增加，可以选择 regular 模式。需要注意的是，高延迟并且丢包率高于 4% 的环境下使用 regular4 或 regular5 可能会遇到 TCP 断流的情况。
+如果想减少流量浪费、不介意延迟稍微增加，可以选择 regular 模式。<br />
+标记了星号的模式 (regular3 ~ 5) 启用了 kcp_conserve 选项，丢包造成的延迟稍微高一些，浪费的流量稍微少一点。
 
 ### Log 文件
 在首次获取打洞后的 IP 地址与端口后，以及打洞的 IP 地址与端口发生变化后，会向 Log 目录创建 ip_address.txt 文件（若存在就覆盖），将 IP 地址与端口写进去。
@@ -397,6 +399,36 @@ make
 如果系统不支持 IPv6，或者禁用了 IPv6，请在配置文件中设置 ipv4_only=true，这样 kcptube 会退回到使用 IPv4 单栈模式。
 
 ## 其它注意事项
+### 多种系统都遇到的 Too Many Open Files
+#### GhostBSD
+一般情况下，绝大多数 BSD 系统都不会遇到这种事，只有 2023 年下半年更新后的 GhostBSD 才会遇到这种现象。
+
+这是因为 GhostBSD 在 `/etc/sysctl.conf` 当中加了这一行：
+```
+kern.maxfiles=100000
+```
+这一行缩减了上限，远低于原版 FreeBSD 的对应数值。
+
+解决办法很简单，删掉这一行即可。注释掉也可以。<br />
+还可以使用命令 `sysctl kern.maxfiles=300000` 临时修改上限值。
+
+#### Linux
+由于 Linux 系统的 Open Files 数量限制为 1024，所以很容易会遇到这种问题。
+
+临时解决办法：
+1. 运行命令 `ulimit -n`，查看输出的数值
+2. 如果数值确实只有 1024，请运行命令 `ulimit -n 300000`
+
+永久解决办法：<br />
+编辑 /etc/security/limits.conf，在末尾加上
+
+```
+*         hard    nofile       300000
+*         soft    nofile       300000
+root      hard    nofile       300000
+root      soft    nofile       300000
+```
+
 ### NetBSD
 使用命令
 ```
@@ -409,17 +441,20 @@ sysctl -w net.inet6.ip6.v6only=0
 ### OpenBSD
 因为 OpenBSD 彻底屏蔽了 IPv4 映射地址，所以在 OpenBSD 平台使用双栈的话，需要将配置文件保存成两个，其中一个启用 ipv4_only=1，然后在使用 kcptube 时同时载入两个配置文件。
 
-## 数据校验
+## 加密与数据校验
 由于需要传送 TCP 数据，因此数据校验是不可忽略的，正如 TCP 本身那样。
 
-如果已经使用了加密选项，那么就可以忽略本节内容。kcptube 选择的加解密算法已经附带验证能力，可以顺便保证传送内容不出错。
+无论是否加密，kcptube 都会将 MTU 缩小 2 个字节，尾附 2 字节的数据。
+
+如果已经使用了加密选项，那么尾附的 2 字节数据就是临时生成的IV。
 
-如果选择不使用加密功能，那么 kcptube 就会将 MTU 缩小 2 个字节，以便尾附 2 字节的校验码。
+如果选择不使用加密功能，那么尾附的 2 字节数据就是校验码，分别为两种 8-bit 校验码：
 
-然而 kcptube 使用的 Botan 库并不附带 16-bit 校验算法，因此 kcptube 同时使用了两种 8-bit 校验码：
 - 纵向冗余校验 (LRC, Longitudinal Redundancy Check)
 - 8-bit checksum
 
+这是因为 kcptube 使用的 Botan 库并不附带 16-bit 校验算法，因此 kcptube 同时使用了这两种 8-bit 校验码。
+
 这两种校验码的计算速度都足够快，简明又实用，并不是偏门的计算方式。例如 Modbus 就用到了 LRC。
 
 需要提醒的是，使用两种校验码仍然无法 100% 避免内容错误，TCP 本身也是一样。如果确实需要精确无误，请启用加密选项。
@@ -450,10 +485,11 @@ KCP Tube 虽然有“多路复用”的功能，但默认并不主动打开。
 为了降低延迟，kcptube 启用了 TCP_NODELAY 选项。对于某些大流量应用场景，可能会造成 TCP 数据传输量减少。
 
 ### KCP
-kcptube 用的是原版 [KCP](https://github.com/skywind3000/kcp)，作了些许修改：
+在原版 [KCP](https://github.com/skywind3000/kcp) 的基础上，作了以下修改：
 
-1. 原版的 `flush()` 是先把待发送数据转移到发送队列后，在同一个循环重做完“发送新数据包”、“数据包重发”、“ACK包发送”三件事。修改后的版本变为先做“数据包重发”、“ACK包发送”，然后再做“待发送数据转移到发送队列”，在转移期间顺便发送。
-2. 原版的 `check()` 每次都会重新遍历一遍发送队列，查找已到点的重传时间戳。修改后的版本变为直接在KCP结构体中新增 `min_resendts` 变量，该变量在 `flush()` 的发送循环当中顺便找出最小的时间戳，`check()` 就不再需要每次都重新遍历，直接读取 `min_resendts` 变量的值即可。
+1. 原版的“已发送数据包缓存”采用的是队列，修改后的版本改成使用 std::map，共三个映射表：总队列，按包号排序；两个待重发队列，一个按时间排序，另一个按丢包跳跃次数排序。
+2. 原版的 `flush()` 是先把待发送数据转移到发送队列后，在同一个循环重做完“发送新数据包”、“数据包重发”、“ACK包发送”三件事。修改后的版本变为先做“数据包重发”、“ACK包发送”，然后再做“待发送数据转移到发送队列”，在转移期间顺便发送。
+3. 原版的 `check()` 每次都会重新遍历一遍发送队列，查找已到点的重传时间戳。修改后的版本变为：从已经排好序的映射表读取第一个时间戳，免除查找步骤。
 
 除此之外，原版的存在“bug”，kcptube 也会有。例如：
 

README_EN.md

@@ -169,6 +169,7 @@ encryption_algorithm=AES-GCM
 | stun_server  | STUN Server's address |No| Cannot be used if listen_port option is port range mode|
 | log_path  | The directory where the Logs are stored |No|Cannot point to the file itself|
 | kcp_mtu  | Positive Integer |No|Default value is 1440|
+| kcp_conserve  | yes<br>true<br>1<br>no<br>false<br>0 |No|Ease up on the 'fast ACK' a little bit.|
 | kcp  | manual<br>fast1 - 6<br>regular1 - 5<br> &nbsp; |Yes|Setup Manually<br>Fast Modes<br>Regular Speeds<br>(the number at the end: the smaller the value, the faster the speed)|
 | kcp_sndwnd  | Positive Integer |No|See the table below for default values, which can be overridden individually|
 | kcp_rcvwnd  | Positive Integer |No|See the table below for default values, which can be overridden individually|
@@ -218,13 +219,15 @@ This bandwidth values should not larger than your actual bandwidth, otherwise th
 |  ----        | :----:     | :----:    | :----:    | :----:     | :----:   |:----: |
 | regular1     | 1024       |   1024    |      1    |   1        |   5      |   1   |
 | regular2     | 1024       |   1024    |      2    |   1        |   5      |   1   |
-| regular3     | 1024       |   1024    |      0    |   1        |   2      |   1   |
-| regular4     | 1024       |   1024    |      0    |   1        |   3      |   1   |
-| regular5     | 1024       |   1024    |      0    |   1        |   0      |   1   |
+| regular3*    | 1024       |   1024    |      1    |   1        |   5      |   1   |
+| regular4*    | 1024       |   1024    |      2    |   1        |   5      |   1   |
+| regular5*    | 1024       |   1024    |      0    |   1        |   3      |   1   |
 
 Note: If the packet loss rate is high enough (higner than 10%), kcp_nodelay=1 may better than kcp_nodelay=2. If the packet loss rate is not too high, kcp_nodelay=2 can make the network latency smoother.
 
-If you want to reduce traffic waste and also accept a little bit more latency increase, please try choosing regular modes. However, using regular 4 or regular 5 in the envirnment of high latency with packet loss rate high than 4% may encounter the situation of TCP disconnection. 
+If you want to reduce traffic waste and also accept a little bit more latency increase, please try choosing regular modes.<br /> 
+The star-marked modes (regular3 ~ 5) have enabled the kcp_conserve option, which leads to slightly higher latency caused by packet loss, but reduces wastage of traffic to some extent.
+
 
 ### Log File
 After obtaining the IP address and port after NAT hole punching for the first time, and after the IP address and port of NAT hole punching change, an ip_address.txt file will be created in the Log directory (overwrite if it exists), and the IP address and port will be written in.
@@ -398,6 +401,36 @@ As kcptube uses IPv6 single-stack + enabled IPv4 mapped addresses (IPv4-mapped I
 If the system does not support IPv6 or IPv6 is disabled, please set ipv4_only=true in the configuration file, so that kcptube will fall back to using IPv4 single-stack mode.
 
 ## Other Considerations
+### ‘Too Many Open Files’ of multiple Operation Systems
+#### GhostBSD
+In general, most BSD systems will not encounter this issue, only GhostBSD updated in the second half of 2023 will encounter it.
+
+This is because GhostBSD has added this line in `/etc/sysctl.conf`:
+```
+kern.maxfiles=100000
+```
+This line reduces the upper limit, far below the corresponding value of the original FreeBSD.
+
+The solution is simple, just delete or comment out this line.<br />
+Alternatively, use the command `sysctl kern.maxfiles=300000` to temporarily change the limit value.
+
+#### Linux
+Since the Open Files limit for Linux systems is 1024, it's easy to encounter such problems.
+
+Temporary solution:
+1. Run the command `ulimit -n` to check the output value.
+2. If the value is indeed only 1024, run the command `ulimit -n 300000`.
+
+Permanent solution:<br />
+Edit /etc/security/limits.conf and add at the end:
+
+```
+*         hard    nofile       300000
+*         soft    nofile       300000
+root      hard    nofile       300000
+root      soft    nofile       300000
+```
+
 ### NetBSD
 After running command
 ```
@@ -410,18 +443,20 @@ However, for unknown reasons, it may not be possible to actively connect to an I
 ### OpenBSD
 OpenBSD completely blocks IPv4-mapped IPv6, if you use dual-stack on the OpenBSD platform, you need to save the configuration file as two files, one of which enables ipv4_only=1, and then load both configuration files when using kcptube.
 
-## Data verification
+## Encryption and Data verification
 Since TCP data transmission is required, data verification cannot be ignored, just like TCP itself.
 
-If the encryption option has been used, then the content of this section can be ignored. The encryption algorithm selected by kcptube has verification capabilities built in, which can also ensure that the transmitted content is error-free.
+Regardless of whether encryption is enabled or not, this program will reduce the MTU by 2 bytes and append 2-byte data at the end.
 
-If the encryption function is not used, kcptube will reduce the MTU by 2 bytes so that a 2-byte checksum can be attached to the end.
+If the encryption option is used, then the 2-byte data appended at the end will be a temporarily generated IV.
 
-However, the Botan library used by kcptube does not include a 16-bit verification algorithm, so kcptube uses two 8-bit checksums at the same time:
+If the encryption feature is not selected, the 2-byte data appended at the end will be the checksum, consisting of two different 8-bit checksums:
 
 - Longitudinal Redundancy Check (LRC)
 - 8-bit checksum
 
+This is because the Botan library used in this program does not come with a 16-bit checksum algorithm. Therefore, this program simultaneously utilizes these two 8-bit checksums.
+
 The calculation speed of these two checksums is fast enough, concise and practical, and is not an obscure calculation method. For example, Modbus uses LRC.
 
 It should be reminded that using two checksums still cannot completely avoid content errors, just like TCP itself. If you really need accuracy, please enable the encryption option.
@@ -452,10 +487,11 @@ The timeout period for KCP channel is 30 seconds after enabling the multiplexing
 To reduce latency, kcptube has enabled the TCP_NODELAY option. For some high-traffic application scenarios, this may result in a reduction in the amount of TCP data transmitted.
 
 ### KCP
-KCP Tube uses the original version of the [KCP](https://github.com/skywind3000/kcp), with some modifications:
+KCP Tube uses a modified version of [KCP](https://github.com/skywind3000/kcp):
 
-1. The original `flush()` function first moves the data to be sent to the sending queue, and then completes ‘sending new packet’, ‘packet retransmission’, and ‘ACK packet sending’ in the same loop. In the modified version, ‘packet retransmission’ and ‘ACK packet sending’ are done first, and then ‘move the data to be sent to the sending queue’ is done, sending it during the transfer.
-2. The original `check()` function would traverse the sending queue every time to find the retransmission timestamp for packets that have reached their timeout. In the modified version, a new variable `min_resendts` is added to the KCP struct. During the `flush()` sending loop, the minimum timestamp is found and stored in min_resendts. `check()` no longer needs to traverse the queue every time. It can directly read the value of `min_resendts`.
+1. The original ‘sent data packet cache’ used a queue, and the modified version changed to use std::map, with three mapping tables: a total queue sorted by packet number, and two wait-for-resend queues, one sorted by time and the other sorted by the number of lost packets.
+2. The original `flush()` function first transfers the data to be sent to the sending queue, and then completes the three things of ‘sending new data packet’, ‘resending data packet’, and ‘sending ACK packet’ in the same loop. The modified version changes to first do ‘resend data packet’ and ‘send ACK packet’, and then do ‘transfer data to be sent to sending queue’, while sending it during the transfer.
+3. The original `check()` function would traverse the sending queue every time to look for the timestamp of the already arrived resend. In the modified version: the first timestamp is read from the already sorted mapping table, eliminating the searching step.
 
 And other ‘bugs’ in the original version, will also exist in kcptube. For example:
 

docs/mtu_en.md

@@ -0,0 +1,25 @@
+# Set MTU Value
+
+If the MTU value needs to be specified, it can be calculated by subtracting the IP header size from the MTU value of the network link where the device is located.
+
+For example, the network for IPoE may have an MTU value of 1500, while the network for PPPoE may have an MTU value of 1492. The size of the IP header is approximately 40 bytes. Thus, KCPTube's MTU value can be calculated as follows:
+
+- IPoE
+    - MTU = 1500 - 40 = 1460
+- PPPoE
+    - MTU = 1492 - 40 = 1452
+
+## Transfer VPN data in KCPTube tunnel
+
+To avoid traffic fragmentation and ensure that each packet transmitted matches the MTU value exactly, the following calculation can be used:
+
+VPN MTU = KCPTube MTU - KCP Header - KCPTube Header - 2 bytes (tail)
+
+If encryption options are enabled:
+
+> VPN MTU = 1440 - 24 - 5 - 2 = 1409
+
+If encryption options are disabled:
+
+> VPN MTU = 1440 - 24 - 9 - 2 = 1405
+

docs/mtu_zh-hans.md

@@ -0,0 +1,24 @@
+# 设置 MTU 值
+
+如果想自行指定 MTU 值，计算的方法是，把设备所在网络链路的 MTU 值减去 IP 头的大小。
+
+比如，IPoE 的网络可能是 1500，PPPoE 的网络可能是 1492。然后 IP 头大小约为 40 字节。那么，KCPTube 的 MTU 值可以如下计算：
+
+- IPoE
+    - MTU = 1500 - 40 = 1460
+- PPPoE
+    - MTU = 1492 - 40 = 1452
+
+## KCPTube 通道内转发 VPN
+
+如果想减少流量拆分，使每次发送的数据刚好匹配 MTU 值，那么可以如下计算：
+
+VPN MTU = KCPTube MTU - KCP Header - KCPTube Header - 2 bytes (tail)
+
+如果启用了加密选项：
+
+> VPN MTU = 1440 - 24 - 5 - 2 = 1409
+
+如果不启用加密选项：
+
+> VPN MTU = 1440 - 24 - 9 - 2 = 1405