You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Can we please provide a faster default for the number of parallel file transfers? I think a default setting of 4 concurrent file transfers would really improve out of the box performance.
4 matches similar concurrency settings for FTP clients.
4 integrates well with modern CPU core count configurations.
4 has low risk of triggering transfer errors in Kubernetes.
4 has low risk of triggering transfer errors in S3.
Also, I wonder if the value of 0 for the concurrency setting should be rejected by skbn. This is because full concurrency of all files at once, tends to trigger transfer cooldown errors. Because file trees with very many files, will basically create a denial of service attack as far as S3 sees.
The text was updated successfully, but these errors were encountered:
mcandre
changed the title
Default parallelism to full
Default parallelism to the number of host CPU cores
May 10, 2021
mcandre
changed the title
Default parallelism to the number of host CPU cores
Better parallelism default
May 10, 2021
mcandre
changed the title
Better parallelism default
Minor parallelism improvements
May 10, 2021
Can we please provide a faster default for the number of parallel file transfers? I think a default setting of 4 concurrent file transfers would really improve out of the box performance.
Also, I wonder if the value of
0
for the concurrency setting should be rejected by skbn. This is because full concurrency of all files at once, tends to trigger transfer cooldown errors. Because file trees with very many files, will basically create a denial of service attack as far as S3 sees.The text was updated successfully, but these errors were encountered: