Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sftp: speed / latency / protocol issues #44

Open
ThomasWaldmann opened this issue Sep 16, 2024 · 1 comment
Open

sftp: speed / latency / protocol issues #44

ThomasWaldmann opened this issue Sep 16, 2024 · 1 comment
Labels
documentation Improvements or additions to documentation wontfix This will not be worked on

Comments

@ThomasWaldmann
Copy link
Member

ThomasWaldmann commented Sep 16, 2024

Documenting some fundamental issues of the sftp protocol:

  • sftp protocol uses 32KB blocks for data transfer and usually each block has to get acknowledged individually, which slows down transfers significantly on connections with notable latency.
  • paramiko offers pipelining (write) and prefetching (read) to optimize this and borgstore uses these features.
  • but in the end, sftp is just not a fast protocol and paramiko is not a fast sftp client implementation either.

Thus, using sftp is not recommended for big stores nor for high latency connections.

We can't do much about that in borgstore besides offering alternatives that use other protocols.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation wontfix This will not be worked on
Projects
None yet
Development

No branches or pull requests

1 participant