Trouble transferring large data with Sente

[ptaoussanis]

Issue #439 has raised a Sente usability concern:

It's currently a bad idea to use Sente for large data transfers (> 1MB, say).

The reason is that Sente will by default operate over a WebSocket when possible. This is great for realtime bidirectional communications, but it does mean that there's a bottleneck on that single socket.

If a WebSocket connection is saturated dealing with a large transfer, other communications won't be able to get through until the large transfer completes.

In the worst case (with very large payloads and/or very slow connections), this could even cause the client to prematurely disconnect before the long transfer is complete. (During the large transfer, the server will be unable to respond to ping requests from the client, causing the client to believe that its connection has gone bad).

In practice this is rarely a major problem since Sente tends to be used for relatively small payloads, but as #439 shows - this limitation can cause unexpected problems. It'd be worth considering what can be done to better protect Sente users against this possibility.

As a start, I'll document the limitation and offer an easy recommended workaround: just use Sente for smaller payloads and for signalling. If you want to do a large transfer, use a dedicated Ajax call. There's already a util in Sente for this.

But beyond that, might be some other ideas worth pursuing.

Checklist

• Immediately

  • Document limitation and recommended workaround. Done

• Next Sente update

  • Maybe log a warning on large transfers?
  • Improve documentation for Sente's ajax util.

• For future consideration

  • Consider auto-splitting large payloads into size-limited chunks.
  • Consider maintaining >1 WebSocket per Sente channel socket.
  • Consider/investigate other options

I'll note: since there's also other benefits to doing large transfers over Ajax (e.g. HTTP caching), my current inclination is to keep things simple on Sente's implementation (i.e. no auto chunking or >1 sockets) and instead just document+warn about risky usage.

Will take some time to properly consider this though when I'm next on batched Sente work.

And in the meantime: feedback welcome!