Handle Stream Closing for Non-HTTP

[dnwiebe]

A TCP stream has two sides, one for each end. Let's imagine that one end is blue and the other is green. The TCP stream will consist of a blue channel over which the blue end sends data to the green end, and a green channel over which the green end sends data to the blue end.

When it's time for the stream to be destroyed, one end always decides first. Let's imagine that it's the blue end that has finished its last transmission down the blue channel and has decided to close the stream.

Once that happens, the blue channel is dead and the blue end can no longer send any data; but the green end may well still be in the middle of sending something important, so the green channel is still alive. When the green end sees that the blue channel is dead, it will finish its business and then close the green channel, whereupon the entire TCP stream will be officially dead and gone: but for the channel to disappear each side has to close its channel.

In HTTP connections, however, when the server kills a stream, it is never interested in anything further the browser may have to say, and so when the ProxyServer receives a last_data = true CORES package from the MASQ Network, it immediately retires the corresponding StreamKey.

This will always work for HTTP 1.1 (it may cause some problems with HTTP 2.0, but we don't support that yet). However, we want the MASQ Network to be able to operate with protocols other than HTTP, and under those other protocols, it may be unacceptable to assume that the originating Node will have nothing further to say after the exit Node closes its side of the virtual TCP stream.

Task: Separate the operations of retiring the StreamKey and closing the StreamHandlerPool-to-browser side of the TCP stream. When we get notification from the exit Node that the entity beyond it has closed the stream, we should instruct the StreamHandlerPool to unregister and kill its StreamWriter, thus signaling the originating Node's client that its peer has dropped the stream; however, since the client may have something further to say, we should not retire the StreamKey until after we sense that the originating Node's client has actually closed its own end of the connection.

When the originating Node's client initiates the close, we already (after GH-800) give the exit Node's client a chance to control the retiring of the StreamKey; when the exit Node's client initiates the close, we should give the originating Node's client the chance to control the retiring of the StreamKey, and at the moment we don't.