You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently all internal communications and code flows are built around primitive go channels and signals, as a result the complexity has exploded especially in baseoutput related to shutdown mechanism:
There are async sender and receiver to enable pipelining / reduce latency
There are soft stop, hard stop (abort) to reconnect, and hard stop to shut down.
Network reading and writing can fail independently and the other side may continue to work (unintended)
All pending chunks whether in queue or being processed in functions need to be collected for recovery
The code is difficult to test. Currently the only tests involving real output are in run package and they can't cover any exceptional situations.
Instead of a free form that each piece of code decides how to use channel and signals, there could be some unified framework or library that help making:
The top-level flow should be visualized, in a declarative form. It needs to cover all exceptional situations because dealing with them is the main cause of complexity.
The components (e.g. acknowledger or sub unit) should have clear input, output, responsibilities and no implicit side-effects
The internal communication in baseoutput is not performance critical as it works on compressed log chunks
The text was updated successfully, but these errors were encountered:
Currently all internal communications and code flows are built around primitive go channels and signals, as a result the complexity has exploded especially in
baseoutput
related to shutdown mechanism:Instead of a free form that each piece of code decides how to use channel and signals, there could be some unified framework or library that help making:
The internal communication in
baseoutput
is not performance critical as it works on compressed log chunksThe text was updated successfully, but these errors were encountered: