Implement simple key ranges for broadcast #230
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
Broadcasting a range of keys currently requires application to populate the full range into a container and pass it to
ttg::broadcast
. These keys then have to be processed and send to peers, leading to large messages (in addition to the actual data transfer).Proposal
Many (dense) applications send a range of keys that can be expressed as a linear function in an N-dimensional space. With key ranges, application can express the start and end of a range as well as a step size for iterating through the range (a vector in the N dimensional space). Some applications may require splitting the range, i.e., providing a list or ranges (e.g., the Floyd-Warshall example).
These ranges can be serialized and send to peers instead of the fully populated list of keys. Such compact representation of the key range is cheaper to transfer and store.
An example for using key ranges:
It is possible to define a different type for the step vector by defining
Key::difference_type
or by specializing thettg::key_diff_type
trait. In order to use a key type in a range, it must provideoperator!=
as well asoperator+
oroperator+=
. A version ofmake_keyrange
without the step size tries to figure out the step vector by applyingoperator-
to the start element and the++start
value so both operators must be provided to use that variant. Unfortunately, due to the coupling between the input terminal and the TT, the key range must be compilable even if these operators are not provided. I've added the appropriate checks tomake_keyrange
and moved theLinearKeyRange
intottg::detail
to prevent users from accidentally instantiate a key range even though not all requirements are met.This PR uses floyd warshall as example but using key ranges will be most useful in #222 as well (see https://github.com/TESSEorg/ttg/pull/222/files#diff-6f4fdbb279035f63f17441c5c665d4a7f64c2a9e858463aa8859016a90c73122R177 where the dispatcher has to populate multiple vectors to feed to
ttg::broadcast
, all of which represent linear key ranges).Signed-off-by: Joseph Schuchart [email protected]