[WIP] Aggregators #229
Open
[WIP] Aggregators #229
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
devreal
force-pushed
the
aggregators2
branch
2 times, most recently
from
03bb105 to
8a2cce3
Compare
Aggregators are meant to replace reduction terminals where the sole purpose
is to collect a number of inputs before passing them into the
task. An aggregator is created by wrapping the input type in a
ttg::aggregator, which will cause values to be aggregated until either
the target count is reached (if provided) or the stream size has been reached.
The aggregator is then passed directly into the task and can be iterated over.
In combination with make_tt, an aggregator is created using
`ttg::make_aggregator(inedge)` or `ttg::make_aggregator(inedge, target)`.
These calls will return a specialized Edge that allows to check at
compile-time and create aggregators in the backend.
Example:
```
ttg::Edge<int, int> edge;
auto tt = make_tt(
[](const int& key, ttg::aggregator<int>& agg){
for (auto&& v : agg) {
...
}
},
ttg::edges(ttg::make_aggregator(edge)), ...);
```
Signed-off-by: Joseph Schuchart <[email protected]>
Some use-cases have per-key aggregation targets so we need the added flexibility. Signed-off-by: Joseph Schuchart <[email protected]>
Signed-off-by: Joseph Schuchart <[email protected]>
Signed-off-by: Joseph Schuchart <[email protected]>
…rload Signed-off-by: Joseph Schuchart <[email protected]>
Signed-off-by: Joseph Schuchart <[email protected]>
Signed-off-by: Joseph Schuchart <[email protected]>
Signed-off-by: Joseph Schuchart <[email protected]>
Signed-off-by: Joseph Schuchart <[email protected]>
This feature is useful if there is sufficient concurrency in the application, with multiple consumers and one subsequent producer on a data. By seting defer_writer(), the application can signal that it is ok to defer the next writer on the data until all readers completed, avoiding the data to be copied. This may lead to a deadlock though, if the execution of the readers depends on the writer. This feature thus has to be used cautiously. Signed-off-by: Joseph Schuchart <[email protected]>
Signed-off-by: Joseph Schuchart <[email protected]>
For small aggregators the dynamic memory management of a vector is potentially too costly. Unfortunately, we cannot use unions with non-POD datatypes and std::variant won't work with arrays and requires initialization of the member that is used. Signed-off-by: Joseph Schuchart <[email protected]>
Look at the in_data_count instead of the remove_from_hash field Signed-off-by: Joseph Schuchart <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Aggregators are meant to replace reduction terminals where the sole purpose is to collect a number of inputs before passing them into the task. An aggregator is created by wrapping the input type in a
ttg::aggregator, which will cause values to be aggregated until either the target count is reached (if provided) or the stream size has been reached. The aggregator is then passed directly into the task and can be iterated over.In combination with
make_tt, an aggregator is created usingttg::make_aggregator(inedge)orttg::make_aggregator(inedge, target). These calls will return a specialized Edge that allows to check at compile-time and create aggregators in the backend.Example:
Still to do:
[ ] Adapt the MADNESS backend.
[ ] Add support for
voidterminals (aggregator terminals will aggregate signals, not values)[ ] Make sure the copies stored in the aggregator are properly found and released (needs #206 first)
Signed-off-by: Joseph Schuchart [email protected]