Draft : Add infra to generate extra tensor #13486

zetwhite · 2024-07-23T06:00:49Z

I moved the old draft(https://github.com/Samsung/ONE/pull/13326/files#diff-f4aa0ebe5af90acde526ab63faaa7352df19cfbfef06379f2dbeb8da5a5dab86) to new PR.
Because My old code and modifications are mixed up and I feel hard to debug.

zetwhite · 2024-07-30T02:05:42Z

@nnfw-bot test onert-x64-release

hseok-oh · 2024-07-30T08:47:12Z

android build fail

/opt/jenkins_agent/workspace/nnfw/master/pr-onert-android-release/runtime/onert/backend/train/BackendContext.cc:238:42: error: captured structured bindings are a C++20 extension [-Werror,-Wc++20-extensions]
      extra_tensor_gen->register_tensors(operation_index, fn->requestExtraTensors());
                                         ^
/opt/jenkins_agent/workspace/nnfw/master/pr-onert-android-release/runtime/onert/backend/train/BackendContext.cc:235:15: note: 'operation_index' declared here
  for (auto &[operation_index, fn_seq] : ret)
              ^
1 error generated.

zetwhite · 2024-08-01T10:02:45Z

runtime/onert/backend/train/ExtraTensorGenerator.h

+namespace train
+{
+
+class ExtraTensorGenerator


Is this class necessary?

_idx_to_request can moved to TrainableGraph

plan() can be moved to TensorPlanner

allocate() is just a single line function

jyoungyun · 2024-08-07T06:51:51Z

runtime/onert/core/include/exec/train/ITrainableFunction.h

+  // Implement this if extra tensor is needed
+  virtual backend::train::ExtraTensorRequests requestExtraTensors()
+  {
+    backend::train::ExtraTensorRequests r;


(Q) Why should this object be created in every TrainableFunction?

No, It isn't.
Only (Pool, Conv, DConv, FC, BinaryArithmetic) Layer requires ExtraTensors.

I put more comments here: https://github.com/Samsung/ONE/pull/13486/files#r1706502494

Is there a better way - not to create the object every time?

zetwhite · 2024-08-07T07:15:47Z

runtime/onert/backend/train/BackendContext.cc

+    fn_seq->iterate([&](exec::train::ITrainableFunction &fn) {
+      extra_tensor_gen.register_tensors(op_idx, (&fn)->requestExtraTensors());
+    });


But, ExtraTensorRequests are accessed through ITrainableFunction without downcasting to the specific layer. So, I'd like to define the interface function(?) in ITrainbleFunction.

runtime/onert/backend/train/ExtraTensorGenerator.cc

This PR adds registerLayerScopeTensors to ITrainableFunction. 'registerLayerScopeTensors` is to register LayerScopeTensor into TensorReigstry. ONE-DCO-1.0-Signed-off-by: seunghui youn <[email protected]> draft : Samsung#13486 for : Samsung#13282

zetwhite · 2024-09-11T03:04:59Z

runtime/onert/backend/train/ops/BinaryArithmeticLayer.cc

+std::optional<ExtraTensors> BinaryArithmeticLayer::registerExtraTensors()
+{
+  ExtraTensors tensors;
+
+  if (_act_back_prop_output != nullptr)
+  {
+    tensors.push_back(_act_back_prop_output);
+  }
+
+  return std::optional<ExtraTensors>(tensors);
+}


example of registerLayerScopeTensors usage

This PR introduces LayerScopeMemoryManager. This Manager will be added to TensorManager and used to allocate LayerScopeTensors. ONE-DCO-1.0-Signed-off-by: seunghui youn <[email protected]> draft : Samsung#13486 for : Samsung#13282

This PR adds LayerScopeTensors into TensorRegistry. ONE-DCO-1.0-Signed-off-by: seunghui youn <[email protected]> draft : Samsung#13486 for : Samsung#13282

This PR introduces LayerScopeMemoryManager. This Manager will be added to TensorManager and used to allocate LayerScopeTensors. ONE-DCO-1.0-Signed-off-by: seunghui youn <[email protected]> draft : #13486 for : #13282

This PR adds LayerScopeManager into TensorManager. ONE-DCO-1.0-Signed-off-by: seunghui youn <[email protected]> draft : Samsung#13486 for : Samsung#13282

This PR adds LayerScopeTensors into TensorRegistry. ONE-DCO-1.0-Signed-off-by: seunghui youn <[email protected]> draft : Samsung#13486 for : Samsung#13282

This PR adds LayerScopeManager into TensorManager. ONE-DCO-1.0-Signed-off-by: seunghui youn <[email protected]> draft : Samsung#13486 for : Samsung#13282

zetwhite · 2024-10-10T10:58:03Z

runtime/onert/backend/train/TensorPlanner.cc

+  // forwading order
+  const auto f_order = _tgraph.topolSortOperations();
+  for (const auto &op_index : f_order)
+  {
+    if (not tensor_builder->isRegisteredLayerScopeTensor(op_index))
+      continue;
+
+    auto indices = tensor_builder->getRegisteredLayerScopeTensorIndex(op_index);
+    for (const auto &idx : indices)
+    {
+      const auto lt = tensor_builder->getLayerScopeTensorLifeTime(idx);
+      if (lt == LayerScopeTensorLifeTime::FORWARD_TO_BACKWARD)
+        tensor_builder->notifyLayerScopeFirstUse(idx);
+    }


For planning, TensorBuilder has to provide

LayerScopeTensorIndex - matched by each OprationIndex

LifeTime information of each LayerScopeTensor.

This PR adds TensorPlanner::planLayerScopeTensors() and its caller part in BackendContext. ONE-DCO-1.0-Signed-off-by: seunghui youn <[email protected]> -------------------------------------- draft : Samsung#13486

This PR registers LayerScopeTensor from each layer into tensor registry. ONE-DCO-1.0-Signed-off-by: seunghui youn <[email protected]> -------------------------------------- draft : Samsung#13486

This PR registers LayerScopeTensor from each layer into tensor registry. ONE-DCO-1.0-Signed-off-by: seunghui youn <[email protected]> -------------------------------------- draft : #13486

zetwhite force-pushed the 0722/etensor branch 5 times, most recently from ae4c45b to dde2bbf Compare July 29, 2024 15:41

zetwhite mentioned this pull request Jul 31, 2024

[onert/train] Attach auxiliary tensors to tensor builder #13282

Open

zetwhite force-pushed the 0722/etensor branch 2 times, most recently from 5ca9a7c to 80c73f7 Compare August 1, 2024 08:40

zetwhite commented Aug 1, 2024

View reviewed changes

zetwhite mentioned this pull request Aug 1, 2024

[onert] Introduce ExtraTensor #13576

Merged

zetwhite force-pushed the 0722/etensor branch from 4c2432d to bae9407 Compare August 7, 2024 02:01

zetwhite mentioned this pull request Aug 7, 2024

[onert] Introduce ExtraTensorRequest #13604

Closed

zetwhite force-pushed the 0722/etensor branch from bae9407 to 8f78928 Compare August 7, 2024 06:15

jyoungyun reviewed Aug 7, 2024

View reviewed changes

zetwhite mentioned this pull request Aug 7, 2024

[onert] Introduce ExtraTensorIndex #13605

Merged

zetwhite commented Aug 7, 2024

View reviewed changes

ragmani reviewed Aug 8, 2024

View reviewed changes

runtime/onert/backend/train/ExtraTensorGenerator.cc Outdated Show resolved Hide resolved

runtime/onert/backend/train/ExtraTensorGenerator.cc Outdated Show resolved Hide resolved

runtime/onert/backend/train/ExtraTensorGenerator.cc Outdated Show resolved Hide resolved

zetwhite force-pushed the 0722/etensor branch from 755ded6 to a86a963 Compare August 19, 2024 06:50

zetwhite force-pushed the 0722/etensor branch 4 times, most recently from 6518364 to b3afcb1 Compare August 29, 2024 11:00

This was referenced Aug 30, 2024

[onert] Add lifetime enum to LayerScopeTensor #13861

Merged

[onert] Templatize memory planners in train backend #13892

Merged

zetwhite mentioned this pull request Sep 11, 2024

[onert] Add registerLayerScopeTensors to ITrainableFuntion #13980

Merged

zetwhite commented Sep 11, 2024

View reviewed changes

zetwhite mentioned this pull request Sep 13, 2024

[onert/backend] Introduce LayerScopeMemoryManager #14021

Merged

zetwhite mentioned this pull request Sep 13, 2024

[onert/backend] Add LayerScopeTensors into TensorRegistry #14024

Merged

zetwhite mentioned this pull request Sep 20, 2024

[onert/train] Add LayerScopeManager into TensorManager #14047

Merged

zetwhite added 2 commits October 10, 2024 16:21

add ops

b89d3e3

update backend

5412be5

zetwhite force-pushed the 0722/etensor branch from b3afcb1 to 82f949c Compare October 10, 2024 08:18

rename extra to layerscope

1f26bc4

zetwhite force-pushed the 0722/etensor branch from 82f949c to 1f26bc4 Compare October 10, 2024 10:37

zetwhite mentioned this pull request Oct 10, 2024

[onert/backend] Add LayerScopeTensor interface to TensorBuilder #14198

Merged

zetwhite commented Oct 10, 2024

View reviewed changes

resolve andriod build err

401dc31

zetwhite mentioned this pull request Oct 10, 2024

[onert/backend] Add LayerScopeTensor Planning #14199

Merged

zetwhite mentioned this pull request Oct 18, 2024

[onert/train] Register LayerScopeTensor to registry #14235

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Draft : Add infra to generate extra tensor #13486

Draft : Add infra to generate extra tensor #13486

zetwhite commented Jul 23, 2024 •

edited

Loading

zetwhite commented Jul 30, 2024

hseok-oh commented Jul 30, 2024 •

edited

Loading

zetwhite Aug 1, 2024

jyoungyun Aug 7, 2024

zetwhite Aug 7, 2024 •

edited

Loading

zetwhite Aug 7, 2024 •

edited

Loading

zetwhite Aug 7, 2024 •

edited

Loading

zetwhite Sep 11, 2024

zetwhite Oct 10, 2024 •

edited

Loading

Draft : Add infra to generate extra tensor #13486

Are you sure you want to change the base?

Draft : Add infra to generate extra tensor #13486

Conversation

zetwhite commented Jul 23, 2024 • edited Loading

zetwhite commented Jul 30, 2024

hseok-oh commented Jul 30, 2024 • edited Loading

zetwhite Aug 1, 2024

Choose a reason for hiding this comment

jyoungyun Aug 7, 2024

Choose a reason for hiding this comment

zetwhite Aug 7, 2024 • edited Loading

Choose a reason for hiding this comment

zetwhite Aug 7, 2024 • edited Loading

Choose a reason for hiding this comment

zetwhite Aug 7, 2024 • edited Loading

Choose a reason for hiding this comment

zetwhite Sep 11, 2024

Choose a reason for hiding this comment

zetwhite Oct 10, 2024 • edited Loading

Choose a reason for hiding this comment

zetwhite commented Jul 23, 2024 •

edited

Loading

hseok-oh commented Jul 30, 2024 •

edited

Loading

zetwhite Aug 7, 2024 •

edited

Loading

zetwhite Aug 7, 2024 •

edited

Loading

zetwhite Aug 7, 2024 •

edited

Loading

zetwhite Oct 10, 2024 •

edited

Loading