[Draft] attach extra tensor to tensor builder

[zetwhite]

(WIP) This draft try to attach extra tensors to tensor builder.

ONE-DCO-1.0-Signed-off-by: SeungHui Youn [email protected]

related : #13282

[zetwhite]

@nnfw-bot test onert-cross-release

[zetwhite]

.

[hseok-oh]

IMO

• Introduce individual tensor manager (and memory manager if we need) for extra tensor
  • Extra tensor manager get request for extra tensor and manage extra tensor
  • Extra tensor manager may use extra memory manager to manage actual memory
• Train backend's BackendContext owns extra tensor manager, and train kernel generator points extra tensor manager
• When kernel generator visits each operator, visitor request extra tensor manager
  • Extra tensor manager return generated extra tensor(s): tensor struct with tensor info only, not allocated yet
  • Visitor create op's kernel layer and backward configure with extra tensor(s) from extra tensor manager
• After kernel generator's visitor loop end, kernel generator request extra tensor manager to allocate in genKernels() (or generateFunctionMap())

ex.) CPU backend create tensor and allocate at once in genTensors(), but acl-cl/acl-neon/gpu-cl backends divide tensor creation and allocation.

Create tensors

ONE/runtime/onert/backend/acl_common/AclBackendContext.h

Lines 70 to 85 in dfd595b

	// TODO Get compiler options from compiler, and use it rather than getting it from Env
	if (util::getConfigString(util::config::EXECUTOR) == "Linear")
	{
	this->planTensors();
	}
	else
	{
	// For the executors that does not have fixed linear execution order:
	// To make tensors never be deallocated, this is a workaround to use static memory planner
	this->graph()->operands().iterate([&](const ir::OperandIndex &ind, const ir::Operand &) {
	if (this->tensor_builder->isRegistered(ind))
	this->tensor_builder->notifyFirstUse(ind);
	});
	}
	this->tensor_builder->prepare();

Allocation after kernel generation

ONE/runtime/onert/backend/cl_common/include/cl_common/BackendContext.h

Lines 49 to 60 in dfd595b

	FunctionMap genKernels() override
	{
	FunctionMap ret;
	// kernel_gen
	for (auto &&op_ind : _data.op_order)
	{
	auto fn_seq = kernel_gen->generate(op_ind);
	ret.emplace(op_ind, std::move(fn_seq));
	}
	tensor_builder->allocate();

[zetwhite]

@hseok-oh Thank you a lot for taking a look 😄

I catched your point. It looks quite good 👍 😄

Since I already implemented ExtraTensorManage and ExtraTensorIndex .. etc, I could quickly implement your suggestions. I'll revise it right now.

[ragmani]

It's template instantiation, not specialization. I'm not sure if it's template instantiation.

[ragmani]

Anyway, I have a question.

If template member functions are defined in a cc file, can these instantiated template class be used in other cc files without including this cc file?

[zetwhite]

If template member functions are defined in a cc file, can these instantiated template class be used in other cc files without including this cc file?

Yes, I don't know why. But this worked well.
I'd better check accurate C++ grammar related to this

[ragmani]

I'm very surprised that it works. I'm curious about the reason why there is no compilation problem.
However, I'm worried about multiple definition problem if those instantiated template classes are used in multiple cc files. Could you check that too?

[zetwhite]

I'm worried about multiple definition problem if those instantiated template classes are used in multiple cc files. Could you check that too?

That's why I put instantiation here.
I first tried it in the header, but the compiler complains about multiple definitions.

I'm very surprised that it works. I'm curious about the reason why there is no compilation problem

I couldn't fully understand why it works. But I could find similar answers in stack overflow :
https://stackoverflow.com/questions/32707522/explicit-instantiation-of-template

[ragmani]

I first tried it in the header, but the compiler complains about multiple definitions.

Usually, we have resolved by using inline keyword. Anyway, it seems that there is no complaint about multiple definition with the template instantiation.

I couldn't fully understand why it works. But I could find similar answers in stack overflow :
https://stackoverflow.com/questions/32707522/explicit-instantiation-of-template

I also found a related answers in https://isocpp.org/wiki/faq/templates#separate-template-fn-defn-from-decl.