ONNX operators needed to be supported

[daquexian]

This issue is to summarize all operators which needed to be supported. If you want an operator to be supported, please feel free to post it here.

• LRN(Unsupported operator LRN #16, supported in Unify the order of paddings, fix bug of LRN and so on #46)
• PRelu(identity, prelu support #13, implemented in Refactor and add naive prelu support #23)
• Identify(identity, prelu support #13, supported in Add more optimizing #24)
• BatchNorm(most batchnorm will be fused into conv so there is no need to support batchnorm in these cases, but a few models(like resnet v2) has a bn->relu->conv sequence, which makes bn unable to be fused into conv.) (Supported in Fix pool shaper bug, fix fused add/mul bug, support batchnorm, suppor… #38 )
• Resize

[AmienKhaled]

Constant operator

[daquexian]

PReLU has been supported in #23

@AmienKhaled I'll implement it soon

[daquexian]

@AmienKhaled @yunhaia

In version 0.6.0, identity, prelu and constant op should be supported (please also use the latest onnx2daq)

[ericdanz]

Reshape can only be the last layer

[daquexian]

@ericdanz reshape layer is somewhat special due to the difference between nhwc(nnapi) and nchw(onnx) layout. I think there is no general, simple way to implement reshape layer, and most reshape layers in an onnx model are indeed generated implicitly. Could you please provide your model so that I can analyze it?

[ericdanz]

I can share the onnx file. Quick warning - it's a bit complicated ;)
https://drive.google.com/file/d/1WdWn12HPr5Ziz69PBk-JyU8YtiEfU2mu/view?usp=sharing

There are a few places reshape is used, in my special batchnorm and shuffle-net like layers.

[daquexian]

@ericdanz Thanks for your model. Unfortunately, the reshape here is hard to implement in nnapi. I'll open another issue to track it. However, it may be a long-term work.

[ericdanz]

Concat is causing some problems - had to disable the shape checking in shaper.cpp

Upsample is also not supported yet.

Made a model that is compatible, about to test it out! Very excited to see more and more ops supported ;)

[daquexian]

Thanks for your valuable information! I will look into it today (It is 7:06 AM, GMT+8 here XD) Thanks again!

…

On Tue, Jan 29, 2019, 4:08 AM ericdanz ***@***.*** wrote: Concat is causing some problems - had to disable the shape checking in shaper.cpp Upsample is also not supported yet. Made a model that is compatible, about to test it out! Very excited to see more and more ops supported ;) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#17 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ALEcn6OwqZ0UXXKevlqqWEUYgcwGG6lvks5vH1i3gaJpZM4ZM5hH> .

[mbishop-neurala]

Transpose would be useful when converting caffe models.

[neginraoof]

@ericdanz @daquexian Do you happen to have such large pytorch model that could be further optimized after ONNX export?