Skip to content

Latest commit

 

History

History
36 lines (29 loc) · 1.23 KB

how-to-use-specific-backend.md

File metadata and controls

36 lines (29 loc) · 1.23 KB
Raw

How to Use Specific Backend during Inference

ONE runtime has many ways to use specific backend during inference

Using NNFW API

nnfw_set_available_backends

  • Multiple backends can be set and they must be separated by a semicolon (ex: "acl_cl;cpu").
  • For each backend string, libbackend_{backend}.so will be dynamically loaded during nnfw_prepare.
  • Among the multiple backends, the 1st element is used as the default backend.

Using Environment Variable

1. BACKENDS

  • Same as nnfw_set_available_backends
  • Example
BACKENDS=cpu ./Product/out/bin/onert_run ...

2. OP_BACKEND_[OP_TYPE]

  • Set backend for specific operator type
  • Example
    • Execute Conv2D operator on ruy backend and others on cpu backend
OP_BACKEND_Conv2D=ruy BACKENDS="cpu;ruy" ./Product/out/bin/onert_run ...

3. OP_BACKEND_MAP

  • Set backend for specific operator by its index
  • Format : <op_id>=<backend>;<op_id>=<backend>...
  • Example
    • Execute operator 10 on acl_cl backend and others on acl_neon backend
OP_BACKEND_MAP="10=acl_cl" BACKENDS="acl_neon;acl_cl" ./Product/out/bin/onert_run ...