Merge branch 'master' into release/2.0

.github/workflows/jupyter-test.yml

@@ -0,0 +1,31 @@
+# on:
+#   push:
+#     tags:
+#       - '*'
+
+# jobs:
+#   test:
+#     name: test-notebook-${{matrix.config.name}}
+#     runs-on: ${{matrix.config.os}}
+#     strategy:
+#       matrix:
+#         config:
+#           - {name: x86_64-macos, os: macos-latest}
+#           - {name: x86_64-linux, os: ubuntu-latest}
+#           - {name: x86_64-windows, os: windows-latest}
+
+#     steps:
+#     - uses: actions/checkout@v2
+
+#     - name: Setup Python
+#       uses: actions/setup-python@v2
+#       with: 
+#         python-version: 3.7
+
+#     - name: Install dependencies
+#       run: pip install --upgrade pip && pip install jupyterlab pytest nbmake
+
+#     - name: Run tests
+#       run: pytest --nbmake examples/user_guide
+#       env:
+#         PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION: python

README.md

@@ -2,8 +2,9 @@
 <img src="docs/logo.png" width="400" alt="nncase" />
 </div>
 
-[![License](https://img.shields.io/badge/license-Apache%202-blue)](https://raw.githubusercontent.com/kendryte/nncase/master/LICENSE)
-[![compiler-build](https://github.com/kendryte/nncase/actions/workflows/compiler-build.yml/badge.svg)](https://github.com/kendryte/nncase/actions/workflows/compiler-build.yml)
+[![GitHub repository](https://img.shields.io/badge/github-repository-blue?logo=github&style=plastic)](https://github.com/kendryte/nncase)
+[![Gitee repository](https://img.shields.io/badge/gitee-repository-blue?logo=gitee&style=plastic)](https://gitee.com/kendryte/nncase)
+[![GitHub release](https://img.shields.io/github/v/release/kendryte/nncase?color=brightgreen&display_name=tag&logo=github&style=plastic)](https://github.com/kendryte/nncase/releases)
 
 `nncase` is a neural network compiler for AI accelerators.
 
@@ -14,47 +15,55 @@
 Telegram: [nncase community](https://t.me/joinchat/PPcEPZMLaTViNDI1)
 
 ## Install from binaries
+
 ## 从二进制安装
 
 Download prebuilt binaries from [Release](https://github.com/kendryte/nncase/releases).
 
 下载预编译的二进制文件 [Release](https://github.com/kendryte/nncase/releases)。
 
 ## Build from source
+
 ## 从源码编译
 
 [Build from source](./docs/build.md)
 
 ## Supported operators
+
 ## 支持的算子
 
 - [TFLite ops](./docs/tflite_ops.md)
 - [Caffe ops](./docs/caffe_ops.md)
 - [ONNX ops](./docs/onnx_ops.md)
 
-
 ## K210/K510
+
 - [Usage](https://github.com/kendryte/nncase/blob/release/1.0/docs/USAGE_EN.md)
 - [FAQ](https://github.com/kendryte/nncase/blob/release/1.0/docs/FAQ_EN.md)
 - [使用说明](https://github.com/kendryte/nncase/blob/release/1.0/docs/USAGE_ZH.md)
 - [常见问题](https://github.com/kendryte/nncase/blob/release/1.0/docs/FAQ_ZH.md)
 - [Example](https://github.com/kendryte/nncase/blob/release/1.0/examples/user_guide/)
+
 ## K230
+
 - [Usage](./docs/USAGE_v2_EN.md)
 - [FAQ](./docs/FAQ_EN.md)
+- [Example](./examples/user_guide/k230_simulate-EN.ipynb)
 - [使用说明](./docs/USAGE_v2.md)
 - [常见问题](./docs/FAQ_ZH.md)
-- [Example](./examples/user_guide/)
+- [示例](./examples/user_guide/k230_simulate-ZH.ipynb)
 
 ## Resources
-## 资源
 
+## 资源
+### K210
 - [K210_Yolo_framework](https://github.com/zhen8838/K210_Yolo_framework)
-- [Shts! 's Blog (Japanese)](https://www.shtsno24.tokyo/2020/03/nncase-v020.html)
+- [Shts!'s Blog (Japanese)](https://www.shtsno24.tokyo/2020/03/nncase-v020.html)
 
 ---
 
 ## Architecture
+
 ## 架构
 
 <div align="center">

conanfile.py

@@ -51,6 +51,7 @@ def requirements(self):
         if self.options.tests:
             self.requires('gtest/1.10.0')
             self.requires('ortki/0.0.2')
+            self.requires('rapidjson/1.1.x')
 
         if self.options.python:
             self.requires('pybind11/2.6.1')

docs/FAQ_EN.md

@@ -1,24 +1,22 @@
 # FAQ
 
-[TOC]
+[toc]
 
 ## 1. Error installing `whl` package
 
 ### 1.1 Q: `xxx.whl is not a supported wheel on this platform`
 
 A: Upgrade pip >= 20.3 using `pip install --upgrade pip`
 
-
-
-----
+---
 
 ## 2. Compile-time errors
 
 ### 2.1 "System.NotSupportedException"
 
 #### 2.1.1 Q: Compile model reported error "System.NotSupportedException: Not Supported *** op: XXX"
 
-A: This exception indicates that there are operators, `XXX`, that are not yet supported. You can create a issue in [nncase Github Issue](https://github.com/kendryte/nncase/issues). In the current directory `***_ops.md`, you can view the operators already supported in each inference framework. 
+A: This exception indicates that there are operators, `XXX`, that are not yet supported. You can create a issue in [nncase Github Issue](https://github.com/kendryte/nncase/issues). In the current directory `***_ops.md`, you can view the operators already supported in each inference framework.
 
 If 'XXX' belongs to quantization-related operators such as `FAKE_QUANT`, `DEQUANTIZE`, `QUANTIZE`, it indicates that the current model is a quantized model, and 'nncase' does not currently support such models, please compile `kmodel` using a floating point model.
 
@@ -28,15 +26,19 @@ If 'XXX' belongs to quantization-related operators such as `FAKE_QUANT`, `DEQUAN
 
 A: Use `sudo gedit /proc/sys/fs/inotify/max_user_instances` to change 128 to a larger value.
 
+### 2.3 `initialize` error
+
+#### 2.3.1 Q："RuntimeError: Failed to initialize hostfxr" appears when compiling the kmodel.
 
+A1：Need to install dotnet-7.0.
 
-----
+---
 
 ## 3. Runtime errors
 
 ### 3.1 Q: Compiling `kmodel` is fine, but when inferring, the error `nncase.simulator.k230.sc: not found`occurs.
 
-A: You need to check whether the versions of `nncase` and `nncase-kpu` are the same.
+A: First, make sure that the path of the nncase installation is added to the PATH environment variable. You need to check whether the versions of `nncase` and `nncase-kpu` are the same.
 
 ```shell
 root@a52f1cacf581:/mnt# pip list | grep nncase
@@ -46,9 +48,7 @@ nncase-kpu 2.1.1.20230721
 
 If inconsistent, install the same version of the Python package `pip install nncase==x.x.x.x nncase-kpu==x.x.x.x`
 
-
-
-----
+---
 
 ## 4. Runtime error on k230 development board
 

docs/FAQ_ZH.md

@@ -6,35 +6,35 @@
 
 A：升级 pip >= 20.3 `pip install --upgrade pip`
 
-
-
-----
+---
 
 ## 2.编译模型时报错
 
 ### 2.1 `System.NotSupportedException`
 
 #### 2.1.1 Q：编译模型报错“System.NotSupportedException: Not Supported *** op: XXX”。
 
-A：该异常表明`XXX`算子尚未支持，可以在[nncase Github Issue](https://github.com/kendryte/nncase/issues)中提需求。当前目录下 `***_ops.md`文档，可以查看各个推理框架中已经支持的算子。
+A：该异常表明 `XXX`算子尚未支持，可以在[nncase Github Issue](https://github.com/kendryte/nncase/issues)中提需求。当前目录下 `***_ops.md`文档，可以查看各个推理框架中已经支持的算子。
 
-如果`XXX`属于 `FAKE_QUANT`、`DEQUANTIZE`、`QUANTIZE`等量化相关的算子，表明当前模型属于量化模型，`nncase`目前不支持这类模型，请使用浮点模型来编译`kmodel`。
+如果 `XXX`属于 `FAKE_QUANT`、`DEQUANTIZE`、`QUANTIZE`等量化相关的算子，表明当前模型属于量化模型，`nncase`目前不支持这类模型，请使用浮点模型来编译 `kmodel`。
 
 ### 2.2 `System.IO.IOException`
 
-#### 2.2.1 Q：下载`nncase`仓库自己编译后，运行test出现这个错误"The configured user limit (128) on the number of inotify instances has been reached, or the per-process limit on the number of open file descriptors has been reached"。
+#### 2.2.1 Q：下载 `nncase`仓库自己编译后，运行test出现这个错误"The configured user limit (128) on the number of inotify instances has been reached, or the per-process limit on the number of open file descriptors has been reached"。
 
 A1：使用 `sudo gedit /proc/sys/fs/inotify/max_user_instances`修改128为更大的值即可。
 
+### 2.3 `initialize`相关
+#### 2.3.1 Q：编译模型出现如下错误`RuntimeError: Failed to initialize hostfxr`
+A1：需要安装dotnet-7.0
 
-
-----
+---
 
 ## 3. 推理时报错
 
-### 3.1 Q：在编译kmodel正常， 但是推理的时候出现`nncase.simulator.k230.sc: not found`的错误。
+### 3.1 Q：在编译kmodel正常， 但是推理的时候出现 `nncase.simulator.k230.sc: not found`的错误。
 
-A：需要检查`nncase`和`nncase-kpu`的版本是否一致。
+A：将nncase的安装路径加入到 `PATH`环境变量中，同时检查一下nncase和nncase-kpu版本是否一致。
 
 ```shell
 root@a52f1cacf581:/mnt# pip list | grep nncase
@@ -44,9 +44,7 @@ nncase-kpu                   2.1.1.20230721
 
 如果不一致，请安装相同版本的Python包 `pip install nncase==x.x.x.x nncase-kpu==x.x.x.x`。
 
-
-
-----
+---
 
 ## 4. k230开发板推理时报错
 

docs/USAGE_v2.md

@@ -46,7 +46,15 @@ Type "help", "copyright", "credits" or "license" for more information.
 
 k230模型编译推理参考Jupyter脚本：[User_guide](../examples/user_guide/k230_simulate.ipynb)，脚本中包含了单输入和多输入的示例。
 
-如果在Docker中运行Jupyter脚本，可以参考[配置Jupyter lab](https://github.com/kunjing96/docker-jupyterlab#32-%E9%85%8D%E7%BD%AEjupyter-lab)进行配置。
+如果在Docker中运行Jupyter脚本，可以参考以下命令，之后在浏览器窗口打开即可。
+
+```shell
+docker run -it --rm --privileged=true -p 8889:8889 --name Kendryte -v `pwd`:/mnt -w /mnt ghcr.io/kendryte/k230_sdk  /bin/bash -c "/bin/bash
+
+pip install jupyterlab
+
+jupyter-lab --ip 0.0.0.0 --allow-root
+```
 
 在执行脚本之前需要根据自身需求修改以下内容：
 
@@ -153,6 +161,8 @@ subgraph A
     end
 
 ```
+##### 动态shape参数
+详见[动态shape参数说明](./shape_bucket.md)
 
 #### 代码示例
 

docs/USAGE_v2_EN.md

@@ -46,7 +46,16 @@ Type "help", "copyright", "credits" or "license" for more information.
 
 Model compilation, inference for k230 can be found in the Jupyter script [User_guide](../examples/user_guide/k230_simulate.ipynb), this script contains single and multiple input examples.
 
-If you run Jupyter scripts in Docker, you can refer to [Configure Jupyter lab](https://github.com/kunjing96/docker-jupyterlab#32-%E9%85%8D%E7%BD%AEjupyter-lab) to configure them.
+If you run the Jupyter script in Docker, you can refer to the command and then open it in your browser.
+
+```shell
+docker run -it --rm --privileged=true -p 8889:8889 --name Kendryte -v `pwd`:/mnt -w /mnt ghcr.io/kendryte/k230_sdk  /bin/bash -c "/bin/bash
+
+pip install jupyterlab
+
+jupyter-lab --ip 0.0.0.0 --allow-root
+```
+
 
 You need to modify the following to suit your needs before executing the script:
 
@@ -154,6 +163,9 @@ subgraph A
 
 ```
 
+##### Dynamice shape args
+Refer to[Dynamic shape args description](./shape_bucket.md)
+
 #### Example
 
 ```python

docs/shape_bucket.md

@@ -0,0 +1,48 @@
+# ShapeBucket使用说明
+
+ShapeBucket是针对动态shape的一种解决方案，会根据输入长度的范围以及指定的段的数量来对动态shape进行优化。该功能默认为false，需要打开对应的option才能生效，除了指定对应的字段信息，其他流程与编译静态模型没有区别。
+
+对应的不同CompileOptions中的字段
+
+| 字段名称                    | 类型                  | 是否必须 | 描述                                                            |
+| --------------------------- | --------------------- | -------- | --------------------------------------------------------------- |
+| shape_bucket_enable         | bool                  | 是       | 是否开启ShapeBucket功能，默认为False。在 `dump_ir=True`时生效 |
+| shape_bucket_range_info     | Dict[str, [int, int]] | 是       | 每个输入shape维度信息中的变量的范围，最小值必须大于等于1        |
+| shape_bucket_segments_count | int                   | 是       | 输入变量的范围划分为几段                                        |
+| shape_bucket_fix_var_map    | Dict[str, int]        | 否       | 固定shape维度信息中的变量为特定的值                             |
+
+## onnx
+
+在模型的shape中会有些维度为变量名字，这里以一个onnx模型的输入为例
+
+> tokens: int64[batch_size, tgt_seq_len]
+>
+> step: float32[seq_len, batch_size]
+
+对应这个输入有如下的配置
+
+```python
+shape_bucket_options = nncase.ShapeBucketOptions()
+shape_bucket_options.shape_bucket_enable = True
+shape_bucket_options.shape_bucket_range_info = {"seq_len":[1, 100], "tgt_seq_len":[1, 100]}
+shape_bucket_options.shape_bucket_segments_count = 2
+shape_bucket_options.shape_bucket_fix_var_map = {"batch_size" : 3}
+```
+
+shape的维度信息中存在seq_len，tgt_seq_len，batch_size这三个变量。首先是batch_size，虽然是变量的但实际应用的时候固定为3，因此在**fix_var_map**中添加batch_size = 3，在运行的时候会将这个维度固定为3。
+
+seq_len，tgt_seq_len两个是实际会发生改变的，因此需要配置这两个变量的实际范围，也就是**range_info**的信息。**segments_count**是实际分段的数量，会根据范围等分为几份，对应的编译时间也会相应增加几倍。
+
+## tflite
+
+tflite的模型与onnx不同，shape上暂未标注维度的名称，目前只支持输入中具有一个维度是动态的，并且名称统一配置为-1，配置方式如下
+
+```cpp
+shape_bucket_options = nncase.ShapeBucketOptions()
+shape_bucket_options.shape_bucket_enable = True
+shape_bucket_options.shape_bucket_range_info = {"-1":[1, 100]}
+shape_bucket_options.shape_bucket_segments_count = 2
+shape_bucket_options.shape_bucket_fix_var_map = {"batch_size" : 3}
+```
+
+配置完这些选项后整个编译的流程和静态shape一致。