Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add TVM deploy #1326

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

empty line

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

4 changes: 4 additions & 0 deletions scripts/deployment/tvm-deploy/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
build/*
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove .git files under this dir

pack/*
!pack/pack.py
!pack/imagenet_classes.txt
23 changes: 23 additions & 0 deletions scripts/deployment/tvm-deploy/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
cmake_minimum_required(VERSION 3.6)
project(gluoncv_lite)


set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -ldl -pthread")
#SET(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
#SET(CMAKE_LIBRARY_OUTPUT_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR})
SET(HOME_TVM ${CMAKE_CURRENT_SOURCE_DIR}/tvm)

INCLUDE_DIRECTORIES(${HOME_TVM}/include)
INCLUDE_DIRECTORIES(${HOME_TVM}/3rdparty/dmlc-core/include)
INCLUDE_DIRECTORIES(${HOME_TVM}/3rdparty/dlpack/include)

set(PNG_LIBS libpng.a)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't find libpng, libz, libjpeg anywhere in the PR so this much be something hardcoded.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we want to build libjpeg, libpng statically and link to them, these libraries should be prepared before building the full project. I didn't provide these files because they depend on specific operating system. I write some comments about the procedure in README and thought developers would build these libraries according to their target platform. Could you please give me some suggestions on how to make changes? Thanks.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ideal way is to provide the build system for libjpeg and libpng in a cross-platform way(cmake).
We can include the source of libpng/jpeg in this case. Build for static linking, and finally include them in this demo probject CMakeList.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is using CMake, Why not use find_package() for these?

set(Z_LIBS libz.a)
set(JPEG_LIBS libjpeg.a)
link_directories(${PROJECT_SOURCE_DIR}/build)

add_executable(gcv_lite_classify tvm_runtime_pack.cc src/classification/classification.cpp)
target_link_libraries(gcv_lite_classify ${CMAKE_DL_LIBS} ${JPEG_LIBS} ${PNG_LIBS} ${Z_LIBS})

add_executable(gcv_lite_detection tvm_runtime_pack.cc src/detection/detection.cpp)
target_link_libraries(gcv_lite_detection ${CMAKE_DL_LIBS} ${JPEG_LIBS} ${PNG_LIBS} ${Z_LIBS})
60 changes: 60 additions & 0 deletions scripts/deployment/tvm-deploy/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# GluonCV lite models
This is a demo application which illustrates how to use pretrained GluonCV models in c++ environments with the support of TVM.

## Build instruction
Please clone the full TVM repository by `git clone --recursive https://github.com/apache/incubator-tvm.git tvm`

Since we want to build libjpeg and libpng statically and link to them in order that the executables could work on user's clean environment, you cannot directly build using the existing files.

You need to statically build Zlib, libjpeg and libpng, the outputs should be libz.a, libpng.a and libjpeg.a respectively. I didn't provide these files because they depend on specific operating system. Remember to add the path of the three output `*.a` files using `link_directories()` in CMakeLists.txt. During my building process, I put those three files in the `build` directory under current path which correspond to the command of line 17 in CMakeLists.txt.

Also remember to add path of the libraries' header files using `INCLUDE_DIRECTORIES()` in CMakeLists.txt.

After doing things above, you can build with the following commands:


```
mkdir -p build && cd build
cmake ..
make
```

I also write a script to automatically pack binary with different models in the pack folder.

Some prebuilt binaries are uploaded to [this website](https://zyliu-cv.s3-us-west-2.amazonaws.com/gluoncv-lite/index.html).

Usage of image classification models:
```
SYNOPSIS
./<model name> <image file> [-o <outfile>] [--class-file <classfile>]
[--topk <topk>]

OPTIONS
-o, --output <outfile>
output file for text results

--class-file <classfile>
plain text file for class names, one name per line

--topk <topk> number of the most probable classes to show as output
```

Usage of object detection models:
```
SYNOPSIS
./<model name> <image file> [-f <outfile>] [-i <outimagr>]
[--class-file <classfile>] [-t <thresh>]

OPTIONS
-f, <outfile>
output text file

-i, <outimage>
output image

--class-file <classfile>
plain text file for class names, one name per line

-t, --thresh <thresh>
Visualize threshold, from 0 to 1, default 0.3.
```
Loading