Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

残りのターゲットをビルドする #26

Merged
merged 19 commits into from
Jan 9, 2024
Merged

残りのターゲットをビルドする #26

Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
6651915
残りのターゲットをビルドする
qryxip Oct 14, 2023
1bed685
パッケージ名に`_dev`を付ける
qryxip Jan 5, 2024
b330807
`cuobjdump`と`nvdisasm`を追加
qryxip Jan 5, 2024
1846780
`thrust`追加
qryxip Jan 5, 2024
8f98e1d
`cuda_sub_packages`を並べ替える
qryxip Jan 5, 2024
7ca90fe
不要な`tree`とコメントを削除
qryxip Jan 5, 2024
33b5fad
DML版とCUDA版を統合する
qryxip Jan 5, 2024
a04af35
不要な`tree`を削除
qryxip Jan 6, 2024
b62b292
Merge remote-tracking branch 'upstream/main' into build-the-rest
qryxip Jan 6, 2024
76a8294
Python 3.8のインストールを削除
qryxip Jan 6, 2024
c1462d3
`artifact_name`の修正
qryxip Jan 6, 2024
c6ef5c6
Minor refactor
qryxip Jan 6, 2024
ab22eb1
tools/ci_build/build.pyを直接実行する
qryxip Jan 6, 2024
a2aa7e2
v1.16.3に上げる
qryxip Jan 6, 2024
8cf3d80
[skip ci] "for debug"と明記する
qryxip Jan 9, 2024
e734b03
[skip ci] "for debug"
qryxip Jan 9, 2024
3de493b
[skip ci] "for non-x86_64 Linux"と明記する
qryxip Jan 9, 2024
756a313
`arch` → `linux-cross-arch`
qryxip Jan 9, 2024
2ac685f
`cuda_sub_package`についてコメント
qryxip Jan 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
172 changes: 143 additions & 29 deletions .github/workflows/build.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ on:
env:
ONNXRUNTIME_VERSION:
|- # releaseタグ名か、workflow_dispatchでのバージョン名が入る。無指定なら適当なバージョン
${{ github.event.release.tag_name || github.event.inputs.version || '1.15.1' }}
${{ github.event.release.tag_name || github.event.inputs.version || '1.16.3' }}
RELEASE:
|- # releaseタグ名か、workflow_dispatchでのreleaseフラグがあればリリースする
${{ github.event.release.tag_name != '' || github.event.inputs.release == 'true' }}
Expand All @@ -34,48 +34,100 @@ jobs:
fail-fast: false
matrix:
include:
- artifact_name: onnxruntime-win-x64
os: windows-2022
build_opts: --cmake_extra_defines CMAKE_SYSTEM_NAME=Windows CMAKE_SYSTEM_PROCESSOR=x86_64 --config Release --parallel --compile_no_warning_as_error --update --build --build_shared_lib
result_dir: build/Release
release_config: Release
- artifact_name: onnxruntime-win-x64-gpu
os: windows-2022
cuda_version: 11.8.0
# Windowsの場合デフォルトのパッケージ群では不十分であるため、必要そうなパッケージを指定する。ただしいくつかは不要かもしれない
cuda_sub_packages: '["cudart", "cuobjdump", "nvcc", "nvdisasm", "thrust", "cublas_dev", "cufft_dev", "curand_dev", "cusolver_dev", "cusparse_dev", "visual_studio_integration"]'
cudnn_url: https://developer.download.nvidia.com/compute/redist/cudnn/v8.5.0/local_installers/11.7/cudnn-windows-x86_64-8.5.0.96_cuda11-archive.zip
build_opts: --cmake_extra_defines CMAKE_SYSTEM_NAME=Windows CMAKE_SYSTEM_PROCESSOR=x86_64 --config Release --parallel --compile_no_warning_as_error --update --build --build_shared_lib --use_dml --use_cuda --cuda_version 11.8
result_dir: build/Release
release_config: Release
- artifact_name: onnxruntime-win-x86
os: windows-2022
build_opts: --x86 --cmake_extra_defines CMAKE_SYSTEM_NAME=Windows CMAKE_SYSTEM_PROCESSOR=x86 --config Release --parallel --compile_no_warning_as_error --update --build --build_shared_lib
result_dir: build/Release
release_config: Release
- artifact_name: onnxruntime-linux-x64
os: ubuntu-20.04
build_opts: --cmake_extra_defines CMAKE_SYSTEM_NAME=Linux CMAKE_SYSTEM_PROCESSOR=x86_64 --config Release --parallel --compile_no_warning_as_error --update --build --build_shared_lib
result_dir: build
release_config: Release
- artifact_name: onnxruntime-linux-x64-gpu
os: ubuntu-20.04
cuda_version: 11.8.0
cuda_sub_packages: "[]" # デフォルト
cudnn_url: https://developer.download.nvidia.com/compute/redist/cudnn/v8.2.4/cudnn-11.4-linux-x64-v8.2.4.15.tgz
build_opts: --cmake_extra_defines CMAKE_SYSTEM_NAME=Linux CMAKE_SYSTEM_PROCESSOR=x86_64 --config Release --parallel --compile_no_warning_as_error --update --build --build_shared_lib --use_cuda
result_dir: build
release_config: Release
- artifact_name: onnxruntime-linux-armhf
os: ubuntu-20.04
cc_version: "8"
cxx_version: "8"
arch: arm-linux-gnueabihf
cc_version: "9"
cxx_version: "9"
linux_cross_arch: arm-linux-gnueabihf
symlink_workaround: true
build_opts: --arm --cmake_extra_defines CMAKE_SYSTEM_NAME=Linux CMAKE_SYSTEM_PROCESSOR=armv7l --config Release --parallel --update --build --build_shared_lib
result_dir: build/Linux
build_opts: --arm --cmake_extra_defines CMAKE_SYSTEM_NAME=Linux CMAKE_SYSTEM_PROCESSOR=armv7l --config Release --parallel --compile_no_warning_as_error --update --build --build_shared_lib
result_dir: build
release_config: Release
- artifact_name: onnxruntime-linux-arm64
os: ubuntu-20.04
cc_version: "8"
cxx_version: "8"
arch: aarch64-linux-gnu
cc_version: "9"
cxx_version: "9"
linux_cross_arch: aarch64-linux-gnu
symlink_workaround: true
build_opts: --arm64 --cmake_extra_defines CMAKE_SYSTEM_NAME=Linux CMAKE_SYSTEM_PROCESSOR=aarch64 --config Release --parallel --update --build --build_shared_lib
result_dir: build/Linux
build_opts: --arm64 --cmake_extra_defines CMAKE_SYSTEM_NAME=Linux CMAKE_SYSTEM_PROCESSOR=aarch64 --config Release --parallel --compile_no_warning_as_error --update --build --build_shared_lib
result_dir: build
release_config: Release
- artifact_name: onnxruntime-osx-arm64
os: macos-12
Comment on lines +87 to +88
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

あ、やらかした。macos-11で使えない...
https://github.com/VOICEVOX/voicevox_core/actions/runs/7483136460/job/20367924235?pr=725

build_opts: --arm64 --cmake_extra_defines CMAKE_SYSTEM_NAME=Darwin CMAKE_SYSTEM_PROCESSOR=aarch64 --config Release --parallel --compile_no_warning_as_error --update --build --build_shared_lib
result_dir: build
release_config: Release
- artifact_name: onnxruntime-osx-x86_64
os: macos-12
build_opts: --cmake_extra_defines CMAKE_SYSTEM_NAME=Darwin CMAKE_SYSTEM_PROCESSOR=x86_64 --config Release --parallel --compile_no_warning_as_error --update --build --build_shared_lib
result_dir: build
release_config: Release
- artifact_name: onnxruntime-android-x64
os: ubuntu-20.04
build_opts: --android_abi x86_64 --cmake_extra_defines CMAKE_SYSTEM_NAME=Android CMAKE_SYSTEM_PROCESSOR=x86_64 --config Release --parallel --compile_no_warning_as_error --update --build --build_shared_lib --android
result_dir: build
release_config: Release
- artifact_name: onnxruntime-android-arm64
os: ubuntu-20.04
build_opts: --android_abi arm64-v8a --cmake_extra_defines CMAKE_SYSTEM_NAME=Android CMAKE_SYSTEM_PROCESSOR=aarch64 --config Release --parallel --compile_no_warning_as_error --update --build --build_shared_lib --android
result_dir: build
release_config: Release
- artifact_name: onnxruntime-ios-arm64
os: macos-12
build_opts: --config Release --parallel --update --build --build_shared_lib --skip_tests --use_xcode --ios --ios_sysroot iphoneos --osx_arch arm64 --apple_deploy_target 16.0
build_opts: --config Release --parallel --compile_no_warning_as_error --update --build --build_shared_lib --skip_tests --use_xcode --ios --ios_sysroot iphoneos --osx_arch arm64 --apple_deploy_target 16.0
build_opts_workaround_protoc: --path_to_protoc_exe /usr/local/opt/protobuf@21/bin/protoc
result_dir: build/iOS/Release
result_dir: build/Release
release_config: Release-iphoneos
- artifact_name: onnxruntime-ios-sim-arm64
os: macos-12
build_opts: --config Release --parallel --update --build --build_shared_lib --skip_tests --use_xcode --ios --ios_sysroot iphonesimulator --osx_arch arm64 --apple_deploy_target 16.0
build_opts: --config Release --parallel --compile_no_warning_as_error --update --build --build_shared_lib --skip_tests --use_xcode --ios --ios_sysroot iphonesimulator --osx_arch arm64 --apple_deploy_target 16.0
build_opts_workaround_protoc: --path_to_protoc_exe /usr/local/opt/protobuf@21/bin/protoc
result_dir: build/iOS/Release
result_dir: build/Release
release_config: Release-iphonesimulator
- artifact_name: onnxruntime-ios-sim-x86_64
os: macos-12
build_opts: --config Release --parallel --update --build --build_shared_lib --skip_tests --use_xcode --ios --ios_sysroot iphonesimulator --osx_arch x86_64 --apple_deploy_target 16.0
build_opts: --config Release --parallel --compile_no_warning_as_error --update --build --build_shared_lib --skip_tests --use_xcode --ios --ios_sysroot iphonesimulator --osx_arch x86_64 --apple_deploy_target 16.0
build_opts_workaround_protoc: --path_to_protoc_exe /usr/local/opt/protobuf@21/bin/protoc
result_dir: build/iOS/Release
result_dir: build/Release
release_config: Release-iphonesimulator

env:
# prefix usage: "", "arm-linux-gnueabihf-" => "gcc-8", "arm-linux-gnueabihf-gcc-8" (command name)
# suffix usage: "", "-arm-linux-gnueabihf" => "gcc-8", "gcc-8-arm-linux-gnueabihf" (package name)
ARCH_PREFIX: "${{ (matrix.arch != '' && matrix.arch) || '' }}${{ (matrix.arch != '' && '-') || '' }}"
ARCH_SUFFIX: "${{ (matrix.arch != '' && '-') || '' }}${{ (matrix.arch != '' && matrix.arch) || '' }}"
ARCH_PREFIX: "${{ (matrix.linux_cross_arch != '' && matrix.linux_cross_arch) || '' }}${{ (matrix.linux_cross_arch != '' && '-') || '' }}"
ARCH_SUFFIX: "${{ (matrix.linux_cross_arch != '' && '-') || '' }}${{ (matrix.linux_cross_arch != '' && matrix.linux_cross_arch) || '' }}"

runs-on: ${{ matrix.os }}

Expand Down Expand Up @@ -113,8 +165,18 @@ jobs:
path: build/
key: ${{ matrix.artifact_name }}-v${{ env.ONNXRUNTIME_VERSION }}-cache-v1-${{ hashFiles('matrix.json') }}

- name: Install tree
if: steps.cache-build-result.outputs.cache-hit != 'true' && runner.os == 'Windows'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cache-build-result、このジョブのアップロード以外の全てに入ってて、あまりキャッシュの意味がない気がちょっとしました！
（まあこちらのプルリクエストには関係ないのですが）

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

このjobにおけるcacheですが、意味合いとしては「アップロード以外の全て」で正しいという理解を私はしています。というのもキャッシュ対象は「ビルドが成功した状態のbuild/全体」なので。

むしろ #24 のもこっち側に移すのがよいのではと思っています。

Copy link
Member

@Hiroshiba Hiroshiba Jan 9, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

なるほどです！

おっしゃることは結構わかるのですが、結局キャッシュがあった場合にできるのは「完成物を再アップロードする」だけで、それが有効なのは間違えてreleaseを削除してしまった時くらいなんですよね。
デバッグなどで不要なのをビルドしたくない時は、不要などコメントアウトした状態でチェックすれば良いので･･･。

他にメリットといえば、build-xcframeworkの後続タスクに便利だとは思います。

そのメリットと、ifを書き忘れて実行してしまったせいで変なキャッシュが出来上がって気づかずにリリースに支障をきたしてしまうのとで、リスクの大きさと天秤にかける必要があるなあと。
例えばarm linux版のビルドだけ変なファイルが入っちゃって、それに気づかずリリース＋コアビルド＋エンジンビルドまでしてしまって、エンジンのリリースをした後にarm版が動かないissueが経って、ここまで戻ってきてエンジン再リリース、みたいな･･･。

色々考えましたが、リスクは承知で、まあ慎重にコードを書いて行く感じで良いかなと思いました！
ミスったら笑いましょう！

それとは別に、どちらの利点も得る方法もあると思います。
例えばシェルスクリプト全部一つにまとめてしまって、ステップを1つにしてキャッシュ有無のifを1回だけにするとか。
そこを目指すissueを立てるとかが良いのかなと思いました。

uses: msys2/setup-msys2@v2
with:
install: tree

- name: Install tree for debug
if: steps.cache-build-result.outputs.cache-hit != 'true' && runner.os == 'macOS'
run: brew install tree

- name: Install build dependencies on ubuntu
if: steps.cache-build-result.outputs.cache-hit != 'true' && startsWith(matrix.os, 'ubuntu')
if: steps.cache-build-result.outputs.cache-hit != 'true' && startsWith(matrix.os, 'ubuntu') && matrix.linux_cross_arch
run: |
sudo apt-get update
sudo apt-get install -y \
Expand All @@ -131,17 +193,54 @@ jobs:
# Workaround for protoc https://github.com/microsoft/onnxruntime/issues/16238#issuecomment-1590398821
brew install protobuf@21

# ONNX Runtime v1.15.1 requires CMake 3.26 or higher.
# ONNX Runtime v1.16.1 requires CMake 3.27 or higher.
- name: Install CMake
if: steps.cache-build-result.outputs.cache-hit != 'true' && startsWith(matrix.os, 'ubuntu')
env:
CMAKE_VERSION: 3.26.4
CMAKE_VERSION: 3.27.7
run: |
wget -O cmake.sh "https://github.com/Kitware/CMake/releases/download/v${{ env.CMAKE_VERSION }}/cmake-${{ env.CMAKE_VERSION }}-linux-x86_64.sh"
sudo bash cmake.sh --skip-license --prefix=/usr/local

- name: Configure build environment
if: steps.cache-build-result.outputs.cache-hit != 'true' && startsWith(matrix.os, 'ubuntu')
- name: Set up CUDA
if: steps.cache-build-result.outputs.cache-hit != 'true' && matrix.cuda_version && matrix.cudnn_url
uses: Jimver/[email protected]
with:
cuda: ${{ matrix.cuda_version }}
sub-packages: ${{ matrix.cuda_sub_packages }}
method: network

- name: Set `$CUDA_HOME`
if: steps.cache-build-result.outputs.cache-hit != 'true' && matrix.cuda_version && matrix.cudnn_url
run: echo "CUDA_HOME=$CUDA_PATH" >> "$GITHUB_ENV"

- name: Set up cuDNN
if: steps.cache-build-result.outputs.cache-hit != 'true' && matrix.cuda_version && matrix.cudnn_url
run: |
mkdir download

if [ ${{ runner.os }} = Windows ]; then
curl -L "${{ matrix.cudnn_url }}" > download/cudnn.zip
unzip download/cudnn.zip -d download
rm download/cudnn.zip
else
curl -L "${{ matrix.cudnn_url }}" > download/cudnn.tar.gz
tar xvf download/cudnn.tar.gz -C download/
rm download/cudnn.tar.gz
fi

cudnn_path=$(find download -maxdepth 1 -name 'cudnn-*')
: "${cudnn_path:=./download/cuda}" # https://developer.download.nvidia.com/compute/redist/cudnn/v8.2.4/cudnn-11.4-linux-x64-v8.2.4.15.tgz
if [ ${{ runner.os }} = Windows ]; then
cudnn_path=$(cygpath -wa "$cudnn_path")
else
cudnn_path=$(realpath "$cudnn_path")
fi

echo "CUDNN_HOME=$cudnn_path" >> "$GITHUB_ENV"

- name: Configure build environment for non-x86_64 Linux
if: steps.cache-build-result.outputs.cache-hit != 'true' && startsWith(matrix.os, 'ubuntu') && matrix.linux_cross_arch
run: |
# Required for arm build
# https://github.com/microsoft/onnxruntime/issues/4189#issuecomment-642528278
Expand All @@ -150,8 +249,8 @@ jobs:

# Prevent Exec Format Error during cross-compiling
if ${{ matrix.symlink_workaround }}; then
find /usr/${{ matrix.arch }}/lib -name '*.so*' -exec sudo ln -s {} /usr/lib/${{ matrix.arch }}/ ';'
sudo ln -s /usr/${{ matrix.arch }}/lib/ld-linux-*.so* /usr/lib/
find /usr/${{ matrix.linux_cross_arch }}/lib -name '*.so*' -exec sudo ln -s {} /usr/lib/${{ matrix.linux_cross_arch }}/ ';'
sudo ln -s /usr/${{ matrix.linux_cross_arch }}/lib/ld-linux-*.so* /usr/lib/
fi

# Set environment variable CC / CXX
Expand All @@ -165,17 +264,32 @@ jobs:
# skip test: https://github.com/microsoft/onnxruntime/issues/2436
# ONNX Runtime v1.9.0 requires CMAKE_SYSTEM_PROCESSOR, https://github.com/microsoft/onnxruntime/releases/tag/v1.9.0
# Both CMAKE_SYSTEM_NAME and CMAKE_SYSTEM_PROCESSOR are required.
bash ./build.sh ${{ matrix.build_opts }} ${{ matrix.build_opts_workaround_protoc }}
python ./tools/ci_build/build.py --build_dir ./build ${{ matrix.build_opts }} ${{ matrix.build_opts_workaround_protoc }}

- name: Inspect the build directory for debug

if: steps.cache-build-result.outputs.cache-hit != 'true'
run: |
if [ ${{ runner.os }} = Windows ]; then
tree() {
msys2.cmd -c "tree $1"
}
fi
tree build

- name: Organize artifact
run: |
# コピー先artifactを予め削除しておく
rm -rf ${{ matrix.result_dir }}/${{ matrix.artifact_name }}
rm -rf ./artifact
# Set library name
if [[ ${{ matrix.artifact_name }} == onnxruntime-linux-* ]]; then
if [[ ${{ matrix.artifact_name }} == onnxruntime-win-* ]]; then
ONNXRUNTIME_NAME=onnxruntime.dll
elif [[ ${{ matrix.artifact_name }} == onnxruntime-linux-* ]]; then
ONNXRUNTIME_NAME=libonnxruntime.so.${{ env.ONNXRUNTIME_VERSION }}
elif [[ ${{ matrix.artifact_name }} == onnxruntime-ios-* ]]; then
elif [[ ${{ matrix.artifact_name }} == onnxruntime-android-* ]]; then
ONNXRUNTIME_NAME=libonnxruntime.so
elif [[ ${{ matrix.artifact_name }} == onnxruntime-osx-* ]] || [[ ${{ matrix.artifact_name }} == onnxruntime-ios-* ]]; then
ONNXRUNTIME_NAME=libonnxruntime.${{ env.ONNXRUNTIME_VERSION }}.dylib
else
echo "Unknown target found : ${{ matrix.artifact_name }}"
Expand Down