Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disabled building tests default (like LLVM), enabled for our CIs #3021

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Disabled building tests default (like LLVM), enabled for our CIs #3021

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
6939f8e
Disabled building tests default (like LLVM), enabled for our CIs
AlexandreEichenberger Dec 2, 2024
0ac4608
add test for onnx import
AlexandreEichenberger Dec 3, 2024
974bc90
add try to onnx import
AlexandreEichenberger Dec 3, 2024
671241d
more
AlexandreEichenberger Dec 3, 2024
41bb62f
fix error
AlexandreEichenberger Dec 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ cmake_minimum_required(VERSION 3.20.0)

project(onnx-mlir)

option(ONNX_MLIR_BUILD_TESTS "Build ONNX-MLIR test executables. If OFF, just generate build targets." ON)
# Like in LLVM, default tests are off. Lit-tests are always on.
option(ONNX_MLIR_BUILD_TESTS "Build ONNX-MLIR test executables. If OFF, just generate build targets." OFF)
option(ONNX_MLIR_CCACHE_BUILD "Set to ON for a ccache enabled build." OFF)
option(ONNX_MLIR_ENABLE_STABLEHLO "Enable StableHLO support." ON)
option(ONNX_MLIR_ENABLE_WERROR "Enable warnings as errors." OFF)
Expand Down
1 change: 1 addition & 0 deletions docker/Dockerfile.onnx-mlir
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ RUN LLVM_PROJECT_ROOT=${WORK_DIR}/llvm-project \
&& CC=clang CXX=clang++ \
cmake -DMLIR_DIR=${LLVM_PROJECT_ROOT}/build/lib/cmake/mlir \
-DCMAKE_BUILD_TYPE=Release \
-DONNX_MLIR_BUILD_TESTS=ON \
-DLLVM_ENABLE_ASSERTIONS=ON \
-DCMAKE_INSTALL_MESSAGE=NEVER \
-DONNX_MLIR_ACCELERATORS=${ACCEL} .. \
Expand Down
1 change: 1 addition & 0 deletions docker/Dockerfile.onnx-mlir-dev
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ RUN LLVM_PROJECT_ROOT=${WORK_DIR}/llvm-project \
&& CC=clang CXX=clang++ \
cmake -DMLIR_DIR=${LLVM_PROJECT_ROOT}/build/lib/cmake/mlir \
-DCMAKE_BUILD_TYPE=Debug \
-DONNX_MLIR_BUILD_TESTS=ON \
-DONNX_MLIR_TEST_OPTLEVEL=0 \
-DONNX_MLIR_ACCELERATORS=${ACCEL} .. \
# Run the cppcheck scanner on s390x
Expand Down
6 changes: 6 additions & 0 deletions docs/BuildOnLinuxOSX.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,10 @@ This project uses lit ([LLVM's Integrated Tester](https://llvm.org/docs/CommandG

To build ONNX-MLIR, use the following commands (maybe with additional `-DCMAKE_CXX_FLAGS` argument described [below](#enable-cpu-optimizations)):

To build a version with extended tests, keep the `-DONNX_MLIR_BUILD_TESTS=ON` line below.
Note that the extended tests requires the installation of `onnx` via a pip install of `third_party/onnx`. To build a version without a dependence on `third_party/onnx`, remove the `-DONNX_MLIR_BUILD_TESTS=ON` line or set to `OFF`.
The lightweight Literal tests are build regardless of this option.

[same-as-file]: <> ({"ref": "utils/install-onnx-mlir.sh", "skip-doc": 2})
```bash
git clone --recursive https://github.com/onnx/onnx-mlir.git
Expand All @@ -68,13 +72,15 @@ if [[ -z "$pythonLocation" ]]; then
cmake -G Ninja \
-DCMAKE_CXX_COMPILER=/usr/bin/c++ \
-DCMAKE_BUILD_TYPE=Release \
-DONNX_MLIR_BUILD_TESTS=ON \
-DLLVM_ENABLE_ASSERTIONS=ON \
-DMLIR_DIR=${MLIR_DIR} \
..
else
cmake -G Ninja \
-DCMAKE_CXX_COMPILER=/usr/bin/c++ \
-DCMAKE_BUILD_TYPE=Release \
-DONNX_MLIR_BUILD_TESTS=ON \
-DLLVM_ENABLE_ASSERTIONS=ON \
-DPython3_ROOT_DIR=$pythonLocation \
-DMLIR_DIR=${MLIR_DIR} \
Expand Down
4 changes: 4 additions & 0 deletions docs/BuildOnWindows.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,10 @@ call cmake --build . --config Release
```
After the above commands succeed, an `onnx-mlir` executable should appear in the `Debug/bin` or `Release/bin` directory.

To build a version with extended tests, add the `-DONNX_MLIR_BUILD_TESTS=ON` line above.
Note that the extended tests requires the installation of `onnx` via a pip install of `third_party/onnx`. To build a version without a dependence on `third_party/onnx`, do not add the `-DONNX_MLIR_BUILD_TESTS=ON` line or set to `OFF`.
The lightweight Literal tests are build regardless of this option.

### Trouble shooting build issues

Check this [page](TestingHighLevel.md) for helpful hints.
4 changes: 4 additions & 0 deletions docs/doc_check/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# SPDX-License-Identifier: Apache-2.0

if (NOT ONNX_MLIR_BUILD_TESTS)
set(EXCLUDE_FROM_ALL ON)
endif()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this does anything. EXCLUDE_FROM_ALL is a target property. Setting a variable called EXCLUDE_FROM_ALL won't do anything. The proper use of EXCLUDE_FROM_ALL should be like the code in MLIR.cmake:

  if (EXCLUDE_FROM_ALL)
    add_executable(${name} EXCLUDE_FROM_ALL ${ARG_UNPARSED_ARGUMENTS})
  else()
    add_executable(${name} ${ARG_UNPARSED_ARGUMENTS})
  endif()

The variable EXCLUDE_FROM_ALL is used to decide whether to add the property EXCLUDE_FROM_ALL to a target.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gongsu832 I reused this pattern from the current onnx-mlir. I know it does not make it right, but on my Mac it just seems to work. When I set the variable as ON, it builds all the tests as before. When I set it to off, the resulting build does not recognize the make check-onyx-backend as a valid target.

Let me run the experiment on LoZ too.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume you meant check-onnx-backend since there is no check-onyx-backend.

Regardless of how the variable is set, check-onnx-backend should always be a valid target. The difference is whether it's included in the default all target.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also note that add_executable, add_library, etc. behave differently from add_custom_target. By default, add_executable, add_library, etc. will add the target to all unless you specify EXCLUDE_FROM_ALL on the target. add_custom_target, on the other hand, by default will not add the target to all unless you specify ALL on the target.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gongsu832 Are there specific steps you would like me to take? Or if it takes you less time, do you want to take over this PR to do it the right way (if the current approach is not the preferred approach).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can do it but probably won't be able to get to it until some time next week.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, there is no rush, glad it will be done using the proper approach


set(check_cmd ${Python3_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/check.py)

add_custom_target(check-doc
Expand Down
4 changes: 4 additions & 0 deletions test/accelerators/NNPA/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# SPDX-License-Identifier: Apache-2.0

if (NOT ONNX_MLIR_BUILD_TESTS)
set(EXCLUDE_FROM_ALL ON)
endif()

add_custom_target(check-onnx-backend-numerical-nnpa)

add_subdirectory(backend)
Expand Down
4 changes: 4 additions & 0 deletions test/backend-cpp/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# SPDX-License-Identifier: Apache-2.0

if (NOT ONNX_MLIR_BUILD_TESTS)
set(EXCLUDE_FROM_ALL ON)
endif()

add_custom_target(backend-cpp)
set_target_properties(backend-cpp PROPERTIES FOLDER "Tests")

Expand Down
4 changes: 4 additions & 0 deletions test/backend/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# SPDX-License-Identifier: Apache-2.0

if (NOT ONNX_MLIR_BUILD_TESTS)
set(EXCLUDE_FROM_ALL ON)
endif()

file(GENERATE
OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/conftest.py
INPUT ${CMAKE_CURRENT_SOURCE_DIR}/conftest.py
Expand Down
10 changes: 9 additions & 1 deletion test/backend/common.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,15 @@
import sys
import os
import ctypes
import onnx

try:
import onnx
except ImportError:
print(
"Failed to include the `onnx` package; please install to successfully use this script."
)
exit(1)

import subprocess
import variables
from variables import *
Expand Down
12 changes: 10 additions & 2 deletions test/backend/inference_backend.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,16 @@
import base64
import numpy as np
import re
import onnx
import onnx.parser

try:
import onnx
import onnx.parser
except ImportError:
print(
"Failed to include the `onnx` package; please install to successfully use this script."
)
exit(1)

import subprocess
from onnx.backend.base import Device, DeviceType, Backend
from onnx.backend.test import BackendTest
Expand Down
12 changes: 10 additions & 2 deletions test/backend/onnxmlir_node_tests.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,16 @@
################################################################################

import numpy as np
import onnx
import onnx.parser

try:
import onnx
import onnx.parser
except ImportError:
print(
"Failed to include the `onnx` package; please install to successfully use this script."
)
exit(1)

from collections import namedtuple

OnnxMlirNodeTestCase = namedtuple(
Expand Down
4 changes: 4 additions & 0 deletions test/compilerlib/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# SPDX-License-Identifier: Apache-2.0

if (NOT ONNX_MLIR_BUILD_TESTS)
set(EXCLUDE_FROM_ALL ON)
endif()

add_onnx_mlir_executable(CompilerLibTest
CompilerLibTest.cpp

Expand Down
2 changes: 2 additions & 0 deletions test/mlir/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# SPDX-License-Identifier: Apache-2.0

# Just like LLVM, ONNX_MLIR_BUILD_TESTS=false does not disable lit tests.

# Turn on lit test for specified accelerators even if the
# accelerator code itself cannot be built.
if (ONNX_MLIR_ACCELERATORS)
Expand Down
4 changes: 4 additions & 0 deletions test/modellib/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# SPDX-License-Identifier: Apache-2.0

if (NOT ONNX_MLIR_BUILD_TESTS)
set(EXCLUDE_FROM_ALL ON)
endif()

add_onnx_mlir_library(ModelLib
CategoryMapperModel.cpp
ConvModel.cpp
Expand Down
4 changes: 4 additions & 0 deletions test/numerical/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# SPDX-License-Identifier: Apache-2.0

if (NOT ONNX_MLIR_BUILD_TESTS)
set(EXCLUDE_FROM_ALL ON)
endif()

add_custom_target(numerical)
set_target_properties(numerical PROPERTIES FOLDER "Tests")

Expand Down
4 changes: 4 additions & 0 deletions test/perf/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# SPDX-License-Identifier: Apache-2.0

if (NOT ONNX_MLIR_BUILD_TESTS)
set(EXCLUDE_FROM_ALL ON)
endif()

add_onnx_mlir_library(PerfLib
PerfHelper.cpp
EXCLUDE_FROM_OM_LIBS
Expand Down
4 changes: 4 additions & 0 deletions test/unit/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# SPDX-License-Identifier: Apache-2.0

if (NOT ONNX_MLIR_BUILD_TESTS)
set(EXCLUDE_FROM_ALL ON)
endif()

add_custom_target(unittest)
set_target_properties(unittest PROPERTIES FOLDER "Tests")

Expand Down
9 changes: 8 additions & 1 deletion utils/CheckONNXModel.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,14 @@
import os
import sys
import argparse
import onnx

try:
import onnx
except ImportError:
print(
"Failed to include the `onnx` package; please install to successfully use this script."
)
exit(1)
import time
import signal
import subprocess
Expand Down
18 changes: 16 additions & 2 deletions utils/RunONNXModel.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,14 @@
import os
import sys
import argparse
import onnx

try:
import onnx
except ImportError:
print(
"Failed to include the `onnx` package; please install to successfully use this script."
)
exit(1)
import time
import signal
import subprocess
Expand Down Expand Up @@ -984,7 +991,14 @@ def run(self, outputname, input_feed, **kwargs):
if args.verify.lower() == "onnxruntime":
input_model_path = args.model
# Reference backend by using onnxruntime.
import onnxruntime

try:
import onnxruntime
except ImportError:
print(
"Failed to include the `onnxruntime` package; please install to successfully use this script."
)
exit(1)

input_feed = dict(zip(input_names, inputs))
print("Running inference using onnxruntime ...")
Expand Down
9 changes: 8 additions & 1 deletion utils/gen_onnx_mlir.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,14 @@
from typing import Any, Text, Sequence, Dict, List, Type, Set, Tuple

import pprint
import onnx

try:
import onnx
except ImportError:
print(
"Failed to include the `onnx` package; please install to successfully use this script."
)
exit(1)

parser = argparse.ArgumentParser()
parser.add_argument(
Expand Down
2 changes: 2 additions & 0 deletions utils/install-onnx-mlir.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,15 @@ if [[ -z "$pythonLocation" ]]; then
cmake -G Ninja \
-DCMAKE_CXX_COMPILER=/usr/bin/c++ \
-DCMAKE_BUILD_TYPE=Release \
-DONNX_MLIR_BUILD_TESTS=ON \
-DLLVM_ENABLE_ASSERTIONS=ON \
-DMLIR_DIR=${MLIR_DIR} \
..
else
cmake -G Ninja \
-DCMAKE_CXX_COMPILER=/usr/bin/c++ \
-DCMAKE_BUILD_TYPE=Release \
-DONNX_MLIR_BUILD_TESTS=ON \
-DLLVM_ENABLE_ASSERTIONS=ON \
-DPython3_ROOT_DIR=$pythonLocation \
-DMLIR_DIR=${MLIR_DIR} \
Expand Down
10 changes: 9 additions & 1 deletion utils/onnx2json.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,15 @@
################################################################################

import argparse
import onnx

try:
import onnx
except ImportError:
print(
"Failed to include the `onnx` package; please install to successfully use this script."
)
exit(1)

from google.protobuf.json_format import MessageToJson

parser = argparse.ArgumentParser()
Expand Down
10 changes: 9 additions & 1 deletion utils/onnx2text.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,15 @@
################################################################################

import argparse
import onnx

try:
import onnx
except ImportError:
print(
"Failed to include the `onnx` package; please install to successfully use this script."
)
exit(1)

import onnx.printer

parser = argparse.ArgumentParser()
Expand Down
10 changes: 9 additions & 1 deletion utils/onnxExternalizeData.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,15 @@
################################################################################

import argparse
import onnx

try:
import onnx
except ImportError:
print(
"Failed to include the `onnx` package; please install to successfully use this script."
)
exit(1)

import os
import sys

Expand Down
10 changes: 9 additions & 1 deletion utils/onnxmlirrun.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,15 @@

import os
import sys
import onnx

try:
import onnx
except ImportError:
print(
"Failed to include the `onnx` package; please install to successfully use this script."
)
exit(1)

import time
import signal
import subprocess
Expand Down
9 changes: 8 additions & 1 deletion utils/pre-onnx-mlir.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,14 @@
python pre-onnx-mlir add.onnx -v --no_convert
"""

import onnx
try:
import onnx
except ImportError:
print(
"Failed to include the `onnx` package; please install to successfully use this script."
)
exit(1)

import argparse
from onnx import version_converter, helper

Expand Down
Loading