-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added code coverage documentation. #70
Merged
Merged
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,5 @@ | ||
## Plugin Documentation | ||
+ [Technical details of Hydra for Maya](./doc/mayaHydraDetails.md) | ||
+ [Selection in Hydra for Maya](./doc/mayaHydraSelection.md) | ||
+ [Selection Highlighting Architecture](./doc/selectionHighlightingArchitecture.md) | ||
+ [Selection Highlighting Architecture](./doc/selectionHighlightingArchitecture.md) | ||
+ [Code Coverage Instructions](./doc/codeCoverageInstructions.md) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,71 @@ | ||
# Code Coverage For Windows | ||
|
||
Maya-hydra has support for obtaining code coverage on Windows platforms. | ||
|
||
The support uses two main tools: Clang for the compilation of maya-hydra and the LLVM toolset for the parsing of code coverage | ||
information and the generation of a code coverage report. | ||
|
||
## Prerequisites | ||
|
||
To install Clang and the LLVM toolset, you can install an optional module with Visual Studio. Refer to these instructions: [Install Clang and LLVM Toolset](https://learn.microsoft.com/en-us/cpp/build/clang-support-msbuild?view=msvc-170) | ||
|
||
## Documentation References | ||
- [llvm-profdata and llvm-cov show](https://llvm.org/docs/CommandGuide/llvm-cov.html) | ||
- [Compiling clang with coverage](https://clang.llvm.org/docs/SourceBasedCodeCoverage.html) | ||
|
||
## Building the Coverage Variant | ||
|
||
The maya-hydra build has a Coverage variant that can be used with the following stages: clean,build,install,test (note that the test stage depends on the install stage). Clang is used with code coverage instrumentation flags enabled (-fprofile-instr-generate -fcoverage-mapping) so that when tests are run after a successful install stage, code coverage data files will be generated. | ||
|
||
To build the coverage variant you can run: | ||
|
||
``` | ||
python build.py --generator=Ninja --stages clean,configure,build,install --maya-location <maya_location> --build-coverage --pxrusd-location <pxrusd_location> --devkit-location <devkit_location> --build-args="-DPYTHON_INCLUDE_DIR=<python_include_dir>,-DPython_EXECUTABLE=<python_executable>,-DPYTHON_LIBRARIES=<python_libraries>,-DBUILD_WITH_PYTHON_3=ON,-DBUILD_WITH_PYTHON_3_VERSION=<version_of_python>,-DQT_VERSION=6.5.0,-DCMAKE_WANT_MATERIALX_BUILD=ON,-DCMAKE_PREFIX_PATH=<cmake_prefix_path>" <installation_location> | ||
``` | ||
|
||
The --build-coverage flag indicates that the variant to be built is the Coverage variant. | ||
|
||
## Running Tests and Getting Raw Coverage Information | ||
|
||
To run tests and generate code coverage information using the Coverage build, run: | ||
|
||
``` | ||
python build.py --generator=Ninja --stages test --maya-location <maya_location> --build-coverage --pxrusd-location <pxrusd_location> --devkit-location <devkit_location> --build-args="-DPYTHON_INCLUDE_DIR=<python_include_dir>,-DPython_EXECUTABLE=<python_executable>,-DPYTHON_LIBRARIES=<python_libraries>,-DBUILD_WITH_PYTHON_3=ON,-DBUILD_WITH_PYTHON_3_VERSION=<version_of_python>,-DQT_VERSION=6.5.0,-DCMAKE_WANT_MATERIALX_BUILD=ON,-DCMAKE_PREFIX_PATH=<cmake_prefix_path>" <installation_location> | ||
``` | ||
|
||
After running tests, there will be raw coverage information files generated that are have a file extension of .profraw. | ||
|
||
## Parsing Coverage Information and Report | ||
|
||
To parse the coverage information, two tools from the LLVM toolset are used: llvm-profdata and llvm-cov show. | ||
llvm-profdata parses and merges all of the raw coverage information files into a single file that has a file extension of .profdata. | ||
llvm-cov show uses this created file to generate an HTML report of the coverage of the maya-hydra plugin. | ||
|
||
There is no build stage in maya-hydra that can automatically run this step of collecting coverage information and creating a report. To complete this step manually | ||
here are the instructions. | ||
|
||
To run llvm-profdata and parse all of the coverage information into a single file, run: | ||
|
||
``` | ||
llvm-profdata merge -sparse -o '<profdata_file_name>' '<profraw_file_location(s)' | ||
``` | ||
|
||
Note: | ||
- Multiple profraw file locations can be specified, each of which should be separated by a space after the profraw file name | ||
- The profdata file name must end with the file extension .profdata | ||
|
||
To run llvm-cov show and to generate an HTML report containing code coverage information, run: | ||
|
||
``` | ||
llvm-cov show -instr-profile=<profdata_file_location> <mayaHydraLib.dll_location> -object=mayaHydra.mll_location -object=flowViewport.dll_location -show-branches=count -show-regions --ignore-filename-regex='artifactory\\.*' -format=html -output-dir=<output_dir_name>' | ||
``` | ||
|
||
Note: | ||
- -instr-profile refers to the previously generated profdata file, which contains all code coverage information | ||
- --ignore-filename-regex removes any files starting with artifactory from the report | ||
|
||
## Viewing Results | ||
|
||
Go to the output directory from the llvm-cov show command and open the index.html file. Any lines that are uncovered are red. Lines that are covered have no highlighting around them. | ||
|
||
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we refer to the build doc here? Regardless, we could simplify the example slightly :
BUILD_WITH_PYTHON_3
no longer existsBUILD_WITH_PYTHON_3_VERSION
is optional, by default automatically determined based on the Maya versionQT_VERSION
: not sure if this is even actually used (the CMake script usesWANT_QT_VERSION
), but in any case, it is not required, as the Qt version to use is also automatically determined based on the Maya versionSame goes for the other example below