From 6728e121ae1dfd74b25e3234037a899661ad9624 Mon Sep 17 00:00:00 2001 From: Pierre Tremblay Date: Wed, 14 Feb 2024 10:09:11 -0500 Subject: [PATCH] Added code coverage documentation. --- README_DOC.md | 3 +- doc/codeCoverageInstructions.md | 71 +++++++++++++++++++++++++++++++++ 2 files changed, 73 insertions(+), 1 deletion(-) create mode 100644 doc/codeCoverageInstructions.md diff --git a/README_DOC.md b/README_DOC.md index 4017936ca2..cbef5923e5 100644 --- a/README_DOC.md +++ b/README_DOC.md @@ -1,4 +1,5 @@ ## Plugin Documentation + [Technical details of Hydra for Maya](./doc/mayaHydraDetails.md) + [Selection in Hydra for Maya](./doc/mayaHydraSelection.md) -+ [Selection Highlighting Architecture](./doc/selectionHighlightingArchitecture.md) \ No newline at end of file ++ [Selection Highlighting Architecture](./doc/selectionHighlightingArchitecture.md) ++ [Code Coverage Instructions](./doc/codeCoverageInstructions.md) \ No newline at end of file diff --git a/doc/codeCoverageInstructions.md b/doc/codeCoverageInstructions.md new file mode 100644 index 0000000000..184af7f116 --- /dev/null +++ b/doc/codeCoverageInstructions.md @@ -0,0 +1,71 @@ +# Code Coverage For Windows + +Maya-hydra has support for obtaining code coverage on Windows platforms. + +The support uses two main tools: Clang for the compilation of maya-hydra and the LLVM toolset for the parsing of code coverage +information and the generation of a code coverage report. + +## Prerequisites + +To install Clang and the LLVM toolset, you can install an optional module with Visual Studio. Refer to these instructions: [Install Clang and LLVM Toolset](https://learn.microsoft.com/en-us/cpp/build/clang-support-msbuild?view=msvc-170) + +## Documentation References +- [llvm-profdata and llvm-cov show](https://llvm.org/docs/CommandGuide/llvm-cov.html) +- [Compiling clang with coverage](https://clang.llvm.org/docs/SourceBasedCodeCoverage.html) + +## Building the Coverage Variant + +The maya-hydra build has a Coverage variant that can be used with the following stages: clean,build,install,test (note that the test stage depends on the install stage). Clang is used with code coverage instrumentation flags enabled (-fprofile-instr-generate -fcoverage-mapping) so that when tests are run after a successful install stage, code coverage data files will be generated. + +To build the coverage variant you can run: + +``` +python build.py --generator=Ninja --stages clean,configure,build,install --maya-location --build-coverage --pxrusd-location --devkit-location --build-args="-DPYTHON_INCLUDE_DIR=,-DPython_EXECUTABLE=,-DPYTHON_LIBRARIES=,-DBUILD_WITH_PYTHON_3=ON,-DBUILD_WITH_PYTHON_3_VERSION=,-DQT_VERSION=6.5.0,-DCMAKE_WANT_MATERIALX_BUILD=ON,-DCMAKE_PREFIX_PATH=" +``` + +The --build-coverage flag indicates that the variant to be built is the Coverage variant. + +## Running Tests and Getting Raw Coverage Information + +To run tests and generate code coverage information using the Coverage build, run: + +``` +python build.py --generator=Ninja --stages test --maya-location --build-coverage --pxrusd-location --devkit-location --build-args="-DPYTHON_INCLUDE_DIR=,-DPython_EXECUTABLE=,-DPYTHON_LIBRARIES=,-DBUILD_WITH_PYTHON_3=ON,-DBUILD_WITH_PYTHON_3_VERSION=,-DQT_VERSION=6.5.0,-DCMAKE_WANT_MATERIALX_BUILD=ON,-DCMAKE_PREFIX_PATH=" +``` + +After running tests, there will be raw coverage information files generated that are have a file extension of .profraw. + +## Parsing Coverage Information and Report + +To parse the coverage information, two tools from the LLVM toolset are used: llvm-profdata and llvm-cov show. +llvm-profdata parses and merges all of the raw coverage information files into a single file that has a file extension of .profdata. +llvm-cov show uses this created file to generate an HTML report of the coverage of the maya-hydra plugin. + +There is no build stage in maya-hydra that can automatically run this step of collecting coverage information and creating a report. To complete this step manually +here are the instructions. + +To run llvm-profdata and parse all of the coverage information into a single file, run: + +``` +llvm-profdata merge -sparse -o '' ' -object=mayaHydra.mll_location -object=flowViewport.dll_location -show-branches=count -show-regions --ignore-filename-regex='artifactory\\.*' -format=html -output-dir=' +``` + +Note: +- -instr-profile refers to the previously generated profdata file, which contains all code coverage information +- --ignore-filename-regex removes any files starting with artifactory from the report + +## Viewing Results + +Go to the output directory from the llvm-cov show command and open the index.html file. Any lines that are uncovered are red. Lines that are covered have no highlighting around them. + +