Skip to content

Commit

Permalink
.pytool/Plugin: Better Rust Support (#580)
Browse files Browse the repository at this point in the history
## Description

Updates the RustHostUnitTestPlugin to allow the developer to enable or
disable the code coverage portion of the plugin.

Additionally fixes a bug in which doctests marked as ignored were still
causing the CI plugin to fail.

Additionally turns off code coverage if it detects its running on a
Windows ARM device as tarpaulin does not support that type of device.

- [ ] Impacts functionality?
- **Functionality** - Does the change ultimately impact how firmware
functions?
- Examples: Add a new library, publish a new PPI, update an algorithm,
...
- [ ] Impacts security?
- **Security** - Does the change have a direct security impact on an
application,
    flow, or firmware?
  - Examples: Crypto algorithm change, buffer overflow fix, parameter
    validation improvement, ...
- [ ] Breaking change?
- **Breaking change** - Will anyone consuming this change experience a
break
    in build or boot behavior?
- Examples: Add a new library class, move a module to a different repo,
call
    a function in a new library class in a pre-existing module, ...
- [ ] Includes tests?
  - **Tests** - Does the change include any explicit test code?
  - Examples: Unit tests, integration tests, robot tests, ...
- [ ] Includes documentation?
- **Documentation** - Does the change contain explicit documentation
additions
    outside direct code modifications (and comments)?
- Examples: Update readme file, add feature readme file, link to
documentation
    on an a separate Web page, ...

## How This Was Tested

CI

## Integration Instructions

CI
  • Loading branch information
Javagedes authored and kenlautner committed Oct 17, 2023
1 parent 074ae3b commit 8e6340a
Show file tree
Hide file tree
Showing 3 changed files with 39 additions and 17 deletions.
16 changes: 10 additions & 6 deletions .pytool/Plugin/RustHostUnitTestPlugin/Readme.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,25 @@
# Rust Host Unit Test Plugin

This CI plugin runs all unit tests with coverage enabled, calculating coverage results on a per package basis. It filters results to only calculate coverage on files within the package.

This CI plugin will also calculate coverage for the entire workspace.
This CI plugin runs all unit tests and requires that they pass. If code coverage is turned on with
`"CalculateCoverage": true`, then coverage percentages will be calculated on a per rust crate basis.

## Plugin Customizations

As a default, this plugin requires 75% coverage, though this can be configured within a packages ci.yaml file by adding the entry `RustCoverageCheck`. The required coverage percent can also be customized on a per (rust) package bases.
- `CalculateCoverage`: true / false - Whether or not to calculate coverage results
- `Coverage`: int (0, 1) - The percentage of coverage required to pass the CI check, if `CalculateCoverage` is enabled
- `CoverageOverrides` int (0, 1) - Crate specific override of percentage needed to pass

As a default, Calculating Coverage is enabled and at least 75% (.75) code coverage is required to pass.

### Example ci settings

``` yaml
"RustHostUnitTestPlugin": {
"CalculateCoverage": true,
"Coverage": 1,
"CoverageOverrides": {
"DxeRust": 0.0,
"UefiEventLib": 0.0,
"DxeRust": 0.4,
"UefiEventLib": 0.67,
}
}
```
15 changes: 13 additions & 2 deletions .pytool/Plugin/RustHostUnitTestPlugin/RustHostUnitTestPlugin.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
from pathlib import Path
import re
import logging
import platform

class RustHostUnitTestPlugin(ICiBuildPlugin):
def GetTestName(self, packagename: str, environment: object) -> tuple[str, str]:
Expand All @@ -20,9 +21,14 @@ def RunsOnTargetList(self) -> List[str]:
return ["NO-TARGET"]

def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, environment, PLM, PLMHelper, tc, output_stream):

ws = Edk2pathObj.WorkspacePath
rust_ws = PLMHelper.RustWorkspace(ws) # .pytool/Plugin/RustPackageHelper

with_coverage = pkgconfig.get("CalculateCoverage", True)

if platform.system() == "Windows" and platform.machine() == 'aarch64':
logging.debug("Coverage skipped by plugin, not supported on Windows ARM")
with_coverage = False

# Build list of packages that are in the EDK2 package we are running CI on
pp = Path(Edk2pathObj.GetAbsolutePathOnThisSystemFromEdk2RelativePath(packagename))
Expand All @@ -43,7 +49,7 @@ def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, environment, PLM,

# Run tests and evaluate results
try:
results = rust_ws.coverage(crate_name_list, ignore_list = ignore_list, report_type = "xml")
results = rust_ws.test(crate_name_list, ignore_list = ignore_list, report_type = "xml", coverage=with_coverage)
except RuntimeError as e:
logging.warning(str(e))
tc.LogStdError(str(e))
Expand All @@ -66,6 +72,11 @@ def RunBuildPlugin(self, packagename, Edk2pathObj, pkgconfig, environment, PLM,
tc.SetFailed(f'Host unit tests failed. Failures {failed}', "CHECK_FAILED")
return failed

# Return if we are not running with coverage
if not with_coverage:
tc.SetSuccess()
return 0

# Calculate coverage
coverage = {}
for file, cov in results["coverage"].items():
Expand Down
25 changes: 16 additions & 9 deletions .pytool/Plugin/RustPackageHelper/RustPackageHelper.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,23 +55,26 @@ def __set_members(self):

self.members = list(members)

def coverage(self, pkg_list = None, ignore_list = None, report_type: str = "html" ):
"""Runs coverage at the workspace level.
def test(self, pkg_list: list[str] = None, ignore_list: list[str] = None, report_type: str = "html", coverage: bool = True):
"""Runs tests on a list of rust packages / crates.
Generates a single report that provides coverage information for all
packages in the workspace.
Will additionally calculate code coverage if coverage is set to True.
"""
if pkg_list is None:
pkg_list = [pkg.name for pkg in self.members]

# Set up the command
command = "cargo"
params = "make"
if ignore_list:
params += f' -e COV_FLAGS="--out {report_type} --exclude-files {" --exclude-files ".join(ignore_list)}"'

if coverage:
if ignore_list:
params += f' -e COV_FLAGS="--out {report_type} --exclude-files {" --exclude-files ".join(ignore_list)}"'
else:
params += f' -e COV_FLAGS="--out {report_type}"'
params += f" coverage {','.join(pkg_list)}"
else:
params += f' -e COV_FLAGS="--out {report_type}"'
params += f" coverage {','.join(pkg_list)}"
params += f" test {','.join(pkg_list)}"

# Run the command
output = io.StringIO()
Expand All @@ -96,9 +99,10 @@ def coverage(self, pkg_list = None, ignore_list = None, report_type: str = "html
line = line.replace("test ", "")
if line.endswith("... ok"):
result["pass"].append(line.replace(" ... ok", ""))
elif line.endswith("... ignored"):
continue
else:
result["fail"].append(line.replace(" ... FAILED", ""))
continue

# Command failed, but we didn't parse any failed tests
if return_value != 0 and len(result["fail"]) == 0:
Expand All @@ -107,6 +111,9 @@ def coverage(self, pkg_list = None, ignore_list = None, report_type: str = "html
if len(result["fail"]) > 0:
return result

if not coverage:
return result

# Determine coverage if all tests passed
for line in lines:
line = line.strip().strip("\n")
Expand Down

0 comments on commit 8e6340a

Please sign in to comment.