Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restrict and better document the calculation and reporting of coverage #95

Merged
merged 9 commits into from
Oct 31, 2023
33 changes: 27 additions & 6 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,18 @@ on:
branches: [ master ]

# Create one single job
# This job performs all necessary checks
# This job performs all of the necessary checks
jobs:
build:
# Use the latest version of Ubuntu, MacOS, and Windows
# Use the latest and most stable version of Python
# Important: test coverage monitoring and reporting
# through a badge and the GitHub Actions job summary
# only takes place with the Linux operating system.
# Important: the MacOS and Windows operating systems
# have test coverage calculation take place but they
# do not report the test coverage beyond its display
# inside of the GitHub Actions panel for that job.
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
Expand Down Expand Up @@ -71,7 +78,7 @@ jobs:
if: always()
run: |
poetry run task lint
# Run the program
# Run the program
- name: Run program
if: always()
run: |
Expand All @@ -81,17 +88,31 @@ jobs:
if: always()
run: |
poetry run task test
# Run the test coverage monitoring
- name: Run Test Coverage
if: always()
# Run and collect the test coverage
# Important: only run and collect test coverage monitoring on Linux
- name: Run and Collect Test Coverage - Linux Only
if: always() && matrix.os == 'ubuntu-latest'
run: |
poetry run task test-coverage-silent > coverage.txt
# Display the Coverage Report
- name: Display Coverage
# Important: only report the monitored test coverage on Linux
- name: Display Collected Test Coverage - Linux Only
if: always() && matrix.os == 'ubuntu-latest'
run: |
export TOTAL=$(python -c "import json;print(json.load(open('coverage.json'))['totals']['percent_covered_display'])")
echo "total=$TOTAL" >> $GITHUB_ENV
echo "### Total coverage: ${TOTAL}%" >> $GITHUB_STEP_SUMMARY
CURRENT_GITHUB_STEP_SUMMARY="\`\`\`\n$(cat coverage.txt)\n\`\`\`"
echo "$CURRENT_GITHUB_STEP_SUMMARY" >> $GITHUB_STEP_SUMMARY
# Run and display the test coverage
# If the current operating system is MacOS, then only run test
# coverage monitoring and display it inside of the GitHub Actions
# panel. This allows for test coverage to be calculated for each
# operating system. However, coverage is only reported for Linux
# through the badge and through the GitHub job summary. Do not
# run any test coverage monitoring in Windows because it seems
# to be much slower and cause hypothesis-based tests to fail.
- name: Run and Report Test Coverage - MacOS Only
if: always() && matrix.os == 'macOS'
run: |
poetry run task test-coverage
3 changes: 2 additions & 1 deletion tests/test_validate.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
"""Pytest test suite for the validate module."""

import pytest
from hypothesis import given, strategies
from hypothesis import HealthCheck, given, settings, strategies
from hypothesis_jsonschema import from_schema

from chasten.validate import JSON_SCHEMA_CONFIG, validate_configuration
Expand Down Expand Up @@ -44,6 +44,7 @@ def test_validate_empty_config(config):


@given(from_schema(JSON_SCHEMA_CONFIG))
@settings(suppress_health_check=[HealthCheck.too_slow])
@pytest.mark.fuzz
def test_integers(config):
"""Use Hypothesis and the JSON schema plugin to confirm validation works for all possible valid instances."""
Expand Down