Skip to content

Commit

Permalink
Return benchmark results publication back to the main workflow
Browse files Browse the repository at this point in the history
  • Loading branch information
OptimumCode committed Jul 27, 2024
1 parent 0f031a9 commit a1fec70
Show file tree
Hide file tree
Showing 2 changed files with 68 additions and 79 deletions.
79 changes: 0 additions & 79 deletions .github/workflows/benchmark-upload.yml

This file was deleted.

68 changes: 68 additions & 0 deletions .github/workflows/benchmark.yml
Original file line number Diff line number Diff line change
Expand Up @@ -138,3 +138,71 @@ jobs:
with:
name: bench-comparison-result-${{ matrix.os }}
path: ${{ env.BENCHMARK_RESULTS }}/comparison/**/*.json

upload-benchmark-results:
if: (github.event_name == 'push' || github.event_name == 'pull_request') && github.repository == 'OptimumCode/json-schema-validator'
needs:
- benchmark-matrix
runs-on: ubuntu-latest
env:
RESULTS_DIR: bench-results
permissions:
# deployments permission to deploy GitHub pages website
deployments: write
# contents permission to update benchmark contents in gh-pages branch
contents: write
# pull-requests permission to create comments on PR in case of alert
pull-requests: write
strategy:
# to make sure results are submitted one by one
max-parallel: 1
matrix:
include:
- artifact-pattern: 'bench-result-*'
results-name: KMP JSON schema validator
alert: true
- artifact-pattern: 'bench-comparison-result-*'
results-name: Compare KMP JSON schema validator
alert: false
name: 'Process benchmark results for ${{ matrix.results-name }}'
steps:
- name: 'Checkout Repository'
uses: actions/checkout@v4
- name: Download benchmark results
uses: actions/download-artifact@v4
with:
pattern: ${{ matrix.artifact-pattern }}
path: ${{ env.RESULTS_DIR }}
merge-multiple: true
- name: Show downloaded artifacts
run: tree ${{ env.RESULTS_DIR }}
- name: Prepare and join benchmark reports
id: prep
run: |
for report in $(find ./${{ env.RESULTS_DIR }} -type f -name "*.json")
do
file_name=$(basename "$report")
platform="${file_name%.*}"
jq "[ .[] | .benchmark |= \"${platform}.\" + ltrimstr(\"io.github.optimumcode.json.schema.benchmark.\") | .params |= map_values(. |= split(\"/\")[-1]) ]" $report > ${{ env.RESULTS_DIR }}/$platform.json
done
AGGREGATED_REPORT=aggregated.json
# Joined reports looks like this: [[{},{}], [{},{}]]
# We need to transform them into this: [{},{}]
ls ${{ env.RESULTS_DIR }}/*.json
jq -s '[ .[] | .[] ]' ${{ env.RESULTS_DIR }}/*.json > $AGGREGATED_REPORT
echo "report=$AGGREGATED_REPORT" >> $GITHUB_OUTPUT
- name: Store benchmark result
uses: benchmark-action/github-action-benchmark@v1
with:
name: ${{ matrix.results-name }}
tool: 'jmh'
output-file-path: ${{ steps.prep.outputs.report }}
alert-comment-cc-users: "@OptimumCode"
comment-on-alert: ${{ matrix.alert }}
summary-always: true
alert-threshold: '150%'
fail-threshold: '200%'
max-items-in-chart: 50
github-token: ${{ secrets.GITHUB_TOKEN }}
# Push and deploy GitHub pages branch automatically only if run in main repo and not in PR
auto-push: ${{ github.event_name != 'pull_request' }}

1 comment on commit a1fec70

@github-actions
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark 'KMP JSON schema validator'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.50.

Benchmark suite Current: a1fec70 Previous: 6aa5b1f Ratio
macosArm64.CommonThroughputBench.validateBasic ( {"objectPath":"openapi.json","schemaPath":"openapi_schema.json"} ) 999.6032017598785 ops/sec 617.474341829956 ops/sec 1.62

This comment was automatically generated by workflow using github-action-benchmark.

CC: @OptimumCode

Please sign in to comment.