Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark crate for core features #3487

Merged
merged 3 commits into from
Oct 27, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 71 additions & 0 deletions .github/workflows/benchmark-core.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
---
name: Benchmark - core

on:
pull_request:
branches: [master]
paths:
- .github/workflows/benchmark-core.yml
- "packages/yew/**"
- "tools/benchmark-core/**"

jobs:
benchmark-core:
name: Benchmark - core
runs-on: ubuntu-latest

steps:
- name: Checkout master
uses: actions/checkout@v3
with:
repository: "yewstack/yew"
ref: master
path: yew-master

- name: Checkout pull request
uses: actions/checkout@v3
with:
path: current-pr

- name: Setup toolchain
uses: dtolnay/rust-toolchain@master
with:
toolchain: stable

- name: Restore Rust cache for master
uses: Swatinem/rust-cache@v2
with:
working-directory: yew-master
key: master

- name: Restore Rust cache for current pull request
uses: Swatinem/rust-cache@v2
with:
working-directory: current-pr
key: pr

- name: Run pull request benchmark
run: cargo bench -q > ../output.log
working-directory: current-pr/tools/benchmark-core

- name: Run master benchmark
run: cargo bench -q > ../output.log
continue-on-error: true
working-directory: yew-master/tools/benchmark-core
Comment on lines +51 to +54
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can probably cache this somewhere (perhaps in firestore db or storage bucket, since we already use firebase) but this is fine for now

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isnt it already cached by the 2 previous steps (swatinem/rust-cache)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's true. I was thinking of just downloading the expected output file so benchmark is never run when it can be avoided

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh... well. tbh I dont think it's worth the effort haha


- name: Make sure master's output log exists
run: touch yew-master/tools/output.log

- name: Write Pull Request ID
run: |
echo "${{ github.event.number }}" > .PR_NUMBER

- name: Upload Artifact
uses: actions/upload-artifact@v3
with:
name: benchmark-core
path: |
.PR_NUMBER
yew-master/tools/output.log
current-pr/tools/output.log
retention-days: 1
100 changes: 100 additions & 0 deletions .github/workflows/post-benchmark-core.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
---
name: Post Comment for Benchmark - core

on:
workflow_run:
workflows: ["Benchmark - core"]
types:
- completed

jobs:
post-benchmark-core:
if: github.event.workflow_run.event == 'pull_request'
name: Post Comment on Pull Request
runs-on: ubuntu-latest

steps:
- name: Download Repository
uses: actions/checkout@v3

- name: Download Artifact
uses: Legit-Labs/action-download-artifact@v2
with:
github_token: "${{ secrets.GITHUB_TOKEN }}"
workflow: benchmark-core.yml
run_id: ${{ github.event.workflow_run.id }}
name: benchmark-core
path: "benchmark-core/"

- name: Make pull request comment
run: |
cat - >>comment.txt <<EOF
### Benchmark - core
#### Yew Master
```
EOF
cat benchmark-core/yew-master/tools/output.json >>comment.txt
cat - >>comment.txt <<EOF
```
#### Pull Request" >> comment.txt
```
EOF
cat benchmark-core/current-pr/tools/output.json >>comment.txt
cat - >>comment.txt <<EOF
```
EOF

- name: Read Pull Request ID
run: |
PR_NUMBER=$(cat "benchmark-core/.PR_NUMBER")
if ! [[ "$PR_NUMBER" =~ ^[0-9]+$ ]]; then
echo "pr number invalid"
exit 1
fi
echo "PR_NUMBER=$PR_NUMBER" >> $GITHUB_ENV

- name: Post Comment
uses: actions/github-script@v6
with:
script: |
const fs = require('fs');

const commentInfo = {
...context.repo,
issue_number: ${{ env.PR_NUMBER }},
};

const comment = {
...commentInfo,
body: fs.readFileSync("comment.txt", 'utf-8'),
};

function isCommentByBot(comment) {
return comment.user.type === "Bot" && comment.body.includes("### Benchmark - core");
}

let commentId = null;
const comments = (await github.rest.issues.listComments(commentInfo)).data;
for (let i = comments.length; i--; ) {
const c = comments[i];
if (isCommentByBot(c)) {
commentId = c.id;
break;
}
}

if (commentId) {
try {
await github.rest.issues.updateComment({
...context.repo,
comment_id: commentId,
body: comment.body,
});
} catch (e) {
commentId = null;
}
}

if (!commentId) {
await github.rest.issues.createComment(comment);
}
112 changes: 109 additions & 3 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 14 additions & 0 deletions tools/benchmark-core/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
[package]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious, why not put it in yew's benches instead of creating a new crate?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know... 🤷‍♀️ well I guess I can simply move them there I guess. I tried to keep the directory architecture with the other benchmarks

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so what do you prefer? should I move the bench in the crate yew? or keep with a separate crate for consistency sake? both are fine imo i don't really care

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine either way. It doesn't really make that much of a difference and moving it now is just extra work

name = "benchmark-core"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[[bench]]
name = "vnode"
harness = false

[dependencies]
divan = "0.1.0"
yew = { path = "../../packages/yew" }
Loading
Loading