-
Notifications
You must be signed in to change notification settings - Fork 31
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
smarter action shuffler #39
base: master
Are you sure you want to change the base?
Conversation
1a0566f
to
0b4e0f6
Compare
@@ -43,6 +43,9 @@ dependencies { | |||
implementation("org.apache.commons:commons-math3:3.6.1") | |||
implementation("com.atlassian.performance.tools:concurrency:[1.0.0,2.0.0)") | |||
implementation("com.atlassian.performance:selenium-js:[1.0.0,2.0.0)") | |||
|
|||
runtimeOnly("org.jetbrains.kotlin:kotlin-reflect:${kotlinVersion}") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Uh oh, reflection = danger zone. We're bypassing a lot of compilation-time guarantees.
} | ||
|
||
companion object { | ||
fun createRandomisedScenario(seededRandom: SeededRandom, actionProportions: Map<Action, Int>, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's the problem we're trying to solve here?
Do skipped actions malform the ActionMetric.label
proportions too much? Our initial workaround was to run the test for a longer time so they get averaged out.
If that is not effective enough, then we can avoid action skips by using a self-loading Memory
, e.g. pass this
to an action during Memory.recall
:
fun recall(): String {
if (issueKeys.isEmpty()) {
SearchJqlAction(jira, throwawayActionMeter, IssueJqlMemory(), this).run()
}
}
This way you can avoid skips with local logic rather than a global order-controlling scheme.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. I want to preserve the requested proportions. During our runs, ~1500 view issue calls are skipped. This looks like a minimally invasive way of achieving this, without altering the way scenarios are created.
The problem with hiding the query inside a different action is that you lose control over proportions - if there are permission issues, the JQL action can be executed as often as view issue, which we don't want. If we stick to sorting, everything is transparent and there are no surprises hiding in the implicit initialisation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The same problem affects ViewBoard, btw.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's try a different question: is there any benefit to keeping a fully-randomised shuffle in the scenario?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🤔
I'll come back to this next week
0b4e0f6
to
c9aa5a8
Compare
4d10d91
to
d9a2e73
Compare
d9a2e73
to
7a98551
Compare
No description provided.