Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically detect wrong bisection results due to flapping code #378

Open
AlexDaniel opened this issue Jun 27, 2020 · 1 comment
Open
Labels
bisectable Issues related to Bisectable bot whateverable Issues affecting the bot framework (and therefore all of the bots)

Comments

@AlexDaniel
Copy link
Member

Currently, there's this check:

if $link-msg.ends-with: 07fecb52eb1fd07397659f19a5cf36dc61f84053 {
grumble The result looks a bit unrealistic. Most probably the output
~ is different on every commit (e.g. 「bisect: say rand」)

However, @lizmat++ and @Altai-man++ came up with an idea of doing this check automatically by comparing .moarvm files of the final commit and its parent.

I have a suspicion that rakudo in the past did not produce the same .moarvm file reproducibly, meaning that the check won't work on old commits. However, we can still do some checks, for example we can check if all changed files in the commit were to doc files.

@AlexDaniel AlexDaniel added bisectable Issues related to Bisectable bot whateverable Issues affecting the bot framework (and therefore all of the bots) labels Jun 27, 2020
@AlexDaniel
Copy link
Member Author

AlexDaniel commented Jun 27, 2020

<nine> No, .moarvm files won't be identical since the compiler's id contains the SHA of the full source code

Bummer. So there's no relatively easy way of detecting that. I'll keep this open in case somebody has a better idea on how to achieve this.

Note that the commit that prompted this discussion was rakudo/rakudo@fb6a12a, which is not just a README or doc change, so it's not very easy to figure out that it didn't really change any functionality.

IRC discussion: https://colabti.org/irclogger/irclogger_log/raku-dev?date=2020-06-27#l53

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bisectable Issues related to Bisectable bot whateverable Issues affecting the bot framework (and therefore all of the bots)
Projects
None yet
Development

No branches or pull requests

1 participant