Add backport failed label

[mjlbach]

It would be useful to automatically add a label when the backport failed, to make it easier to track which backports need to be handled manually.

[korthout]

That's an interesting idea 💡

Currently the alerting is done via:

• the action creating a comment on the pull request
• the workflow run error notification if something went really wrong or it was unable to create a comment

Generally the comment can be created in most failure cases, unless:

• there are network issues or issues with GitHub APIs
• the GITHUB_TOKEN does not have permissions to create a comment
• or if there is some bug (e.g. somehow the wrong pull request number is used)

I might have missed some failure cases here, but in general any other cases should be covered by the workflow run error.

Anyways, adding a label can be done for all the same situations as creating the comment, as it also depends on the same properties (GH API, TOKEN, etc). I'd prefer to have it configurable where the default configuration is to not add a label, to keep behaviour the same for other users.

@mjlbach Is this what you had in mind? And how do you feel about the other failure cases? Sadly, this cannot be a catch-all solution.

[mjlbach]

I think those are fine :) Usually the failure cases I'm running into are still commented (backports applied in the wrong order) and I just want a nice way in the GH ui to take a look at the ones that failed. I realize it wouldn't be a catch-all since failures in the GH API itself are inevitably going to result in some misses.

[alxgrk]

Hi @korthout,

thanks for creating this action, really helpful.

I find the idea of labelling the PR very useful as well, especially when the cherry-pick fails. Atm, the creation of the comment is the only notification in such cases, and there is no other way to know whether the cherry-pick failed.

Alternatively, if you could add that information to the action's output that would be sufficient as well. One can then react on the result in a subsequent step.

Best,
Alex

[korthout]

I agree @alxgrk. The failure scenarios can be improved, and your alternative sounds like a good idea, enabling more flexibility. Would you be willing to contribute such a solution? If not, I'll pick this up at some point. I'm sure my team would appreciate this as well.

[alxgrk]

@korthout, created a PR. Wasn't sure how to test it, though - do you have some kind of playground for this project?

[korthout]

Wow! Thanks for this, @alxgrk! I built an E2E testing project for this https://github.com/korthout/backport-action-test 😄 (sadly doesn't cover conflicts yet, but I can do that manually)

[korthout]

There appears to be an issue with GH at the moment

branch 'case1-new-changes' set up to track 'origin/case1-new-changes'.
+ gh pr create --head case1-new-changes --base main --title 'Case(1): Add a changed line' --body 'Adds a changed line' --label 'backport case1-backport-target'
https://github.com/korthout/backport-action-test/pull/210
+ gh pr merge --merge --auto --subject 'case(1): merge pull request'
! Pull request #202 (Case(1): Add a changed line) has diverged from local branch

Clearly, the E2E test created a PR (korthout/backport-action-test#210), but it does not exist. So the CLI struggles because the latest run with changes that looked like this was 202 but doesn't match the same history.

I'll continue the test later when GH has resolved its issue.

EDIT: issue is confirmed now

[korthout]

@mjlbach With release Backport-action v1.1.0, there's now a workaround available. You can use the outputs to determine whether the backport-action was successful. If it was unsuccessful, you could use an action like Labeler to label your pull request.

I still keep this issue open because I think it's valuable if the action could just label the pull request out of the box, with batteries included.