Allow beetmover to receive optional exclude patterns

[oskirby]

This patch attempts to add the ability for tasks to specify artifacts which should be excluded from beetmover. This has previously been done by updating the RELEASE_EXCLUDE in constants.py but that is a bit of a fragile way of making changes, and it would be handy if the tasks could provide this exclusion information in the payload.

With this change, a task can provide an optional exclude field in the payload, which contains an array of regex patterns to exclude from the beetmover action. For example:

payload:
   exclude:
      - somepath/*.exe

As perhaps a bit of an open question, should this be a literal path instead of a regex pattern? I went with regex simply because that's what is currently done with RELEASE_EXCLUDE

[bhearsum]

Thank you for doing this! It will enable some cleanup for Firefox as well.

I'm assuming that it's tasks like https://firefox-ci-tc.services.mozilla.com/tasks/TI0KzPifSVuVLjPBuydzyw that we want to exclude from uploading the .deb packages - please let me know if that's incorrect.

What's here looks good. https://github.com/mozilla-releng/scriptworker-scripts/blob/master/beetmoverscript/src/beetmoverscript/data/release_beetmover_task_schema.json will need to updated for this as well.

Once that change is done, we can push it to the dev beetmoverscript workers to test it against https://github.com/mozilla-releng/staging-mozilla-vpn-client. (You'll need to suffix the worker-type with -dev to do that, as well.)

[bhearsum]

It isn't a problem now, but I will note that this will subject VPN pushes to the existing RELEASE_EXCLUDE patterns, which could theoretically be a problem in the future. Hopefully we'll have removed this hardcode by the time any such thing comes up.

[oskirby]

From my understanding of the beetmover scripts, that was always the case and I felt that this approach was the best way to minimize the potential for unintended changes in behaviour. Once this PR lands we can probably work towards removing RELEASE_EXCLUDE by migrating the patterns into their associated tasks.

The other way we could have done this would be to use RELEASE_EXCLUDE as the default value for existing tasks that rely on RELEASE_EXCLUDE to omit files, for example:

exclude = context.task["payload"].get("exclude", list(RELEASE_EXCLUDE))

[bhearsum]

You're right - this is already the case. Sorry for the confusion. I think what you have here is fine, but the above suggestion would be fine as well, if you want to go that route.

[bhearsum]

Sorry for the delay here.