Add support for D-Bus-based yggdrasil IPC

[subpop]

Add parallel support for communicating with yggd over D-Bus. In order to support both gRPC and D-Bus, the gRPC-based yggdrasil package is included in the source tree of this project and a go module replace directive is used to replace the external package lookup with a local one.

When the program starts, it decides to use gRPC only if the YGG_SOCKET_ADDR variable is present in the environment. If that variable is absent, the worker connects to D-Bus using the upstream yggdrasil "worker" package API.

[ekohl]

Looks like this is vendoring in a complete yggdrasil version. Is that intended?

To make testing easier, I started to add Packit in #20.

[subpop]

Yes, this is deliberate. This is how one worker can support both the D-Bus and the gRPC versions of yggdrasil. You'll notice one of the commits pulls the yggdrasil 0.2.x source code into the repository and renames it to "_v0".

[adamruzicka]

Could we add yggdrasil as a git submodule instead of carrying a copy of it?

[subpop]

Could we add yggdrasil as a git submodule instead of carrying a copy of it?

Good idea. I'll refactor to do that.

[subpop]

One obstacle to using a submodule properly is that the vendored yggdrasil needs its module name edited in order for both yggdrasil-0.2.x and yggdrasil-0.4 to work. I did this "by hand" with the code carried directly in the repository in the PR. Switching to a submodule would mean we would need to first rename the upstream module name to include a version suffix. This is probably good to do, but it will require upstream work first.

[subpop]

See RedHatInsights/yggdrasil#250.

[subpop]

I updated the PR to use a submodule. It turns out I was able to do it with a combination of submodules and go module replace directives. The changes I thought were required upstream (RedHatInsights/yggdrasil#250) were not needed for this particular implementation.

[ekohl]

I kicked off CI, but it's not happy. For the future I also relaxed the repo's GitHub Actions settings from Require approval for first-time contributors to Require approval for first-time contributors who are new to GitHub which should allow you to iterate easier. And to be honest, because I'm lazy and don't want to approve every CI run after you push ;)

[subpop]

Tests are passing now.

[ekohl]

I'm looking at @adamruzicka for the real code. I only look at the build system/CI side of things.

@subpop could you rebase on top of main so that we get packit builds as well?

[ekohl]

Nice, having working RPMs should make testing easier.

[adamruzicka]

Huh, interesting. I would expect the rpm build to fail due to additional files. But the build seems to pass and those files are note present in the resulting rpm.

[ekohl]

From a build perspective 👍 but I'm not qualified to review the Go code.

[adamruzicka]

With yggdrasil-0.2.1 this seems to work.

With yggdrasil-0.4.4, with the same configuration:

• yggdrasil takes the content field from the mqtt notification, discards the host path and replaces it with the value of default data host (compile-time configuration?) and tries to fetch it. The url ends up being something like https://$some_rh_service/ssh/jobs/$uuid, which, funnily enough gets greeted by a 200 OK
• the resulting html is passed to the worker which tries to execute it, believing it to be a script, this fails
• the worker reports back to yggdrasil, yggdrasil tries to call back, but again replaces the hostname from the return_url with the default data host and tries to POST to the resulting url, failing again

Setting data-host = "" in the config file seems to resolve this particular issue, but are we expected to do that?

[adamruzicka]

None of the things I complained in the previous comment can really be addressed in this PR, so I take that back.

The change proposed here look sane and I was able to get the entire flow to work end to end with yggdrasil-0.2.1 and 0.4.4, let's get this in.

[adamruzicka]

Thank you @subpop & @ekohl !