Add Python/Ruby/Dotnet support to GHA runner image

[shahidhs-ibm]

Adds following tools/compilers support in runner images which makes GHA runner to pick them up from cache dir:
Python: 3.8.18, 3.10.13, 3.12.1
Ruby: 3.0.6, 3.1.4, 3.2.3, 3.3.0, jruby-9.4.5.0, mruby-3.2.0 (Ruby versions listed by ruby-build --list from ruby-build-v20240119.tar.gz excluding picoruby and truffleruby)
Dotnet: (this version will always be equal to the dotnet version getting fetched and installed in build-files/convert-rpm.sh)
For Ubuntu 22: 7.0.115, 8.0.101
For Almalinux 9: 7.0.117, 8.0.103

Note: For build-selfhosted.sh, the support has been added for Ubuntu and Almalinux only. Opensuse distro is not supported currently.

[shahidhs-ibm]

@anup-kodlekere @nealef Please take a look at the code changes and let me know if there are any suggestions/comments.

[nealef]

• What's the convert rpm script for?
• The python install appears to have ppc64le hard-coded.
• Have you checked the various distro repost for multiple releases of python then using pyvirt or alternative to switch between them.
• Similar story for ruby
• The ruby installation script appears to assume Ubuntu as it is using apt

[shahidhs-ibm]

@nealef Thanks for reviewing the PR and providing really good suggestions. Please find my comments inline.

• What's the convert rpm script for?

The convert-rpm.sh script converts the dotnet rpm's to deb's using alien. We're doing this as we don't have dotnet binaries hosted for ppc64le and also lacks its support in the deb based distro repositories. We have the rpm support available for few distros, which we're utilizing to avoid building dotnet from source.

• The python install appears to have ppc64le hard-coded.

This a good point. We can make it configurable to support more than one architecture. Will update this in next PR.

• Have you checked the various distro repost for multiple releases of python then using pyvirt or alternative to switch between them.

To make GHA runner pickup Python binary from cache, we have to provide it beforehand at specific location with specific files and directory hierarchy. I don't think we can use pyvirt in this case, have not tried it though but looking at runner behavior, I believe it'll not work.

• Similar story for ruby

Same explanation as above for Python.

• The ruby installation script appears to assume Ubuntu as it is using apt

As of now ruby installation script is getting called in the build-image.sh script which is building only Ubuntu runner images. But yes, this is a good point to make ruby installation independent of distribution (also it has ppc64 arch hard-coded, which we can make configurable as well, as per point#2). I'll work on it in my next update.

[nealef]

But why are you using rpm2cpio etc.? The existing code converted the rpms to debs and then used apt-get to install them. Your script appears to put stuff in /opt rather than just let apt-get put it where the package defines. I'll add comments for the other points later.

[shahidhs-ibm]

But why are you using rpm2cpio etc.? The existing code converted the rpms to debs and then used apt-get to install them. Your script appears to put stuff in /opt rather than just let apt-get put it where the package defines. I'll add comments for the other points later.

As I said in my earlier comment (for Python), To make GHA runner pickup binary from cache, we have to provide it beforehand at specific location with specific files and directory hierarchy. This is applicable for all tools/compilers including dotnet. For dotnet, even though we have it already installed in the runner image through distro repo, still GHA runner will look for it in the cache directory specified in the workflow file. Following is the sample workflow file for dotnet GHA job:

jobs:
  dotnet-action-test:
    runs-on: ubuntu-latest-ppc64le
    env:
      DOTNET_INSTALL_DIR: "/opt/dotnet"
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-dotnet@v3
        with:
          dotnet-version: '7.0.115'
      - run: |
          dotnet --version

So in order to provide dotnet setup beforehand to GHA runner, I am extracting dotnet rpm's using rpm2cpio and then copying to /opt/dotnet which is specified as 'DOTNET_INSTALL_DIR' by the user in their workflow file(s). I am doing this as building dotnet from source will be more hectic and time consuming.

I agree that with this approach we'll have two installations for dotnet but I couldn't think of any better workaround for now.

[nealef]

The question it now raises is why DOTNET_INSTALL_DIR is set to /opt/dotnet instead of /usr/bin?

[shahidhs-ibm]

The question it now raises is why DOTNET_INSTALL_DIR is set to /opt/dotnet instead of /usr/bin?

I did tried that and it didn't worked. From this behavior, I can say that GHA runner does not check only for the dotnet binary but also the whole installation files in specific dir structure and format like below:

$ ls -lstrh /opt/dotnet/
total 200K
   0 drwxr-xr-x. 1 ubuntu ubuntu   20 Mar  4 16:29 templates
   0 drwxr-xr-x. 1 ubuntu ubuntu   17 Mar  4 16:29 host
   0 drwxr-xr-x. 1 ubuntu ubuntu   35 Mar  4 16:29 shared
   0 drwxr-xr-x. 1 ubuntu ubuntu   21 Mar  4 16:29 sdk
   0 drwxr-xr-x. 1 ubuntu ubuntu   23 Mar  4 16:29 metadata
   0 drwxr-xr-x. 1 ubuntu ubuntu   21 Mar  4 16:29 sdk-manifests
100K -rwxr-xr-x. 1 ubuntu ubuntu  97K Mar  4 16:29 dotnet
 96K -rw-r--r--. 1 ubuntu ubuntu  93K Mar  4 16:29 ThirdPartyNotices.txt
4.0K -rw-r--r--. 1 ubuntu ubuntu 1.1K Mar  4 16:29 LICENSE.txt
   0 drwxr-xr-x. 1 ubuntu ubuntu   88 Mar  4 16:29 packs

[nealef]

In which case you want to set it to /usr/lib64/dotnet.

[shahidhs-ibm]

In which case you want to set it to /usr/lib64/dotnet.

Oh yes! Good catch! I'll test with DOTNET_INSTALL_DIR set to /usr/lib64/dotnet. Will update the PR, if this works.

[anup-kodlekere]

@shahidhs-ibm have we tested this with an actual workflow? If so, can you point me to the logs?

[shahidhs-ibm]

@nealef Have incorporated the changes suggested in review comments. Kindly check.

@anup-kodlekere I have tried few random Python/Ruby/dotnet repos:
Python: https://github.com/shahidhs-ibm/awesomecalculator/actions/runs/8306974279/job/22735680458
Ruby: https://github.com/shahidhs-ibm/minitest/actions/runs/8307324932/job/22736407441
Dotnet: https://github.com/shahidhs-ibm/PutridParrot.Randomizer/actions/runs/8307758388/job/22737316812

[anup-kodlekere]

@shahidhs-ibm can you also try a simple matrix job? I'm interested to see what the workflow file for that will look like.

[nealef]

@nealef Have incorporated the changes suggested in review comments. Kindly check.

@anup-kodlekere I have tried few random Python/Ruby/dotnet repos: Python: https://github.com/shahidhs-ibm/awesomecalculator/actions/runs/8306974279/job/22735680458 Ruby: https://github.com/shahidhs-ibm/minitest/actions/runs/8307324932/job/22736407441 Dotnet: https://github.com/shahidhs-ibm/PutridParrot.Randomizer/actions/runs/8307758388/job/22737316812

Note, you may also want to install multiple dotnet SDKs (i.e. for s390x adding 7 and 8; for ppc64le adding 8). They are happy to co-exist and any repo that builds with dotnet will be able to find and use it.

[shahidhs-ibm]

@shahidhs-ibm can you also try a simple matrix job? I'm interested to see what the workflow file for that will look like.

@anup-kodlekere Feel free to try any workflow by yourself as I am not aware of many real-time github repos which can address your use case. Let me know if you have any particular repo to try.

[anup-kodlekere]

This seems like a good use case?
https://github.com/benoitc/gunicorn/blob/master/.github/workflows/tox.yml

[nealef]

Should we also cater for opensuse? We have in the rest of the system.

[shahidhs-ibm]

I had a talk with Anup and he said we are not planning to support opensuse for ppc64le at least for the initial release. I have asked him to remove opensuse support as the present code is giving wrong impression to user.
@anup-kodlekere Please create a PR for removing opensuse from the code.

[nealef]

Make it a single PR so that we can revisit and revert that to re-instate the opensuse code.

[shahidhs-ibm]

@nealef Have incorporated the changes suggested in review comments. Kindly check.
@anup-kodlekere I have tried few random Python/Ruby/dotnet repos: Python: https://github.com/shahidhs-ibm/awesomecalculator/actions/runs/8306974279/job/22735680458 Ruby: https://github.com/shahidhs-ibm/minitest/actions/runs/8307324932/job/22736407441 Dotnet: https://github.com/shahidhs-ibm/PutridParrot.Randomizer/actions/runs/8307758388/job/22737316812

Note, you may also want to install multiple dotnet SDKs (i.e. for s390x adding 7 and 8; for ppc64le adding 8). They are happy to co-exist and any repo that builds with dotnet will be able to find and use it.

Thanks Neale, I wasn't aware of this. I tried it out and it seems working. However, in order to use multiple dotnet SDKs in GHA workflow we'll have to tell user to create temporary global.json specifying which dotnet version they want to use. Does that sound ok? I hope that will not make user confuse. WDYT?

[nealef]

@nealef Have incorporated the changes suggested in review comments. Kindly check.
@anup-kodlekere I have tried few random Python/Ruby/dotnet repos: Python: https://github.com/shahidhs-ibm/awesomecalculator/actions/runs/8306974279/job/22735680458 Ruby: https://github.com/shahidhs-ibm/minitest/actions/runs/8307324932/job/22736407441 Dotnet: https://github.com/shahidhs-ibm/PutridParrot.Randomizer/actions/runs/8307758388/job/22737316812

Note, you may also want to install multiple dotnet SDKs (i.e. for s390x adding 7 and 8; for ppc64le adding 8). They are happy to co-exist and any repo that builds with dotnet will be able to find and use it.

Thanks Neale, I wasn't aware of this. I tried it out and it seems working. However, in order to use multiple dotnet SDKs in GHA workflow we'll have to tell user to create temporary global.json specifying which dotnet version they want to use. Does that sound ok? I hope that will not make user confuse. WDYT?

The globals.json should only be being used when the action runner stuff is built. When the runner is actually used I don't think it's needed.

[shahidhs-ibm]

@nealef Have incorporated the changes suggested in review comments. Kindly check.
@anup-kodlekere I have tried few random Python/Ruby/dotnet repos: Python: https://github.com/shahidhs-ibm/awesomecalculator/actions/runs/8306974279/job/22735680458 Ruby: https://github.com/shahidhs-ibm/minitest/actions/runs/8307324932/job/22736407441 Dotnet: https://github.com/shahidhs-ibm/PutridParrot.Randomizer/actions/runs/8307758388/job/22737316812

Note, you may also want to install multiple dotnet SDKs (i.e. for s390x adding 7 and 8; for ppc64le adding 8). They are happy to co-exist and any repo that builds with dotnet will be able to find and use it.

Thanks Neale, I wasn't aware of this. I tried it out and it seems working. However, in order to use multiple dotnet SDKs in GHA workflow we'll have to tell user to create temporary global.json specifying which dotnet version they want to use. Does that sound ok? I hope that will not make user confuse. WDYT?

The globals.json should only be being used when the action runner stuff is built. When the runner is actually used I don't think it's needed.

@nealef Pushed new commit for multiple dotnet support. Kindly check and test for s390x. Test build is here for reference.

[shahidhs-ibm]

This seems like a good use case? https://github.com/benoitc/gunicorn/blob/master/.github/workflows/tox.yml

@anup-kodlekere Check it out - https://github.com/shahidhs-ibm/gunicorn/actions/runs/8371642249

[shahidhs-ibm]

@nealef Thanks for reviewing the code changes, appreciated! I have incorporated all review comments suggested by you. Please let me know if you have any more review comment/ suggestion. Also let us know if you have tested it on s390x and provide us the feedback.

[anup-kodlekere]

@anup-kodlekere Check it out - https://github.com/shahidhs-ibm/gunicorn/actions/runs/8371642249

Can you include the original x86 jobs as well? The whole idea behind this exercise was to see how the workflow file changes with the addition of the architecture key in the setup-python action.

I'm just spitballing here, but I assume instead of using architecture: ppc64le, we will have to do some regex-magic and extract only the architecture part from the OS label (ubuntu-latest-ppc64le). But that's just a wild guess.

[shahidhs-ibm]

@anup-kodlekere Check it out - https://github.com/shahidhs-ibm/gunicorn/actions/runs/8371642249

Can you include the original x86 jobs as well? The whole idea behind this exercise was to see how the workflow file changes with the addition of the architecture key in the setup-python action.

I'm just spitballing here, but I assume instead of using architecture: ppc64le, we will have to do some regex-magic and extract only the architecture part from the OS label (ubuntu-latest-ppc64le). But that's just a wild guess.

@anup-kodlekere It would be great if you try out by yourself. I'd be happy to help.