Unified Test runs in Docker

[AlvinSchiller]

The test executions are currently running sequentially and build the docker image multiple times. Also the cache is not used, as the code base is included as one of the first layer, so a change in the repo triggers an entire rebuild of the image.
Furthermore are the files mostly duplicates and hard to maintain.

Included Changes:

Dockerfiles

• all Dockerfiles have been merged into a single file (based on debian)
• Debian Version Name can dynamically be passed as buildargs --build-args="DEBIAN_VERSION_NAME=xxx"
• the used platform is defined by the caller --platform=linux/arm/v7
• the command order has been changed to maximize cache hits (least to most frequently changing).
  • installation of repo lists and packages (takes the most build time if not used from cache)
  • Add Default User (pi) and depending on target also an alternative User (hans) (selectable with --user=xxx from caller)
  • Copy files from repository
  • Update packages to deal with long living caches
• Different targets have been added to support different use cases, e.g build for automatic tests (test-code and test-update), build for user tests (code)
• for automatic test builds only the installscript and the testscripts are copied into the image, instead of the whole codebase

Workflows

• all "docker" workflow-files have been merged into two files. A trigger workflow and a reusable subworkflow
• tested debian versions are defined in a matrix in the trigger workflow. Each version entry will call the reusable subworkflow with all needed parameter, e.g. user and scripts to test. These run in parallel.
• the subworkflow performes following steps:
  • checkout and setup steps
  • Build the base docker image for the specific debian version (see Dockerfile target 'test-code'). Use cache to speed up creation.
  • Build an updated docker image based on the base image (see Dockerfile target 'test-update'). Dont use Cache to deal will long living caches of the base image.
  • Run test execution for users and testscripts in a matrix. Each will run in parallel and uses the previously build 'updated docker image'
• the workflow has now a concurreny setting which will cancel previous runs if a new one is triggered (new commits mostly invalided previous results, so why proceed). This is based on the current branch ref (differentiates between branches and pull requests -> https://docs.github.com/en/actions/learn-github-actions/contexts#github-context 'github_ref')
• the cache is also differentiated on branch ref, so the first run on a branch / pull request will always create a new base image.

Additional changes

• "ci "Readme updated
• replaced deprecated apt-key usage
• fix installscript checkout only from 'https://github.com/MiczFlor/RPi-Jukebox-RFID.git#develop'. GIT_BRANCH and GIT_URL are now set based on current Branch.
• fix 'debconf: unable to initialize frontend: Dialog' warning during installation (export DEBIAN_FRONTEND=noninteractive not working with sudo apt-get ... )
• determine current user in installscript with whoami rather then $USER to have higher compatibility with containers
• pythonpackage_future3.yml has been removed (not relevant in 2.x branches)

Information about the caching

• each repository has 10GB of cache space. if the limit is exceeded the oldest cached items will be deleted.
• caches will be deleted after not been used for 7 days (scheduled tasks once per week keeping cache alive?)

[s-martin]

Cool enhancement, thanks!

Anything missing from your side otherwise I would merge it?

I would also manually adapt my Bookworm PR on the new implementation (still struggling with the enforcement of PEP668 on bookworm).

[AlvinSchiller]

Anything missing from your side otherwise I would merge it?

Should be good for now. I have some further ideas but that would be another PR.