Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Experimenting with passing matrix outputs from previous matrix jobs #4

Closed
wants to merge 16 commits into from

Commits on Jul 1, 2024

  1. Experimenting with passing matrix parameters and outputs from previou…

    …s set of matrix jobs
    
    I need a way to reuse the tags, image names, contexts which are parameters used for building the image.
    Currently these are available in the matrix jobs but I need to access these in the subsequent jobs so that I can rebuild images from cache.
    
    Using custom github action for this:
    Write outputs: https://github.com/cloudposse/github-action-matrix-outputs-write
    Read outputs: https://github.com/cloudposse/github-action-matrix-outputs-read
    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    a3e55a3 View commit details
    Browse the repository at this point in the history
  2. Update cicd.yaml

    Got an error but values are being passed between steps of matrix jobs. Perhaps missing double quotes was the problem, let's see if that resolves it.
    
    ------
    Got this error:
    
    Error: Outputs should be valid YAML
    ---------------------
    host_namespace: ghcr.io/everest/everest-demo
    image: mqtt-server
    context: ./mosquitto
    tags: ghcr.io/everest/everest-demo/mqtt-server:0.0.16
    ghcr.io/everest/everest-demo/mqtt-server:latest
    labels: org.opencontainers.image.created=2024-07-01T17:35:09.789Z
    
    ---------------------
    YAMLParseError: Implicit keys need to be on a single line at line 5, column 1:
    
    tags: ghcr.io/everest/everest-demo/mqtt-server:0.0.16
    ghcr.io/everest/everest-demo/mqtt-server:latest
    ^
    
    ----------
    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    d51b117 View commit details
    Browse the repository at this point in the history
  3. Using block level scalar >- to handle multilines

    So the quotes weren't the issue actually but the fact that the tags passed from the meta step were multiline values.
    Hence using block level scalar >- to let YAML interpret them as multi-line commands.
    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    1a9ab54 View commit details
    Browse the repository at this point in the history
  4. Using | instead of >- to attempt to resolve YAML Parser error

    Still got the same error for multiline values when using >-
    
    YAMLParseError: Implicit keys need to be on a single line at line 6, column 1:
    
      ghcr.io/everest/everest-demo/mqtt-server:0.0.16
    ghcr.io/everest/everest-demo/mqtt-server:latest
    ^
    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    d8806a1 View commit details
    Browse the repository at this point in the history
  5. Accessing Tags and Labels via JSON object output of meta action

    The direct access to tags and labels was giving errors since tags and labels were multiline values whereas the matrix output custom action needed single line values for the YAML parser to correctly parse the parameters.
    
    I found that we can access the tags and labels via the json object which is an output from the meta action.
    So, trying that approach now.
    
    -------
    
    Another possible approach that I came across, but didn't try yet.
    
    I found that with the meta action, we can specify the delimiter / separator type for tags and labels:
    https://github.com/docker/metadata-action#inputs
    
    While tags can have a CSV datatype in docker build push action, labels can only have a List datatype:
    https://github.com/docker/build-push-action?tab=readme-ov-file#inputs
    
    Hence I believe the aforementioned approach would not work.
    
    -------
    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    cc5998f View commit details
    Browse the repository at this point in the history
  6. Got a YAML / JSON related error again

    The custom write and read seems to have passed.
    But error when trying to read data in the env variables part.
    
    Error: The template is not valid. .github/workflows/cicd.yaml (Line: 138, Col: 26): A mapping was not expected
    -----
    
    Now trying to output results directly first, let's see.
    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    8c82d52 View commit details
    Browse the repository at this point in the history
  7. Tags Labels value not passed correctly via matrix outputs custom writ…

    …e / read action
    
    JSON object data is simply passed as "Object" for tags_labels.
    Expected list of tags and labels.
    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    b2895a8 View commit details
    Browse the repository at this point in the history
  8. Printing out tags and labels

    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    a1c4f8d View commit details
    Browse the repository at this point in the history
  9. Print using fromJSON

    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    9b02982 View commit details
    Browse the repository at this point in the history
  10. ENV vars removed fromJSON

    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    fc23388 View commit details
    Browse the repository at this point in the history
  11. ENV vars removed fromJSON

    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    c6eeeaa View commit details
    Browse the repository at this point in the history
  12. Printing different values

    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    450619a View commit details
    Browse the repository at this point in the history
  13. Merge branch 'automate-tests-2' of https://github.com/MukuFlash03/eve…

    …rest-demo into automate-tests-2
    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    099daa0 View commit details
    Browse the repository at this point in the history
  14. Removed from JSON from tags_labels in matrix write action

    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    e0025ae View commit details
    Browse the repository at this point in the history
  15. Added build push action step to build images

    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    fb527a7 View commit details
    Browse the repository at this point in the history
  16. Printing values

    Mahadik, Mukul Chandrakant authored and Mahadik, Mukul Chandrakant committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    94aebc7 View commit details
    Browse the repository at this point in the history