Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: fix artifact upload issue on release build #2389

Merged
merged 5 commits into from
Dec 13, 2024
Merged

Conversation

lklimek
Copy link
Contributor

@lklimek lklimek commented Dec 13, 2024

Issue being fixed or feature implemented

Update to upload-artifact v4 caused the following issue:

Error: Failed to CreateArtifact: Received non-retryable error: Failed request: (409) Conflict: an artifact with this name already exists on the workflow run

What was done?

Fixed the issue by updating the workflow

How Has This Been Tested?

GHA https://github.com/dashpay/platform/actions/runs/12315436050/job/34373645414

Breaking Changes

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have added "!" to the title and described breaking changes in the corresponding section if my code contains any
  • I have made corresponding changes to the documentation if needed

For repository code-owners and collaborators only

  • I have assigned this pull request to a milestone

Summary by CodeRabbit

  • New Features

    • Added a cargo_profile input to enhance workflow configuration.
    • Improved artifact naming conventions to include platform details.
    • Introduced a new step to derive suffixes from tags for Docker labels.
  • Bug Fixes

    • Updated artifact download process for better compatibility with multiple files.
  • Chores

    • Enhanced flexibility in handling Docker image tags and digest uploads.

@lklimek lklimek added this to the v1.7.0 milestone Dec 13, 2024
Copy link
Contributor

coderabbitai bot commented Dec 13, 2024

Warning

Rate limit exceeded

@lklimek has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 4 minutes and 55 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between a024cfc and 755299a.

📒 Files selected for processing (1)
  • .github/workflows/release-docker-image.yml (2 hunks)

Walkthrough

The changes in this pull request focus on updating the workflow configuration for building and publishing Docker images within the .github/workflows/release-docker-image.yml file. Key modifications include the addition of a cargo_profile input, updates to the DIGEST_NAME environment variable, and enhancements to artifact upload and download processes. The workflow now supports more flexible handling of image tags and digest uploads, ensuring compatibility with multiple platforms.

Changes

File Change Summary
.github/workflows/release-docker-image.yml - Added cargo_profile input parameter with default value release.
- Modified DIGEST_NAME to include ${{ inputs.cargo_profile }}.
- Updated artifact upload name to append platform.
- Changed artifact download action to version 4 of actions/download-artifact and set merge-multiple to true.
- Added step to derive a suffix from the tag input.
- Updated Docker tags and labels to include the derived suffix.
- Utilized jq command in the Create manifest list and push step.

Possibly related PRs

Suggested reviewers

  • QuantumExplorer
  • shumkov

🐰 In the garden, we hop and play,
With Docker images, we pave the way.
A profile added, tags refined,
In our workflow, joy we find!
With every build, we leap so high,
A rabbit's cheer, oh my, oh my! 🐇


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Experiment)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (1)
.github/workflows/release-docker-image.yml (1)

Line range hint 23-24: Consider adding documentation for the artifact naming scheme.

The DIGEST_NAME environment variable now includes platform-specific information in the artifact names. Consider adding comments to document this naming scheme and its importance for multi-platform builds.

 env:
+  # Format: digests-{org}-{image}-{tag}-{profile}-{sha}-{platform}
+  # Platform suffix is added during artifact upload to prevent conflicts in multi-platform builds
   DIGEST_NAME: digests-${{ inputs.image_org }}-${{ inputs.image_name }}-${{ inputs.tag }}-${{ inputs.cargo_profile }}-${{ github.sha }}
   DIGEST_DIR_PATH: /tmp/digests
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 40ae73f and cfe059d.

📒 Files selected for processing (1)
  • .github/workflows/release-docker-image.yml (2 hunks)
🔇 Additional comments (2)
.github/workflows/release-docker-image.yml (2)

85-85: Excellent fix for the artifact conflict issue!

Adding the platform to the artifact name is a good solution that prevents the 409 Conflict error by ensuring unique names for artifacts from different platforms.


96-100: Verify the artifact pattern matching scope.

The download configuration looks good and properly complements the upload changes. However, let's verify that the pattern ${{ env.DIGEST_NAME }}* only matches the intended artifacts.

✅ Verification successful

This will help us verify if there are any potential naming conflicts with artifacts in other workflows and understand how DIGEST_NAME is defined to ensure its uniqueness.


Pattern matching is safe and specific to this workflow

The artifact pattern is well-scoped and safe. The DIGEST_NAME environment variable includes multiple unique components (org, image name, tag, cargo profile, and commit SHA) making it highly specific to this workflow's execution. No other workflows in the repository use similar naming patterns for their artifacts, eliminating the risk of unintended matches.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check for potential naming conflicts with other workflow artifacts

# Search for other workflow files that might create artifacts with similar names
echo "Checking for other workflows that create artifacts with similar naming patterns..."
rg -l "actions/(upload|download)-artifact" .github/workflows/

Length of output: 459


Script:

#!/bin/bash
# Check artifact naming patterns in other workflows
echo "Checking artifact names in other workflows..."
rg -A 3 -B 1 "actions/(upload|download)-artifact.*with:" .github/workflows/ | grep -E "name:|pattern:"

# Check DIGEST_NAME definition
echo -e "\nChecking DIGEST_NAME environment variable definition..."
rg "DIGEST_NAME" .github/workflows/release-docker-image.yml

Length of output: 633

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (2)
.github/workflows/release-docker-image.yml (2)

Line range hint 137-144: Add error handling for manifest creation

While the manifest creation logic is correct, consider adding error handling to make the workflow more robust.

Consider this improvement:

  - name: Create manifest list and push
    working-directory: ${{ env.DIGEST_DIR_PATH }}
    run: |
      ls -lah
+     if [ -z "$(ls)" ]; then
+       echo "Error: No digest files found in ${{ env.DIGEST_DIR_PATH }}"
+       exit 1
+     fi
      docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
        $(printf '${{ inputs.image_org }}/${{ inputs.image_name }}@sha256:%s ' *) || {
+       echo "Error: Failed to create manifest list"
+       exit 1
+     }
🧰 Tools
🪛 actionlint (1.7.4)

85-85: parser did not reach end of input after parsing the expression. 5 remaining token(s) in the input: "(", "STRING", ",", "STRING", ")"

(expression)


Line range hint 1-144: Overall implementation looks solid

The changes effectively solve the artifact upload conflict issue by:

  1. Making artifact names unique per platform and profile
  2. Properly handling multi-platform builds using download-artifact@v4's new features
  3. Maintaining robust manifest creation with proper tag handling

The implementation follows GitHub Actions best practices and should prevent the "409 Conflict" errors mentioned in the PR description.

For future maintenance:

  • Consider documenting the artifact naming convention in the workflow comments
  • Monitor GitHub Actions version updates for potential breaking changes in artifact handling
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between cfe059d and a024cfc.

📒 Files selected for processing (1)
  • .github/workflows/release-docker-image.yml (2 hunks)
🧰 Additional context used
🪛 actionlint (1.7.4)
.github/workflows/release-docker-image.yml

85-85: parser did not reach end of input after parsing the expression. 5 remaining token(s) in the input: "(", "STRING", ",", "STRING", ")"

(expression)

🔇 Additional comments (2)
.github/workflows/release-docker-image.yml (2)

85-87: LGTM: Improved artifact naming strategy

The updated artifact naming strategy that includes the platform identifier helps prevent naming conflicts when uploading artifacts from multiple platforms. This directly addresses the 409 Conflict error mentioned in the PR description.

🧰 Tools
🪛 actionlint (1.7.4)

85-85: parser did not reach end of input after parsing the expression. 5 remaining token(s) in the input: "(", "STRING", ",", "STRING", ")"

(expression)


96-100: LGTM: Proper artifact download configuration

The configuration properly handles multi-platform builds by:

  • Using download-artifact@v4
  • Using pattern matching to download all platform-specific digests
  • Merging multiple artifacts into one directory

Let's verify the artifact pattern matches all platform variants:

✅ Verification successful

Let me verify the actual pattern used in the workflow file to ensure it matches what we tested.


Pattern correctly matches all platform-specific digest artifacts

The workflow correctly handles multi-platform artifacts:

  • The DIGEST_NAME env var forms the base name: digests-{org}-{name}-{tag}-{profile}-{sha}
  • The upload uses this pattern with platform suffix: {DIGEST_NAME}-{platform}
  • The download pattern {DIGEST_NAME}* will match all platform variants
  • The merge-multiple: true option ensures all matched artifacts are combined
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the artifact pattern matches all platform variants
# Expected: Pattern should match both amd64 and arm64 platform digests

# Test the pattern against example artifact names
echo "digests-dashpay-drive-v0.1.0-release-abc123-linux-amd64" | grep -E "digests-.*"
echo "digests-dashpay-drive-v0.1.0-release-abc123-linux-arm64" | grep -E "digests-.*"

Length of output: 283


Script:

#!/bin/bash
# Get the actual pattern used in the workflow file
cat .github/workflows/release-docker-image.yml | grep -A3 "download-artifact@v4"

# Also check if there are any env var definitions for DIGEST_NAME
cat .github/workflows/release-docker-image.yml | grep "DIGEST_NAME"

Length of output: 549

Copy link
Member

@shumkov shumkov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants