Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retrying logic when pulling an image and platform doesn't match #1969

Merged
merged 3 commits into from
Nov 14, 2023

Conversation

jjbustamante
Copy link
Member

@jjbustamante jjbustamante commented Nov 9, 2023

Summary

The Paketo team is trying to create arm64 builders and Buildpacks, but they are doing iteratively, because of that, they are in a situation where they need some mixed support for arm64 and amd64 users. With the latest pack release 0.32.0 we tried to fix an issue where users running M1 laptops and trying to build a amd64 application image ended up with an application image having amd64 code on the top of a arm64 run image. see this PR

The problem was, we added a platform configuration (os/arch) when pulling Buildpacks and unfortunately this is not valid because most of the Buildpack ecosystem doesn't specify architecture for their buildpacks.

As a consequence, running pack build or pack builder create or any command that is pulling Builpacks throws an error, I added a retry logic to pull again, without platform (previous behavior), when an error is found with the message does not match the specified platform

Output

Before

ERROR: failed to add buildpacks to builder: downloading buildpack: extracting from registry docker://gcr.io/paketo-buildpacks/dotnet-core: fetching image: image with reference gcr.io/paketo-buildpacks/dotnet-core:latest was found but does not match the specified platform: wanted linux/amd64, actual: linux

After

pack will silently retry

latest: Pulling from paketo-buildpacks/dotnet-core
Digest: sha256:bf28b7631c222794f01e637c2de0d097f39079c0aa6c8fbdaefeef239d404219
Status: Image is up to date for gcr.io/paketo-buildpacks/dotnet-core:latest
latest: Pulling from paketo-buildpacks/dotnet-core
Digest: sha256:bf28b7631c222794f01e637c2de0d097f39079c0aa6c8fbdaefeef239d404219

Screenshot 2023-11-14 at 10 52 52 AM

Documentation

  • Should this change be documented?
    • Yes, see #___
    • No

Related

Resolves #1968

@jjbustamante jjbustamante added the type/bug Issue that reports an unexpected behaviour. label Nov 9, 2023
@jjbustamante jjbustamante added this to the 0.32.1 milestone Nov 9, 2023
@github-actions github-actions bot modified the milestones: 0.32.1, 0.33.0 Nov 9, 2023
@github-actions github-actions bot added the type/enhancement Issue that requests a new feature or improvement. label Nov 9, 2023
@jjbustamante jjbustamante modified the milestones: 0.33.0, 0.32.1 Nov 9, 2023
@jjbustamante jjbustamante removed the type/enhancement Issue that requests a new feature or improvement. label Nov 9, 2023
@jjbustamante
Copy link
Member Author

cc @natalieparellano

@github-actions github-actions bot added the type/enhancement Issue that requests a new feature or improvement. label Nov 9, 2023
@github-actions github-actions bot modified the milestones: 0.32.1, 0.33.0 Nov 9, 2023
@jjbustamante jjbustamante modified the milestones: 0.33.0, 0.32.1 Nov 9, 2023
@jjbustamante jjbustamante force-pushed the bugfix/jjbustamante/issue-1968 branch from d00f1a9 to 66da647 Compare November 9, 2023 21:31
@github-actions github-actions bot modified the milestones: 0.32.1, 0.33.0 Nov 9, 2023
@jjbustamante jjbustamante modified the milestones: 0.33.0, 0.32.1 Nov 9, 2023
@jjbustamante jjbustamante marked this pull request as ready for review November 9, 2023 21:46
@jjbustamante jjbustamante requested review from a team as code owners November 9, 2023 21:46
@jjbustamante jjbustamante removed the type/enhancement Issue that requests a new feature or improvement. label Nov 9, 2023
@github-actions github-actions bot modified the milestones: 0.32.1, 0.33.0 Nov 10, 2023
@github-actions github-actions bot added the type/enhancement Issue that requests a new feature or improvement. label Nov 10, 2023
@jjbustamante jjbustamante removed the type/enhancement Issue that requests a new feature or improvement. label Nov 10, 2023
@jjbustamante jjbustamante removed this from the 0.33.0 milestone Nov 10, 2023
@github-actions github-actions bot added this to the 0.33.0 milestone Nov 10, 2023
@jjbustamante jjbustamante removed the type/enhancement Issue that requests a new feature or improvement. label Nov 10, 2023
@jjbustamante jjbustamante modified the milestones: 0.33.0, 0.32.1 Nov 10, 2023
…atch, in this case we will retry without any platform defined (previous behavior)

Signed-off-by: Juan Bustamante <[email protected]>
@jjbustamante jjbustamante force-pushed the bugfix/jjbustamante/issue-1968 branch from 119750f to d022a54 Compare November 10, 2023 22:58
@github-actions github-actions bot added the type/enhancement Issue that requests a new feature or improvement. label Nov 10, 2023
@github-actions github-actions bot modified the milestones: 0.32.1, 0.33.0 Nov 10, 2023
@jjbustamante jjbustamante removed the type/enhancement Issue that requests a new feature or improvement. label Nov 10, 2023
@jjbustamante jjbustamante modified the milestones: 0.33.0, 0.32.1 Nov 10, 2023
Copy link
Contributor

@c0d1ngm0nk3y c0d1ngm0nk3y left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think the better approach would be to have 3 attempts:

  • os/arch
  • os (currently this one is skipped and "linux" is never requested, right?)
  • anything (is this even needed?)

@@ -192,6 +197,11 @@ func (f *Fetcher) pullImage(ctx context.Context, imageID string, platform string

err = jsonmessage.DisplayJSONMessagesStream(rc, &colorizedWriter{writer}, termFd, isTerm, nil)
if err != nil {
// sample error from docker engine:
// image with reference <image> was found but does not match the specified platform: wanted linux/amd64, actual: linux
if strings.Contains(err.Error(), "does not match the specified platform") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comparing an error string feels a bit fragile.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree... but, the type of error returned is *jsonmessage.JSONError 😕

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The alternative I guess would be to always retry

Signed-off-by: Natalie Arellano <[email protected]>
@github-actions github-actions bot added type/enhancement Issue that requests a new feature or improvement. type/chore Issue that requests non-user facing changes. labels Nov 13, 2023
@github-actions github-actions bot modified the milestones: 0.32.1, 0.33.0 Nov 13, 2023
@natalieparellano
Copy link
Member

I added an acceptance test in 5451248 and cleaned up the error wrapping so that we're passing less strings around.

I played around with retrying with platformlinux but I still see the wanted linux/amd64 error and I'm not sure why - perhaps Docker is filling in the arch when platform is non-nil? For now, platform empty seems like a good fallback option since it always worked before.

WDYT @jjbustamante @c0d1ngm0nk3y ?

@c0d1ngm0nk3y
Copy link
Contributor

I added an acceptance test...

Great. Because for use, this is issue is quite nasty, so this should help prevent such things in future.

@jjbustamante jjbustamante modified the milestones: 0.33.0, 0.32.1 Nov 14, 2023
@jjbustamante jjbustamante removed type/enhancement Issue that requests a new feature or improvement. type/chore Issue that requests non-user facing changes. labels Nov 14, 2023
@jjbustamante
Copy link
Member Author

@jkutner , @hone Can I get some approvals here? I can't approve my own pull requests 😄

@jjbustamante jjbustamante merged commit b14250b into main Nov 14, 2023
18 checks passed
@jjbustamante jjbustamante deleted the bugfix/jjbustamante/issue-1968 branch November 14, 2023 19:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/bug Issue that reports an unexpected behaviour.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

platform mismatch during pack build / pack builder create
4 participants