mirror-image.sh keeps mirror image in local repository without specifying preserved_images.conf

[yj-amano]

Description

mirror-image.sh is supposed to use the following files:
repo-quay.conf --- application:tag list to be mirrored
preserved-images.conf --- preserved list to be kept in local repository as well as private repository

So if we does not specify the application:tag in preserved-images.conf, the container image is only copied to private repository.

However I tried to do so, the copied image is also left in local repository.

Describe How to Reproduce

OS:RHEL8.2
podman:1.6.4 (in OS)

To check the image existence I used the following ways:
local repository : podman images
private repository : curl -u : -sk https:///v2/_catalog (docker.io/library/registry:2)

Suppose I try to mirror fn_slack:1.0.2

local repository and private repository does not have fn_slack.

$ cat repo_quay.conf <--- 1 entry
fn_slack:1.0.2
$
$ cat preserved_images.conf <--- no entry
$

the following is the verbose output when running script.
As you can see the below, "+ IMAGES_TO_PRESERVE_LOCALLY=repo_quay.conf " is NOT a expected behavior.
Since IMAGES_TO_PRESERVE_LOCALLY is used to refer to the preserving entry, setting repo_quay.conf to IMAGES_TO_PRESERVE_LOCALLY implicitly means all images to be mirrored and preserved regardless of preserved_images.conf .

[root@registry01 mirroring]# /bin/bash mirror-images.sh registry01.example.com podman

• readonly IMAGES_TO_TRANSFER=repo_quay.conf
• IMAGES_TO_TRANSFER=repo_quay.conf
• readonly IMAGES_TO_PRESERVE_LOCALLY=repo_quay.conf
• IMAGES_TO_PRESERVE_LOCALLY=repo_quay.conf
• readonly IMAGE_REGISTRY=quay.io
• IMAGE_REGISTRY=quay.io
• readonly REGISTRY_ORG=ibmresilient
• REGISTRY_ORG=ibmresilient
• readonly SOURCE_REGISTRY=quay.io/ibmresilient
• SOURCE_REGISTRY=quay.io/ibmresilient
• destination_registry=
• [[ -z registry01.example.com ]]
• container_engine=
• [[ ! -z podman ]]
• cmd_exists podman
• command -v podman
• container_engine=podman
• destination_registry=registry01.example.com
• IFS=
• read -r image
• echo 'Now starting to pull image: fn_slack:1.0.2'
  Now starting to pull image: fn_slack:1.0.2
• podman pull quay.io/ibmresilient/fn_slack:1.0.2
  Trying to pull quay.io/ibmresilient/fn_slack:1.0.2...
  Getting image source signatures
  Copying blob 71391dc11a78 skipped: already exists
  Copying blob 77c58f19bd6e skipped: already exists
  Copying blob 47db82df7f3f skipped: already exists
  Copying blob 6cc5487eb078 skipped: already exists
  Copying blob d5e8f7f1544a skipped: already exists
  Copying blob 82659365d36b done
  Copying blob c79715adbbf7 done
  Copying blob 5a5ccde32f71 done
  Copying blob c6ff0a363ec0 done
  Copying blob 1dd50ec86b1e done
  Copying blob 02077f4b6b25 done
  Copying blob 5d51fc0d344e done
  Copying blob 2fa1b041e11f done
  Copying blob 8f702ba8a0ac done
  Copying config cd3992046b done
  Writing manifest to image destination
  Storing signatures
  cd3992046b9baf4f6aa47f5b3c4bfee0336b425212f8d4b1ee31b0ffe9571729
• echo 'Image pulled; Retagging image before pushing'
  Image pulled; Retagging image before pushing
• podman tag quay.io/ibmresilient/fn_slack:1.0.2 registry01.example.com/ibmresilient/fn_slack:1.0.2
• echo 'Image tagged; Pushing now to destination registry: registry01.example.com'
  Image tagged; Pushing now to destination registry: registry01.example.com
• podman push registry01.example.com/ibmresilient/fn_slack:1.0.2
  Getting image source signatures
  Copying blob 323c4edeb0f7 done
  Copying blob 0bfe5b62a1ad skipped: already exists
  Copying blob 70056249a0e2 skipped: already exists
  Copying blob 7fbe9fcfa2da skipped: already exists
  Copying blob 226bfaae015f skipped: already exists
  Copying blob 8c900de07a06 skipped: already exists
  Copying blob 2399639ecc09 done
  Copying blob 5d22555f5f54 done
  Copying blob a57ed7012959 done
  Copying blob 6cb744d823b4 done
  Copying blob fb1e10d6adba done
  Copying blob 9cb7bbae6471 done
  Copying blob e3c4f8786e48 done
  Copying blob 74e270c1131e done
  Copying config cd3992046b done
  Writing manifest to image destination
  Storing signatures
• grep -Fxq fn_slack:1.0.2 repo_quay.conf
• echo 'Transfer completed for image fn_slack:1.0.2. The image fn_slack:1.0.2 was found in the list of images to be preserved and will not be removed locally'
  Transfer completed for image fn_slack:1.0.2. The image fn_slack:1.0.2 was found in the list of images to be preserved and will not be removed locally
• IFS=
• read -r image
• [[ -n '' ]]
  [root@registry01 mirroring]#

[Ryan-Gordon1]

Hi yj,
Thank you for bringing this to our attention.

When I take a look at the script I see this line

resilient-community-apps/.scripts/mirror-containers/mirror-images.sh

Line 26 in e93f391

readonly IMAGES_TO_PRESERVE_LOCALLY="${IMAGES_TO_TRANSFER:-preserved_images.conf}"

readonly IMAGES_TO_PRESERVE_LOCALLY="${IMAGES_TO_TRANSFER:-preserved_images.conf}"
This should be changed so that IMAGES_TO_PRESERVE_LOCALLY references a different environment variable.

A quick fix for you would be to change the line to this
readonly IMAGES_TO_PRESERVE_LOCALLY="${IMAGES_TO_PRESERVE:-preserved_images.conf}"
We will log this and submit a change but also welcome a PR if you are available.

[yj-amano]

Hi Ryan
Thank you for your quick reply.

With your suggestion, I could copy container image only to the private repository. No longer preserved it as expected.