Registry URL Malformed

[jardon]

Issue Description

Despite the image already existing locally, podman attempts to download the image again and seemingly replaces the image name with the tag. I attempted to create a container wit the following command:

jardon@lagann:~/Projects/podman$ podman --log-level debug create --hostname lagann --name apx-test --privileged --security-opt label=disable --security-opt apparmor=unconfined --pids-limit=-1 --user root:root --ipc host --network host --pid host --label manager=distrobox --label distrobox.unshare_groups=0 --env SHELL=bash --env HOME=/home/jardon --env container=podman --log-level debug --env TERMINFO_DIRS=/usr/share/terminfo:/run/host/usr/share/terminfo --env CONTAINER_ID=apx-test --volume /tmp:/tmp:rslave --volume /home/jardon/Projects/distrobox/distrobox-init:/usr/bin/entrypoint:ro --volume /home/jardon/Projects/distrobox/distrobox-export:/usr/bin/distrobox-export:ro --volume /home/jardon/Projects/distrobox/distrobox-host-exec:/usr/bin/distrobox-host-exec:ro --volume /home/jardon:/home/jardon:rslave --volume /:/run/host/:rslave --volume /dev:/dev:rslave --volume /sys:/sys:rslave --volume /dev/pts --volume /dev/null:/dev/ptmx --volume /var/log/journal --volume /var/home/jardon:/var/home/jardon:rslave --volume /run/user/1001:/run/user/1001:rslave --volume /etc/hosts:/etc/hosts:ro --volume /etc/resolv.conf:/etc/resolv.conf:ro --volume /etc/hostname:/etc/hostname:ro --annotation run.oci.keep_original_groups=1 --ulimit host --userns keep-id jammy --entrypoint /usr/bin/entrypoint docker.io/library/ubuntu:jammy --verbose --name jardon --user 1001 --group 1001 --home /home/jardon --init 0 --nvidia 0 --pre-init-hooks  --additional-packages  -- 
INFO[0000] podman filtering at log level debug          
DEBU[0000] Called create.PersistentPreRunE(podman --log-level debug create --hostname lagann --name apx-test --privileged --security-opt label=disable --security-opt apparmor=unconfined --pids-limit=-1 --user root:root --ipc host --network host --pid host --label manager=distrobox --label distrobox.unshare_groups=0 --env SHELL=bash --env HOME=/home/jardon --env container=podman --log-level debug --env TERMINFO_DIRS=/usr/share/terminfo:/run/host/usr/share/terminfo --env CONTAINER_ID=apx-test --volume /tmp:/tmp:rslave --volume /home/jardon/Projects/distrobox/distrobox-init:/usr/bin/entrypoint:ro --volume /home/jardon/Projects/distrobox/distrobox-export:/usr/bin/distrobox-export:ro --volume /home/jardon/Projects/distrobox/distrobox-host-exec:/usr/bin/distrobox-host-exec:ro --volume /home/jardon:/home/jardon:rslave --volume /:/run/host/:rslave --volume /dev:/dev:rslave --volume /sys:/sys:rslave --volume /dev/pts --volume /dev/null:/dev/ptmx --volume /var/log/journal --volume /var/home/jardon:/var/home/jardon:rslave --volume /run/user/1001:/run/user/1001:rslave --volume /etc/hosts:/etc/hosts:ro --volume /etc/resolv.conf:/etc/resolv.conf:ro --volume /etc/hostname:/etc/hostname:ro --annotation run.oci.keep_original_groups=1 --ulimit host --userns keep-id jammy --entrypoint /usr/bin/entrypoint docker.io/library/ubuntu:jammy --verbose --name jardon --user 1001 --group 1001 --home /home/jardon --init 0 --nvidia 0 --pre-init-hooks --additional-packages --) 
DEBU[0000] Using conmon: "/usr/bin/conmon"              
INFO[0000] Using sqlite as database backend             
DEBU[0000] Using graph driver overlay                   
DEBU[0000] Using graph root /home/jardon/.local/share/containers/storage 
DEBU[0000] Using run root /run/user/1001/containers     
DEBU[0000] Using static dir /home/jardon/.local/share/containers/storage/libpod 
DEBU[0000] Using tmp dir /run/user/1001/libpod/tmp      
DEBU[0000] Using volume path /home/jardon/.local/share/containers/storage/volumes 
DEBU[0000] Using transient store: false                 
DEBU[0000] [graphdriver] trying provided driver "overlay" 
DEBU[0000] Cached value indicated that overlay is supported 
DEBU[0000] Cached value indicated that overlay is supported 
DEBU[0000] Cached value indicated that metacopy is not being used 
DEBU[0000] Cached value indicated that native-diff is usable 
DEBU[0000] backingFs=btrfs, projectQuotaSupported=false, useNativeDiff=true, usingMetacopy=false 
DEBU[0000] Initializing event backend file              
DEBU[0000] Configured OCI runtime runj initialization failed: no valid executable found for OCI runtime runj: invalid argument 
DEBU[0000] Configured OCI runtime kata initialization failed: no valid executable found for OCI runtime kata: invalid argument 
DEBU[0000] Configured OCI runtime ocijail initialization failed: no valid executable found for OCI runtime ocijail: invalid argument 
DEBU[0000] Configured OCI runtime crun-wasm initialization failed: no valid executable found for OCI runtime crun-wasm: invalid argument 
DEBU[0000] Configured OCI runtime youki initialization failed: no valid executable found for OCI runtime youki: invalid argument 
DEBU[0000] Configured OCI runtime krun initialization failed: no valid executable found for OCI runtime krun: invalid argument 
DEBU[0000] Configured OCI runtime runc initialization failed: no valid executable found for OCI runtime runc: invalid argument 
DEBU[0000] Configured OCI runtime runsc initialization failed: no valid executable found for OCI runtime runsc: invalid argument 
DEBU[0000] Using OCI runtime "/usr/bin/crun"            
INFO[0000] Setting parallel job count to 49             
DEBU[0000] Successfully loaded 1 networks               
DEBU[0000] Pulling image jammy (policy: missing)        
DEBU[0000] Looking up image "jammy" in local containers storage 
DEBU[0000] Normalized platform linux/amd64 to {amd64 linux  [] } 
DEBU[0000] Loading registries configuration "/etc/containers/registries.conf" 
DEBU[0000] Loading registries configuration "/etc/containers/registries.conf.d/shortnames.conf" 
DEBU[0000] Trying "localhost/jammy:latest" ...          
DEBU[0000] reference "[overlay@/home/jardon/.local/share/containers/storage+/run/user/1001/containers]localhost/jammy:latest" does not resolve to an image ID 
DEBU[0000] Trying "docker.io/library/jammy:latest" ...  
DEBU[0000] reference "[overlay@/home/jardon/.local/share/containers/storage+/run/user/1001/containers]docker.io/library/jammy:latest" does not resolve to an image ID 
DEBU[0000] Trying "docker.io/library/jammy:latest" ...  
DEBU[0000] reference "[overlay@/home/jardon/.local/share/containers/storage+/run/user/1001/containers]docker.io/library/jammy:latest" does not resolve to an image ID 
DEBU[0000] Trying "jammy" ...                           
DEBU[0000] Normalized platform linux/amd64 to {amd64 linux  [] } 
DEBU[0000] Attempting to pull candidate docker.io/library/jammy:latest for jammy 
DEBU[0000] parsed reference into "[overlay@/home/jardon/.local/share/containers/storage+/run/user/1001/containers]docker.io/library/jammy:latest" 
DEBU[0000] Resolving "jammy" using unqualified-search registries (/etc/containers/registries.conf) 
Resolving "jammy" using unqualified-search registries (/etc/containers/registries.conf)
Trying to pull docker.io/library/jammy:latest...
DEBU[0000] Copying source image //jammy:latest to destination image [overlay@/home/jardon/.local/share/containers/storage+/run/user/1001/containers]docker.io/library/jammy:latest 
DEBU[0000] Using registries.d directory /etc/containers/registries.d 
DEBU[0000] Trying to access "docker.io/library/jammy:latest" 
DEBU[0000] No credentials matching docker.io/library/jammy found in /run/user/1001/containers/auth.json 
DEBU[0000] No credentials matching docker.io/library/jammy found in /home/jardon/.config/containers/auth.json 
DEBU[0000] No credentials matching docker.io/library/jammy found in /home/jardon/.docker/config.json 
DEBU[0000] No credentials matching docker.io/library/jammy found in /home/jardon/.dockercfg 
DEBU[0000] No credentials for docker.io/library/jammy found 
DEBU[0000]  No signature storage configuration found for docker.io/library/jammy:latest, using built-in default file:///home/jardon/.local/share/containers/sigstore 
DEBU[0000] Looking for TLS certificates and private keys in /etc/docker/certs.d/docker.io 
DEBU[0000] GET https://registry-1.docker.io/v2/         
DEBU[0000] Ping https://registry-1.docker.io/v2/ status 401 
DEBU[0000] GET https://auth.docker.io/token?scope=repository%3Alibrary%2Fjammy%3Apull&service=registry.docker.io 
DEBU[0000] GET https://registry-1.docker.io/v2/library/jammy/manifests/latest 
DEBU[0000] Detected insufficient_scope error, will retry request with updated scope 
DEBU[0000] GET https://auth.docker.io/token?scope=repository%3Alibrary%2Fjammy%3Apull&scope=repository%3Alibrary%2Fjammy%3Apull&service=registry.docker.io 
DEBU[0000] GET https://registry-1.docker.io/v2/library/jammy/manifests/latest 
DEBU[0001] Content-Type from manifest GET is "application/json" 
DEBU[0001] Discarding non-primary errors:               
DEBU[0001]   unauthorized: authentication required      
DEBU[0001] Accessing "docker.io/library/jammy:latest" failed: reading manifest latest in docker.io/library/jammy: requested access to the resource is denied 
DEBU[0001] Error pulling candidate docker.io/library/jammy:latest: initializing source docker://jammy:latest: reading manifest latest in docker.io/library/jammy: requested access to the resource is denied 
Error: initializing source docker://jammy:latest: reading manifest latest in docker.io/library/jammy: requested access to the resource is denied
DEBU[0001] Shutting down engines

Here is the image list:

jardon@lagann:~/Projects/podman$ podman image list
REPOSITORY                         TAG         IMAGE ID      CREATED      SIZE
registry.fedoraproject.org/fedora  latest      184191c01257  5 days ago   467 MB
docker.io/library/ubuntu           jammy       53a843653cbc  8 days ago   80.4 MB
docker.io/library/ubuntu           latest      edbfe74c41f8  2 weeks ago  161 MB
ghcr.io/vanilla-os/vso             main        4cd82a4c8424  3 weeks ago  1.29 GB

The container conf lists the short name:

"ubuntu" = "docker.io/library/ubuntu"

Steps to reproduce the issue

Steps to reproduce the issue

1. podman version 5.3.0-dev or 4.9.5
2. run the above command from the output

Describe the results you received

the verbose output shows that the image is not located in docker.io/library/jammy:latest which is different from the requested docker.io/library/ubuntu:latest

Describe the results you expected

podman should look locally for docker.io/library/ubuntu:latest

podman info output

ardon@lagann:~/Projects/podman$ podman info
host:
  arch: amd64
  buildahVersion: 1.33.7
  cgroupControllers:
  - cpu
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: Unknown
    path: /usr/bin/conmon
    version: 'conmon version 2.1.10, commit: unknown'
  cpuUtilization:
    idlePercent: 95.44
    systemPercent: 1.72
    userPercent: 2.84
  cpus: 16
  databaseBackend: sqlite
  distribution:
    codename: orchid
    distribution: vanilla
    version: "2.0"
  eventLogger: file
  freeLocks: 2033
  hostname: lagann
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1001
      size: 1
    - container_id: 1
      host_id: 165536
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1001
      size: 1
    - container_id: 1
      host_id: 165536
      size: 65536
  kernel: 6.9.8-amd64
  linkmode: dynamic
  logDriver: k8s-file
  memFree: 2399678464
  memTotal: 33373188096
  networkBackend: netavark
  networkBackendInfo:
    backend: netavark
    dns:
      package: Unknown
      path: /usr/lib/podman/aardvark-dns
      version: aardvark-dns 1.6.0
    package: Unknown
    path: /usr/lib/podman/netavark
    version: netavark 1.6.0
  ociRuntime:
    name: crun
    package: Unknown
    path: /usr/bin/crun
    version: |-
      crun version 1.15
      commit: e6eacaf4034e84185fd8780ac9262bbf57082278
      rundir: /run/user/1001/crun
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +WASM:wasmedge +YAJL
  os: linux
  pasta:
    executable: /usr/bin/pasta
    package: Unknown
    version: |
      pasta 0.0~git20240624.1ee2eca-1
      Copyright Red Hat
      GNU General Public License, version 2 or later
        <https://www.gnu.org/licenses/old-licenses/gpl-2.0.html>
      This is free software: you are free to change and redistribute it.
      There is NO WARRANTY, to the extent permitted by law.
  remoteSocket:
    exists: true
    path: /run/user/1001/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: Unknown
    version: |-
      slirp4netns version 1.2.1
      commit: 09e31e92fa3d2a1d3ca261adaeb012c8d75a8194
      libslirp: 4.8.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.5
  swapFree: 16680562688
  swapTotal: 16686592000
  uptime: 52h 13m 36.00s (Approximately 2.17 days)
  variant: ""
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /home/jardon/.config/containers/storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 1
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/jardon/.local/share/containers/storage
  graphRootAllocated: 487390707712
  graphRootUsed: 348147097600
  graphStatus:
    Backing Filesystem: btrfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Supports shifting: "false"
    Supports volatile: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 4
  runRoot: /run/user/1001/containers
  transientStore: false
  volumePath: /home/jardon/.local/share/containers/storage/volumes
version:
  APIVersion: 4.9.5
  Built: 0
  BuiltTime: Wed Dec 31 18:00:00 1969
  GitCommit: ""
  GoVersion: go1.22.5
  Os: linux
  OsArch: linux/amd64
  Version: 4.9.5

Podman in a container

No

Privileged Or Rootless

Privileged

Upstream Latest Release

Yes

Additional environment details

I was attempting to create a jammy container through apx on VanillaOS and hit this issue. apx invokes distrobox which calls podman

Additional information

Related distrobox issue
Related apx issue

[jardon]

This was a result of an incorrect handling of labels by apx as seen here:

... --userns keep-id jammy ...

Removing the space in the distrobox stack's name resolved the issue. Closing :)