Error upgrading nfs-utils

[bionade24]

This gets retried for > 14 days now:

Sep 05 11:07:17 server1 zincati[6122]:     error: Checkout nfs-utils-1:2.6.3-1.rc3.fc38.x86_64: Hardlinking a9/3e5724cba2b79f0f3672459a6f9000b525ee87b96c2c4a036b214a6b0b6fba.fi
le to mount.nfs: File exists

Is this a packaging bug? Why the endless retries? Is this related to #572 ?

Environment

What hardware/cloud provider/hypervisor is being used?
QUEMU

zincati config:

[updates]
strategy = "periodic"

[[updates.periodic.window]]
days = [ "Sat", "Sun" ]
start_time = "02:30"
length_minutes = 120

[travier]

I've moved this issue to the Fedora CoreOS tracker as this is not specific to Zincati.

[travier]

Can you give us the output of rpm-ostree status -b? Thanks

[bionade24]

I can only give you rpm-ostree status I did before reporting since I already removed nfs-utils after confirming that nfs-utils-coreos is installed.

[user@server] ± rpm-ostree status
State: idle
AutomaticUpdatesDriver: Zincati
  DriverState: active; trying to stage 38.20230806.3.0 (failed attempts: 5)
Deployments:
● fedora:fedora/x86_64/coreos/stable
                  Version: 38.20230722.3.0 (2023-08-07T18:56:37Z)
               BaseCommit: bf28f852e934b0c0b9eee232a58970e96adb3e691299b02376f8719530e03fb3
             GPGSignature: Valid signature by 6A51BBABBA3D5467B6171221809A8D7CEB10B464
          LayeredPackages: bat docker-compose fd-find make man moby-engine-zsh-completion neovim nfs-utils python-pip python3-neovim restic speedtest-cli tmux zsh
                           zsh-syntax-highlighting
            LocalPackages: chezmoi-2.11.2-1.x86_64

  fedora:fedora/x86_64/coreos/stable
                  Version: 38.20230709.3.0 (2023-07-24T12:25:01Z)
               BaseCommit: 552de26fe0fe6a5e491f7a4163db125e3d44b144ae53a8f5f488e3f8481c46f9
             GPGSignature: Valid signature by 6A51BBABBA3D5467B6171221809A8D7CEB10B464
          LayeredPackages: bat docker-compose fd-find make man moby-engine-zsh-completion neovim nfs-utils python-pip python3-neovim restic speedtest-cli tmux zsh
                           zsh-syntax-highlighting
            LocalPackages: chezmoi-2.11.2-1.x86_64

Fwiw, I that's the log from removing nfs-utils and zincait afterwards:

[user@server] 6s ± sudo rpm-ostree remove nfs-utils
Checking out tree bf28f85... done
Resolving dependencies... done
Relabeling... done
Checking out packages... done
Running pre scripts... done
Running post scripts... done
Running posttrans scripts... done
Writing rpmdb... done

Writing OSTree commit... done
Staging deployment... done
Freed: 388.4 MB (pkgcache branches: 6)
Removed:
  gssproxy-0.9.1-5.fc38.x86_64
  libev-4.33-7.fc38.x86_64
  libverto-libev-0.3.2-5.fc38.x86_64
  nfs-utils-1:2.6.3-0.fc38.x86_64
  quota-1:4.09-2.fc38.x86_64
  quota-nls-1:4.09-2.fc38.noarch
Changes queued for next boot. Run "systemctl reboot" to start a reboot

-- Skipped

Sep 05 11:34:33 server1 zincati[6122]: [INFO  zincati::update_agent::actor] target release '38.20230806.3.0' selected, proceeding to stage it
Sep 05 11:34:38 server1 zincati[6122]: [ERROR zincati::update_agent::actor] failed to stage deployment: rpm-ostree deploy failed:
Sep 05 11:34:38 server1 zincati[6122]:     error: Checkout nfs-utils-1:2.6.3-1.rc3.fc38.x86_64: Hardlinking a9/3e5724cba2b79f0f3672459a6f9000b525ee87b96c2c4a036b214a6b0b6fba.f>
Sep 05 11:34:38 server1 zincati[6122]:
Sep 05 11:40:05 server1 zincati[6122]: [INFO  zincati::update_agent::actor] target release '38.20230806.3.0' selected, proceeding to stage it
Sep 05 11:40:33 server1 zincati[6122]: [INFO  zincati::update_agent::actor] update staged: 38.20230806.3.0

[jlebon]

It's a packaging bug. The node should be able to upgrade to the 38.20230819.3.0 release currently rolling out containing an updated nfs-utils-coreos (or you can manually force an rpm-ostree upgrade --bypass-driver; this is normally not recommended but there are no barriers that require you to go to 38.20230806.3.0 first). See https://src.fedoraproject.org/rpms/nfs-utils/pull-request/9#comment-59201 for the gory details.

Let's keep #572 to track the larger packaging issue.