Update tests for satellite-maintain update command

[Griffin-Sullivan]

Problem Statement

theforeman/foreman_maintain#802 introduces the update command which will replace z stream upgrades.

Solution

Update robottelo tests that do a z stream upgrade.

Related Issues

[Griffin-Sullivan]

trigger: test-robottelo
pytest: tests/foreman/maintain/test_upgrade.py::test_negative_pre_update_tuning_profile_check tests/foreman/maintain/test_upgrade.py::test_positive_repositories_validate
theforeman:
    foreman_maintain: 802

[Satellite-QE]

PRT Result

Build Number: 6692
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/maintain/test_upgrade.py::test_negative_pre_update_tuning_profile_check tests/foreman/maintain/test_upgrade.py::test_positive_repositories_validate --external-logging
Test Result : ================== 9 warnings, 4 errors in 1765.03s (0:29:25) ==================

[Griffin-Sullivan]

trigger: test-robottelo
pytest: tests/foreman/maintain/test_upgrade.py::test_negative_pre_update_tuning_profile_check tests/foreman/maintain/test_upgrade.py::test_positive_repositories_validate
theforeman:
    foreman_maintain: 802

[Griffin-Sullivan]

trigger: test-robottelo
pytest: tests/foreman/maintain/test_upgrade.py::test_negative_pre_update_tuning_profile_check tests/foreman/maintain/test_upgrade.py::test_positive_repositories_validate
theforeman:
    foreman_maintain: 802

[Satellite-QE]

PRT Result

Build Number: 6694
Build Status: UNSTABLE
PRT Comment: pytest tests/foreman/maintain/test_upgrade.py::test_negative_pre_update_tuning_profile_check tests/foreman/maintain/test_upgrade.py::test_positive_repositories_validate --external-logging
Test Result : ============ 3 failed, 1 passed, 206 warnings in 5329.36s (1:28:49) ============

[Griffin-Sullivan]

Failures here actually make sense.

1. Can't test the preupgrade test because it's making a satellite on the fly within the test not through AAP, therefore it'll never install the packit build for foreman-maintian.

2. The capsule test for repositories validate failed because we don't have a deploy-upstream-capsule yet.

This is good to merge and I will rerun with the one test that is expected to pass.

[Griffin-Sullivan]

trigger: test-robottelo
pytest: tests/foreman/maintain/test_upgrade.py::test_positive_repositories_validate[satellite]
theforeman:
    foreman_maintain: 802

[Griffin-Sullivan]

trigger: test-robottelo
pytest: tests/foreman/maintain/test_upgrade.py::test_positive_repositories_validate[satellite]
theforeman:
    foreman_maintain: 802

[Satellite-QE]

PRT Result

Build Number: 6710
Build Status: SUCCESS
PRT Comment: pytest tests/foreman/maintain/test_upgrade.py::test_positive_repositories_validate[satellite] --external-logging
Test Result : ================= 1 passed, 13 warnings in 2900.65s (0:48:20) ==================

[synkd]

I may be missing some context here, but based on my reading of theforeman/foreman_maintain#802 and https://projects.theforeman.org/issues/37175, it doesn't look to me like this change necessitates changing any existing tests at this point. Eric's PR notes that 'This does not remove using the upgrade command to do a z-stream "upgrade,"' and the Foreman issue is for "add[ing] a command that updates Foreman within a minor release."

To me, that means that foreman-maintain update replaces the use of e.g. foreman-maintain upgrade --target-version 6.15.z when updating all packages on a Foreman/Satellite server but not incrementing the version number. Upgrading from e.g. Satellite 6.15.0 to 6.15.1 still requires running foreman-maintain upgrade --target-version 6.15.z.

So, in my view, we definitely need the CLI command that you've added here, and we may need one or more additional tests for the update functionality, but the existing upgrade tests do not need to be changed.

Could you please explain your changes to the tests in light of this? Alternatively, if I'm off base here, can you help me understand what I'm missing?

[Griffin-Sullivan]

@synkd From my understanding I thought the update command was supposed to replace the upgrade --target-version x.y.z and do z stream upgrades. I understand we won't remove being able to do z-stream upgrades with upgrade right now, but I thought the update command is being brought in to eventually replace that. Let me double check with @ehelms

[ehelms]

The goal of the new update command is to replace the upgrade --target-version method for z-stream updates. I am first adding this new update command so that we can test it and then remove the z-stream through upgrade command after we feel confident in the update command.

[Griffin-Sullivan]

@synkd forgot to address

So, in my view, we definitely need the CLI command that you've added here, and we may need one or more additional tests for the update functionality, but the existing upgrade tests do not need to be changed.

I'm opting to just update these tests to use update since these were using z-stream upgrades to test other features. The main purpose of these tests is not to test that the upgrade works, so I have another task to swap our upgrade testing for z-streams to use update.

[evgeni]

I think tests/upgrades/test_satellite_maintain.py needs updating too, as it seems to be calling satellite-maintain upgrade list-versions --disable-self-upgrade and expect X.Y.z to be in the output, which won't be there anymore.

[Griffin-Sullivan]

@evgeni Since the list-versions command won't be necessary for zstream upgrades anymore, we are going to remove those tests. I will do that in a separate PR. We are good to merge this one.

[Griffin-Sullivan]

Began the work for removing list-versions testing here: #14970