Add upgrade boxes for SecureDrop 1.1.0

[emkll]

Status

Ready for review

Description of Changes

Closes #4887

Add upgrade boxes for SecureDrop 1.1.0

Testing

• make upgrade start
• Source interface is accessible over Tor v3 service and version string is 1.1.0
• make build-debs
• make upgrade-test-local succeeds
• Source and journalist interfaces come up, version string is 1.2.0~rc1

Deployment

Dev env only

If you made non-trivial code changes:

• I have written a test plan and validated it for this PR

[kushaldas]

My Vagrant + libvirt + molecule setup decided not to play together :(

[redshiftzero]

hey @kushaldas - can you explain what problem(s) you encountered so that we can try to resolve them?

[kushaldas]

can you explain what problem(s) you encountered so that we can try to resolve them?

Just regular odd molecule + vagrant + libvirt errors.

For example:

### 2019-10-24 12:20:06 ###
### 2019-10-24 12:20:06 ###
An action 'up' was attempted on the machine 'app-staging',
but another process is already executing an action on the machine.
Vagrant locks each machine for access by only one process at a time.
Please wait until the other Vagrant process finishes modifying this
machine, then try again.

If you believe this message is in error, please check the process
listing for any "ruby" or "vagrant" processes and kill them. Then
try again.

### 2019-10-24 12:20:19 ###
### 2019-10-24 12:20:19 ###
Volume for domain is already created. Please run 'vagrant destroy' first.

Everytime for a different VM in the molecule run, means again trying to clean up first and then trying to re-run the whole scenario (make upgrade-start). molecule destroy -s upgrade or other destroy commands also can not shutdown vms properly and make the system un-usable (same behavior from last year).

[redshiftzero]

can you provide steps to reproduce for getting to either of those error messages? I think it's worth identifying if the challenge here is insufficient documentation, upstream issues, or shortcomings of our development environments that we should resolve.

[rmol]

@kushaldas I've had the same sort of trouble with Vagrant, though I'm not sure if libvirt shares any of the blame. I'm pretty sure I saw this back when I was using Vagrant and Virtualbox. As the error message suggests, I can usually get the Molecule commands to complete after killing those stuck processes. And sometimes cleaning up in virt-manager or virsh.

[kushaldas]

@kushaldas I've had the same sort of trouble with Vagrant, though I'm not sure if libvirt shares any of the blame. I'm pretty sure I saw this back when I was using Vagrant and Virtualbox. As the error message suggests, I can usually get the Molecule commands to complete after killing those stuck processes. And sometimes cleaning up in virt-manager or virsh.

Yes, I generally managed to fix those following the similar commands as you suggested and also using reboot. This time I tried for around 2 hours and just gave up for today. I will try to fix this in morning when my brain is fresh :)

[kushaldas]

can you provide steps to reproduce for getting to either of those error messages? I think it's worth identifying if the challenge here is insufficient documentation, upstream issues, or shortcomings of our development environments that we should resolve.

I don't see any proper reproducer, sometimes it happens with any molecule converge command and sometimes it does not.

[rmol]

OK. I'll try to review this.

[conorsch]

Ran through the test plan here to evaluate libvirt behavior. Encountered no problems running the VM tooling. Further changes appear to be required, however. Results of testing:

• make upgrade start
• Source interface is accessible over Tor v3 service and version string is 1.1.0
• make build-debs
• make upgrade-test-local succeeds
• Source and journalist interfaces come up, version string is 1.2.0~rc1

There was a small tweak required for Ansible 2.7 compatibility, specifically within the upgrade scenario, which I've pushed.

Regarding the libvirt problems reported above, that's most likely related to Molecule's use of an "ephemeral" directory for tracking VM state. During "create/converge" actions, Molecule writes YAML files containing VM information to /tmp/. After a reboot, that state info will be destroyed, but the VM volumes will still exist in libvirt's directories. So a subsequent "create/converge" action will result in Molecule requesting a new VM be created, but libvirt will refuse, since it's already aware of a volume matching the requested name. To resolve, use virt-manager to clean up old VMs, or run virsh and purge the volumes manually. To prevent the issue, make sure to clean up VMs when you're done with them. For example, after using the upgrade scenario: molecule destroy -s upgrade.

[conorsch]

👍