Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Shutting down or rebooting dom0 while VMs are paused takes a very long time #8580

Open
UndeadDevel opened this issue Oct 8, 2023 · 2 comments · May be fixed by QubesOS/qubes-core-admin#616
Open

Shutting down or rebooting dom0 while VMs are paused takes a very long time #8580

UndeadDevel opened this issue Oct 8, 2023 · 2 comments · May be fixed by QubesOS/qubes-core-admin#616
Assignees
@alimirjamali
Labels
affects-4.1 This issue affects Qubes OS 4.1. affects-4.2 This issue affects Qubes OS 4.2. C: core diagnosed Technical diagnosis has been performed (see issue comments). P: default Priority: default. Default priority for new issues, to be replaced given sufficient information. pr submitted A pull request has been submitted for this issue. T: bug Type: bug report. A problem or defect resulting in unintended behavior in something that exists.

Comments

@UndeadDevel
Copy link

UndeadDevel commented Oct 8, 2023 •
edited
Loading

Qubes OS release

4.1.2

Brief summary

When shutting down or rebooting QubesOS while some VMs are paused, the process hangs for a long time (first there is a stop job running for the full 3 mins and then dracut hangs because it can't umount old-root), so shutdown/reboot take >3 mins longer than usual.

If I unpause all VMs beforehand this doesn't happen. I have added autostart commands that will unpause relevant VMs automatically on shut down / reboot / logout etc and that fixes the problem, but I feel like this should be handled by the OS. The best solution would probably be to kill paused VMs shortly after shut down or reboot is triggered. The alternative is to unpause and qvm-shutdown them, but that may not be desired security-wise. It could also be that the real issue lies somewhere else and paused VMs during shutdown merely trigger it, I suppose.

Steps to reproduce

Pause some VMs and then shut down.

Expected behavior

System shuts down normally

Actual behavior

Hangs twice and takes >3 mins longer than usual to shut down

Related issues

may be related to #5987
@UndeadDevel UndeadDevel added P: default Priority: default. Default priority for new issues, to be replaced given sufficient information. T: bug Type: bug report. A problem or defect resulting in unintended behavior in something that exists. labels Oct 8, 2023
@andrewdavidwong andrewdavidwong added C: core needs diagnosis Requires technical diagnosis from developer. Replace with "diagnosed" or remove if otherwise closed. affects-4.1 This issue affects Qubes OS 4.1. labels Oct 8, 2023
@andrewdavidwong andrewdavidwong changed the title QubesOS shutdown or reboot take a long time when done while some VMs are paused Shutting down or rebooting dom0 while VMs are paused takes a very long time Oct 8, 2023
@UndeadDevel
Copy link
Author

Still an issue on Q4.2rc5. I solve it by adding triggers in startup menu to unpause my DVMs on restart / shutdown / logout etc.

@andrewdavidwong andrewdavidwong added the affects-4.2 This issue affects Qubes OS 4.2. label Dec 4, 2023
@alimirjamali
Copy link

alimirjamali commented Sep 3, 2024 •
edited
Loading

This is easy. QubesVM.shutdown function could check via is_paused function if the qube is paused and un-pause it prior to shutdown. It will need a small unittest. I will work on it.

p.s.: Some internal logs of shutting down a paused qube before the patch:

Sep 03 12:29:54 dom0 libvirtd[1296]: internal error: Failed to shutdown domain '10' with libxenlight
Sep 03 12:29:54 dom0 qubesd[28700]: unhandled exception while calling src=b'dom0' meth=b'admin.vm.Shutdown' dest=b'disp9564' arg=b'' len(untrusted_payload)=0
Sep 03 12:29:54 dom0 qubesd[28700]: Traceback (most recent call last):
Sep 03 12:29:54 dom0 qubesd[28700]:   File "/usr/lib/python3.11/site-packages/qubes/api/__init__.py", line 291, in respond
Sep 03 12:29:54 dom0 qubesd[28700]:     response = await self.mgmt.execute(
Sep 03 12:29:54 dom0 qubesd[28700]:                ^^^^^^^^^^^^^^^^^^^^^^^^
Sep 03 12:29:54 dom0 qubesd[28700]:   File "/usr/lib/python3.11/site-packages/qubes/api/admin.py", line 884, in vm_shutdown
Sep 03 12:29:54 dom0 qubesd[28700]:     await self.dest.shutdown(force=force, wait=wait)
Sep 03 12:29:54 dom0 qubesd[28700]:   File "/usr/lib/python3.11/site-packages/qubes/vm/qubesvm.py", line 1371, in shutdown
Sep 03 12:29:54 dom0 qubesd[28700]:     self.libvirt_domain.shutdown()
Sep 03 12:29:54 dom0 qubesd[28700]:   File "/usr/lib/python3.11/site-packages/qubes/app.py", line 103, in wrapper
Sep 03 12:29:54 dom0 qubesd[28700]:     return attr(*args, **kwargs)
Sep 03 12:29:54 dom0 qubesd[28700]:            ^^^^^^^^^^^^^^^^^^^^^
Sep 03 12:29:54 dom0 qubesd[28700]:   File "/usr/lib64/python3.11/site-packages/libvirt.py", line 2972, in shutdown
Sep 03 12:29:54 dom0 qubesd[28700]:     raise libvirtError('virDomainShutdown() failed')
Sep 03 12:29:54 dom0 qubesd[28700]: libvirt.libvirtError: internal error: Failed to shutdown domain '10' with libxenlight

alimirjamali added a commit to alimirjamali/qubes-core-admin that referenced this issue Sep 3, 2024
@alimirjamali
Unpause a paused qube before shutdown
a059b61 
fixes: QubesOS/qubes-issues#8580
@alimirjamali alimirjamali linked a pull request Sep 3, 2024 that will close this issue
Open
alimirjamali added a commit to alimirjamali/qubes-core-admin that referenced this issue Sep 3, 2024
@alimirjamali
Unpause a paused qube before shutdown
a2ad22f 
fixes: QubesOS/qubes-issues#8580
alimirjamali added a commit to alimirjamali/qubes-core-admin-client that referenced this issue Sep 3, 2024
@alimirjamali
Document killing of paused qubes on shutdown
34e2e39 
supplements: QubesOS/qubes-issues#8580
main patch: QubesOS/qubes-core-admin#616
@alimirjamali alimirjamali mentioned this issue Sep 3, 2024
Open
alimirjamali added a commit to alimirjamali/qubes-core-admin that referenced this issue Sep 3, 2024
@alimirjamali
Unpause a paused qube before shutdown
8d29d4d 
fixes: QubesOS/qubes-issues#8580
alimirjamali added a commit to alimirjamali/qubes-core-admin-client that referenced this issue Sep 3, 2024
@alimirjamali
Document killing of paused qubes on shutdown
85c1ff9 
supplements: QubesOS/qubes-issues#8580
main patch: QubesOS/qubes-core-admin#616
alimirjamali added a commit to alimirjamali/qubes-core-admin that referenced this issue Sep 3, 2024
@alimirjamali
Kill paused qubes on system shutdown
37c9ee9 
fixes: QubesOS/qubes-issues#8580
alimirjamali added a commit to alimirjamali/qubes-core-admin that referenced this issue Sep 3, 2024
@alimirjamali
Kill paused qubes on system shutdown
2ff9968 
fixes: QubesOS/qubes-issues#8580
alimirjamali added a commit to alimirjamali/qubes-core-admin that referenced this issue Sep 3, 2024
@alimirjamali
Kill paused qubes on system shutdown
b2b290f 
fixes: QubesOS/qubes-issues#8580
@andrewdavidwong andrewdavidwong added diagnosed Technical diagnosis has been performed (see issue comments). pr submitted A pull request has been submitted for this issue. and removed needs diagnosis Requires technical diagnosis from developer. Replace with "diagnosed" or remove if otherwise closed. labels Sep 4, 2024
alimirjamali added a commit to alimirjamali/qubes-core-admin that referenced this issue Sep 5, 2024
@alimirjamali
Kill paused qubes on system shutdown
aa7fd22 
fixes: QubesOS/qubes-issues#8580
alimirjamali added a commit to alimirjamali/qubes-core-admin that referenced this issue Sep 5, 2024
@alimirjamali
Kill paused qubes on system shutdown
059171e 
fixes: QubesOS/qubes-issues#8580
alimirjamali added a commit to alimirjamali/qubes-core-admin that referenced this issue Sep 5, 2024
@alimirjamali
Kill paused qubes on system shutdown
be197f0 
fixes: QubesOS/qubes-issues#8580
alimirjamali added a commit to alimirjamali/qubes-core-admin that referenced this issue Sep 5, 2024
@alimirjamali
Kill paused qubes on system shutdown
8344dad 
fixes: QubesOS/qubes-issues#8580
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@alimirjamali alimirjamali

Labels
affects-4.1 This issue affects Qubes OS 4.1. affects-4.2 This issue affects Qubes OS 4.2. C: core diagnosed Technical diagnosis has been performed (see issue comments). P: default Priority: default. Default priority for new issues, to be replaced given sufficient information. pr submitted A pull request has been submitted for this issue. T: bug Type: bug report. A problem or defect resulting in unintended behavior in something that exists.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Kill a paused qube before system shutdown alimirjamali/qubes-core-admin
3 participants
@alimirjamali @andrewdavidwong @UndeadDevel