[PATCH hci-dev v3 2/2]vfio/migration: don't touch the device on guest shutdown #VSTOR-153976

[Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>]

When the guest shuts down and QEMU runs with -no-shutdown, the VM stays
stopped and can only be reset or quit. A libvirt soft reboot is exactly
that: guest shutdown, system_reset, cont. Today vfio_vmstate_change()
treats this stop like any other and moves the device to STOP, then cont
brings it back to RUNNING.

NVIDIA's vGPU manager (595.71.03) takes that STOP for the start of a
migration, finds no guest driver, breaks its dirty page tracking and
never repairs it when the driver loads again. Every live migration of
the VM after a soft reboot then fails: "Error saving page in pipelined
mode".

Do not touch the device state in this case. There is nothing to
quiesce, and the reset that must follow resets the device anyway. The
case is told apart by the shutdown action: only "pause" (-no-shutdown)
keeps the VM alive. A quit of a running VM, a signal and a plain guest
shutdown use "poweroff" and still stop the device as before. A quit or
a migration of the stopped VM stops the device first, see the previous
patch.

With this patch applied, soft reboot + subsequent live migration pass
successfully.

Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>
---
 hw/vfio/migration.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c
index bfafc14e5b9..454c4e3f553 100644
--- a/hw/vfio/migration.c
+++ b/hw/vfio/migration.c
@@ -415,6 +415,21 @@ static bool vfio_precopy_supported(VFIODevice *vbasedev)
     return migration->mig_flags & VFIO_MIGRATION_PRE_COPY;
 }
 
+/*
+ * The guest has shut down and QEMU keeps the VM stopped (-no-shutdown). The
+ * guest has released the device, and only a reset, a quit or a migration can
+ * follow. The device is left running here: the reset resets it, and the two
+ * other paths stop it with vfio_migration_stop_after_shutdown().
+ *
+ * The same run state with the "poweroff" action is a quit of a running VM,
+ * which stops the device as usual.
+ */
+static bool vfio_vm_stopped_by_guest_shutdown(bool running, RunState state)
+{
+    return !running && state == RUN_STATE_SHUTDOWN &&
+           qemu_shutdown_action_is_pause();
+}
+
 /*
  * Stop a device still running while the VM is stopped after a guest
  * shutdown. Nothing else leaves a running device in the SHUTDOWN run state,
@@ -924,6 +939,10 @@ static void vfio_vmstate_change_prepare(void *opaque, bool running,
         return;
     }
 
+    if (vfio_vm_stopped_by_guest_shutdown(running, state)) {
+        return;
+    }
+
     new_state = migration->device_state == VFIO_DEVICE_STATE_PRE_COPY ?
                     VFIO_DEVICE_STATE_PRE_COPY_P2P :
                     VFIO_DEVICE_STATE_RUNNING_P2P;
@@ -961,6 +980,10 @@ static void vfio_vmstate_change(void *opaque, bool running, RunState state)
         return;
     }
 
+    if (vfio_vm_stopped_by_guest_shutdown(running, state)) {
+        return;
+    }
+
     if (running) {
         new_state = VFIO_DEVICE_STATE_RUNNING;
     } else {
-- 
2.47.1