[PATCH QEMU HCI-8.1 v2 16/24] kvm: fix coalesced MMIO flush racing with kvm_close()

[Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>]

kvm_close() destroys the vCPUs, which unmaps their kvm_run areas, and the
coalesced MMIO ring lives in the kvm_run area of the first vCPU.  Nothing
stops the RCU thread from flushing that ring at the same time: finalizing
a memory region of a freed flatview calls memory_region_transaction_begin()
and thus kvm_flush_coalesced_mmio_buffer().  kvm_state is still set at
that point, so the flush reads the ring after it has been unmapped:

  #0  kvm_flush_coalesced_mmio_buffer   accel/kvm/kvm-all.c
  #1  qemu_flush_coalesced_mmio_buffer
  #2  memory_region_transaction_begin
  #3  memory_region_finalize
  ...
  #10 flatview_destroy
  #11 call_rcu_thread

while the main thread is in:

  kvm_close -> cpu_remove_sync
  vfio_kvm_device_close
  vfio_cpr_kvm_close_notifier (MIG_EVENT_PRECOPY_DONE)

Seen as a SIGSEGV of the old QEMU during repeated cpr-exec of a VM with
a VFIO device.

Clear the ring pointer first, so that new flushes skip it, and wait for
the pending RCU callbacks before destroying the vCPUs.

Fixes: 7ed0919119b0 ("migration: close kvm after cpr")
Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>
---
 accel/kvm/kvm-all.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index ea027c37829..c29d467f9f5 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -630,6 +630,14 @@ void kvm_close(void)
         return;
     }
 
+    /*
+     * Destroying vCPUs unmaps the coalesced MMIO ring, while the RCU thread
+     * may be flushing it from a memory region finalizer.  Stop new flushes
+     * and wait for the pending RCU callbacks first.
+     */
+    kvm_state->coalesced_mmio_ring = NULL;
+    drain_call_rcu();
+
     CPU_FOREACH(cpu) {
         cpu_remove_sync(cpu);
         close(cpu->kvm_fd);

-- 
2.47.1