Virtuozzo QEMU development (svt-core@virtuozzo.com)
 help / color / mirror / Atom feed
* [RFC HCI-8.1 PATCH 0/5] qemu: allow qemu-update with VFIO hostdevs on shared memfd RAM #VSTOR-103860
@ 2026-10-09 14:04 Andrey Drobyshev
  2026-10-09 14:04 ` [RFC HCI-8.1 PATCH 1/5] qemu update: allow qemu-update for VMs with PCI and USB hostdevs #VSTOR-103860 #VSTOR-137800 Andrey Drobyshev
                   ` (4 more replies)
  0 siblings, 5 replies; 6+ messages in thread
From: Andrey Drobyshev @ 2026-10-09 14:04 UTC (permalink / raw)
  To: svt-core; +Cc: andrey.drobyshev, efim.shevrin, den

qemu-update (cpr-exec) keeps anonymous guest RAM across exec with
MADV_DOEXEC, which shares the pages with the new process copy-on-write.
For a VM with a VFIO device that breaks the device: VFIO pins all guest
RAM for DMA, so the first guest write after exec copies the page, while
the device keeps doing DMA to the old pinned one.  Shared memfd RAM is
handed over to the new QEMU as is and has no such problem.

This series lifts the blanket hostdev restriction for qemu-update and
makes it safe:

 - only PCI (VFIO) and USB hostdevs are allowed, mdev/scsi/scsi_host as
   well as NVMe disks and vDPA interfaces are still refused;
 - new VMs with devices pinning guest RAM (PCI/mdev hostdevs, NVMe disks,
   vDPA) get shared memfd RAM, unless memory backing is set explicitly;
 - their auxiliary RAM (VRAM, ROMs) is shared too via aux-ram-share=on;
 - qemu-update is refused for a PCI hostdev on non-shared RAM, e.g. a VM
   started by an older libvirt; a cold restart switches it to memfd.

KSM is not lost: it never merges pinned pages anyway.  THP works for memfd
RAM with the QEMU fix aligning fd-backed RAM to 2M and
/sys/kernel/mm/transparent_hugepage/shmem_enabled set to "advise".

Tested on whole GPU passthrough (RTX 6000 Ada): 32 qemu-updates in a
row, the GPU keeps working and all 16G of guest RAM stays on 2M pages.
With the QEMU VFIO CPR fixes also on an NVIDIA vGPU: qemu-update keeps
the vGPU working, a GPU<->guest RAM DMA checker shows no mismatches.

This is an alternative to the vzkernel series moving MADV_DOEXEC memory
into the new mm, which needs no libvirt changes.

Andrey Drobyshev (5):
  qemu update: allow qemu-update for VMs with PCI and USB hostdevs
    #VSTOR-103860 #VSTOR-137800
  qemu: capabilities: probe aux-ram-share machine property #VSTOR-103860
  qemu: back guest RAM with shared memfd for VMs with DMA-pinning
    devices #VSTOR-103860
  qemu: share auxiliary RAM of VMs with DMA-pinning devices
    #VSTOR-103860
  qemu update: refuse PCI hostdevs on non-shared guest RAM #VSTOR-103860

 src/qemu/qemu_capabilities.c                  |  2 +
 src/qemu/qemu_capabilities.h                  |  1 +
 src/qemu/qemu_command.c                       |  3 +
 src/qemu/qemu_domain.c                        | 36 ++++++++
 src/qemu/qemu_domain.h                        |  6 ++
 src/qemu/qemu_driver.c                        | 44 ++++++++-
 src/qemu/qemu_process.c                       | 91 +++++++++++++++++++
 .../caps_10.0.0_aarch64.xml                   |  1 +
 .../caps_10.0.0_ppc64.xml                     |  1 +
 .../caps_10.0.0_s390x.xml                     |  1 +
 .../caps_10.0.0_x86_64+amdsev.xml             |  1 +
 .../caps_10.0.0_x86_64.xml                    |  1 +
 .../caps_10.1.0_x86_64+inteltdx.xml           |  1 +
 tests/qemustatusxml2xmldata/modern-in.xml     |  1 +
 .../disk-nvme.x86_64-latest.args              |  4 +-
 .../disk-slices.x86_64-latest.args            |  4 +-
 ...tdev-mdev-display-ramfb.x86_64-latest.args |  4 +-
 ...play-spice-egl-headless.x86_64-latest.args |  4 +-
 ...ev-display-spice-opengl.x86_64-latest.args |  4 +-
 ...isplay-vnc-egl-headless.x86_64-latest.args |  4 +-
 ...ostdev-mdev-display-vnc.x86_64-latest.args |  4 +-
 .../hostdev-mdev-display.x86_64-latest.args   |  4 +-
 ...hostdev-mdev-precreated.x86_64-latest.args |  4 +-
 ...tdev-pci-address-device.x86_64-latest.args |  4 +-
 ...-pci-address-unassigned.x86_64-latest.args |  4 +-
 .../hostdev-pci-address.x86_64-latest.args    |  4 +-
 ...stdev-pci-display-ramfb.x86_64-latest.args |  4 +-
 ...stdev-pci-multifunction.x86_64-latest.args |  4 +-
 ...tdev-subsys-mdev-vfio-ap.s390x-latest.args |  4 +-
 ...ubsys-mdev-vfio-ccw-boot.s390x-latest.args |  4 +-
 ...dev-subsys-mdev-vfio-ccw.s390x-latest.args |  4 +-
 ...ostdev-vfio-multidomain.x86_64-latest.args |  4 +-
 ...o-zpci-autogenerate-fids.s390x-latest.args |  4 +-
 ...o-zpci-autogenerate-uids.s390x-latest.args |  4 +-
 ...v-vfio-zpci-autogenerate.s390x-latest.args |  4 +-
 ...dev-vfio-zpci-boundaries.s390x-latest.args |  4 +-
 ...vfio-zpci-ccw-memballoon.s390x-latest.args |  4 +-
 ...io-zpci-multidomain-many.s390x-latest.args |  4 +-
 .../hostdev-vfio-zpci.s390x-latest.args       |  4 +-
 .../hostdev-vfio.x86_64-latest.args           |  4 +-
 .../machine-loadparm-hostdev.s390x-9.1.0.args |  2 +-
 ...machine-loadparm-hostdev.s390x-latest.args |  4 +-
 .../net-hostdev-bootorder.x86_64-latest.args  |  4 +-
 ...net-hostdev-multidomain.x86_64-latest.args |  4 +-
 ...ostdev-vfio-multidomain.x86_64-latest.args |  4 +-
 .../net-hostdev-vfio.x86_64-latest.args       |  4 +-
 .../net-hostdev.x86_64-latest.args            |  4 +-
 .../net-vdpa-multiqueue.x86_64-latest.args    |  4 +-
 .../net-vdpa.x86_64-latest.args               |  4 +-
 ...-virtio-teaming-hostdev.x86_64-latest.args |  4 +-
 ...-virtio-teaming-network.x86_64-latest.args |  4 +-
 .../net-virtio-teaming.x86_64-latest.args     |  4 +-
 .../pci-rom.x86_64-latest.args                |  4 +-
 .../pseries-hostdevs-1.ppc64-latest.args      |  4 +-
 .../pseries-hostdevs-2.ppc64-latest.args      |  4 +-
 .../pseries-hostdevs-3.ppc64-latest.args      |  4 +-
 56 files changed, 271 insertions(+), 85 deletions(-)

-- 
2.47.1


^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2026-10-09 14:04 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-10-09 14:04 [RFC HCI-8.1 PATCH 0/5] qemu: allow qemu-update with VFIO hostdevs on shared memfd RAM #VSTOR-103860 Andrey Drobyshev
2026-10-09 14:04 ` [RFC HCI-8.1 PATCH 1/5] qemu update: allow qemu-update for VMs with PCI and USB hostdevs #VSTOR-103860 #VSTOR-137800 Andrey Drobyshev
2026-10-09 14:04 ` [RFC HCI-8.1 PATCH 2/5] qemu: capabilities: probe aux-ram-share machine property #VSTOR-103860 Andrey Drobyshev
2026-10-09 14:04 ` [RFC HCI-8.1 PATCH 3/5] qemu: back guest RAM with shared memfd for VMs with DMA-pinning devices #VSTOR-103860 Andrey Drobyshev
2026-10-09 14:04 ` [RFC HCI-8.1 PATCH 4/5] qemu: share auxiliary RAM of " Andrey Drobyshev
2026-10-09 14:04 ` [RFC HCI-8.1 PATCH 5/5] qemu update: refuse PCI hostdevs on non-shared guest RAM #VSTOR-103860 Andrey Drobyshev

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox