[QEMU HCI-8.0 PATCH v4 0/5] vhost-blk change backend setup

[QEMU HCI-8.0 PATCH v4 0/5] vhost-blk change backend setup

[Andrey Zhadchenko]

This series changes vhost-blk backend setup from block node to
a path to the device.
Getting fd from block node was a hack in the first place. It
looked ugly and broke encapsulation.
Change vhost-blk to have it's own property so it opens and owns
the descriptor.

Patches #4 implement auto-resize by watching netlink. The descripor
comes prepared with some filter from outside.

v2:
patch#1
 - old patch dropped

patch#2
 - moved double close fix to new 'do not double close vhostfd' patch
 - dropped DEFINE_BLOCK_PROPERTIES_BASE exposition to the outside. Let's
just use defaults for now. We can later check if tweaking this values
makes any difference and add them back.
 - change vhost_blk_update_size() to return zero or error and move 'changed'
to a separate argument
 - changed lseek to BLKGETSIZE64 (we already use BLKROGET anyway)

patch#3
 - check BLKROGET only if rw was requested

patch#4
 - rework patch a bit: now we expect opened and filtered fd from
libvirt.

patch#5
 - old one dropped, setup moved to libvirt
 - added new patch to support uevent socket cpr

v3:
patch#1
 - extend the double-close fix to preserve and free vqs after
vhost_dev_init() clears dev

patch#2
 - make vhost_blk_update_size() return the changed flag directly
 - use ERRP_GUARD() for reliable error propagation

patch#4
 - accept a monitor fd name passed with getfd instead of a
/dev/fdset path
 - verify that the socket is subscribed to kernel uevents
 - detach the socket after a permanent receive error

patch#5
 - use the proxy ID for the CPR fd name and block CPR without an ID
 - validate restored sockets and improve failure cleanup

v4:
patch#2
 - document the default geometry values

patch#4
 - keep the uevent socket local until it has been validated

patch#5
 - split cold-boot and restore socket setup paths
 - reject restored socket/property mismatches
 - block CPR after a permanent uevent receive error

Andrey Zhadchenko (5):
  vhost-blk: improve vhost_blk_device_realize cleanup
  vhost-blk: change backend setup
  vhost-blk: add read-only flag
  vhost-blk: watch the device for resize events
  vhost-blk: preserve the uevent socket across cpr-exec

 hw/block/vhost-blk.c          | 452 ++++++++++++++++++++++++++++++----
 include/hw/virtio/vhost-blk.h |  11 +-
 2 files changed, 415 insertions(+), 48 deletions(-)

-- 
2.43.5

[QEMU HCI-8.0 PATCH v4 1/5] vhost-blk: improve vhost_blk_device_realize cleanup

[Andrey Zhadchenko]

vhost_dev_init() does a lot of unexpected things:
 - closes the backend fd on failure
 - memsets dev as 0 (thus clearing dev->vqs)

Cleanup properly: don't double-close vhostfd, factor out vqs
so we do not leak it during cleanup.

Fixes: 0ff3cb5ff2 ("block: add vhost-blk backend")
https://virtuozzo.atlassian.net/browse/VSTOR-143437
Reviewed-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>
Signed-off-by: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
---
v2:
 - old patch dropped

v3:
 - extend the double-close fix to preserve and free vqs after
vhost_dev_init() clears dev

 hw/block/vhost-blk.c | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/hw/block/vhost-blk.c b/hw/block/vhost-blk.c
index c52851fcf8..20c2f41691 100644
--- a/hw/block/vhost-blk.c
+++ b/hw/block/vhost-blk.c
@@ -237,6 +237,7 @@ static void vhost_blk_device_realize(DeviceState *dev, Error **errp)
     VirtIODevice *vdev = VIRTIO_DEVICE(dev);
     VHostBlk *s = VHOST_BLK(vdev);
     VhostBlkConf *conf = &s->conf;
+    struct vhost_virtqueue *vqs;
     int i, ret;
 
     if (!conf->conf.blk) {
@@ -289,7 +290,8 @@ static void vhost_blk_device_realize(DeviceState *dev, Error **errp)
 
     s->dev.nvqs = conf->num_queues;
     s->dev.max_queues = conf->num_queues;
-    s->dev.vqs = g_new0(struct vhost_virtqueue, s->dev.nvqs);
+    vqs = g_new0(struct vhost_virtqueue, s->dev.nvqs);
+    s->dev.vqs = vqs;
     s->dev.vq_index = 0;
 
     virtio_init(vdev, VIRTIO_ID_BLOCK, sizeof(struct virtio_blk_config));
@@ -318,6 +320,8 @@ static void vhost_blk_device_realize(DeviceState *dev, Error **errp)
     if (ret < 0) {
         error_setg(errp, "vhost-blk: vhost initialization failed: %s",
                 strerror(-ret));
+        /* vhost_dev_init() closes vhostfd on failure */
+        s->vhostfd = -1;
         goto cleanup;
     }
 
@@ -327,8 +331,11 @@ cleanup:
     if (s->mighand) {
             qemu_del_vm_change_state_handler(s->mighand);
     }
-    g_free(s->dev.vqs);
-    close(s->vhostfd);
+    g_free(vqs);
+    if (s->vhostfd >= 0) {
+        close(s->vhostfd);
+        s->vhostfd = -1;
+    }
     for (i = 0; i < conf->num_queues; i++) {
         virtio_del_queue(vdev, i);
     }
@@ -470,6 +477,7 @@ static void vhost_blk_instance_init(Object *obj)
 {
     VHostBlk *s = VHOST_BLK(obj);
 
+    s->vhostfd = -1;
     device_add_bootindex_property(obj, &s->conf.conf.bootindex,
                                   "bootindex", "/disk@0,0",
                                   DEVICE(obj));
-- 
2.43.5

[QEMU HCI-8.0 PATCH v4 2/5] vhost-blk: change backend setup

[Andrey Zhadchenko]

Previously we used very ugly and incapsulation-breaking assignment
fd = blk_bs(s->conf.conf.blk)->file->bs->opaque;
It is wrong in a many ways, so let's rework this.

Patch changes default `drive` to new `devpath` option so device fd
is managed by vhost-blk itself. Unfortunately this way we need a
bit more preparational work: finding out disk length, etc. Don't
be too broad and just do the minimal work.
Also drop generic block device properties along with the block
node: the kernel module does all IO in terms of 512 sectors, so
simply report 512 byte logical/physical block size to the guest.
Existing vhost-blk setups will break, but it is expected.

We also lose resize, as this is tied to the block node, which is
now have no place in the setup. We will add this in the next
patches as well as RO mode.

https://virtuozzo.atlassian.net/browse/VSTOR-143437
Signed-off-by: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
---
v2:
 - moved double close fix to 'do not double close vhostfd' patch
 - dropped DEFINE_BLOCK_PROPERTIES_BASE exposition to the outside. Let's
just use defaults for now. We can later check if tweaking this values
makes any difference and add them back.
 - change vhost_blk_update_size() to return zero or error and move 'changed'
to a separate argument
 - changed lseek to BLKGETSIZE64 (we already use BLKROGET anyway)

v3:
 - make vhost_blk_update_size() return the changed flag directly
 - use ERRP_GUARD() for reliable error propagation

v4:
 - document the default geometry values

 hw/block/vhost-blk.c          | 126 ++++++++++++++++++++--------------
 include/hw/virtio/vhost-blk.h |   5 +-
 2 files changed, 79 insertions(+), 52 deletions(-)

diff --git a/hw/block/vhost-blk.c b/hw/block/vhost-blk.c
index 20c2f41691..a517c5e22b 100644
--- a/hw/block/vhost-blk.c
+++ b/hw/block/vhost-blk.c
@@ -25,22 +25,17 @@
 #include "linux-headers/linux/vhost.h"
 #include <sys/ioctl.h>
 #include <linux/fs.h>
-#include "include/block/block_int-common.h"
 #include "system/runstate.h"
 
 static int vhost_blk_start(VirtIODevice *vdev)
 {
     VHostBlk *s = VHOST_BLK(vdev);
     struct vhost_vring_file backend;
-    int ret, i, nworkers, *fd;
+    int ret, i, nworkers;
     BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev)));
     VirtioBusClass *k = VIRTIO_BUS_GET_CLASS(qbus);
     char serial[VIRTIO_BLK_ID_BYTES] = {0};
 
-    bdrv_graph_rdlock_main_loop();
-    fd = blk_bs(s->conf.conf.blk)->file->bs->opaque;
-    bdrv_graph_rdunlock_main_loop();
-
     if (!k->set_guest_notifiers) {
         error_report("vhost-blk: binding does not support guest notifiers");
         return -ENOSYS;
@@ -92,7 +87,7 @@ static int vhost_blk_start(VirtIODevice *vdev)
 
     memset(&backend, 0, sizeof(backend));
     backend.index = 0;
-    backend.fd = *fd;
+    backend.fd = s->backend_fd;
     if (ioctl(s->vhostfd, VHOST_BLK_SET_BACKEND, &backend)) {
         error_report("vhost-blk: unable to set backend");
         ret = -errno;
@@ -208,29 +203,71 @@ static void vhost_blk_vm_state(void *opaque, bool running, RunState state)
     }
 }
 
-static void vhost_blk_resize_cb(void *opaque)
+static bool vhost_blk_update_size(VHostBlk *s, Error **errp)
 {
-    VirtIODevice *vdev = opaque;
+    BlockConf *conf = &s->conf.conf;
+    uint64_t length;
+    bool changed;
+
+    if (ioctl(s->backend_fd, BLKGETSIZE64, &length) < 0) {
+        error_setg_errno(errp, errno,
+                         "vhost-blk: unable to determine size of '%s'",
+                         s->conf.devpath);
+        return false;
+    }
 
-    assert(qemu_get_current_aio_context() == qemu_get_aio_context());
-    virtio_notify_config(vdev);
+    changed = s->length != length;
+    s->length = length;
+
+    /* Default geometry mimicking hw/block/hd-geometry.c */
+    conf->heads = 16;
+    conf->secs = 63;
+    conf->cyls = s->length / BDRV_SECTOR_SIZE /
+                 (conf->heads * conf->secs);
+    conf->cyls = MIN(MAX(conf->cyls, 2U), 16383U);
+
+    return changed;
 }
 
-static void vhost_blk_resize(void *opaque)
+static bool vhost_blk_open_backend(VHostBlk *s, Error **errp)
 {
-    VirtIODevice *vdev = VIRTIO_DEVICE(opaque);
+    ERRP_GUARD();
+    BlockConf *conf = &s->conf.conf;
+    struct stat st;
 
-    /*
-     * virtio_notify_config() needs to acquire the global mutex,
-     * so it can't be called from an iothread. Instead, schedule
-     * it to be run in the main context BH.
-     */
-    aio_bh_schedule_oneshot(qemu_get_aio_context(), vhost_blk_resize_cb, vdev);
-}
+    s->backend_fd = qemu_open(s->conf.devpath, O_RDWR, errp);
+    if (s->backend_fd < 0) {
+        error_prepend(errp, "vhost-blk: unable to open backend: ");
+        return false;
+    }
 
-static const BlockDevOps vhost_blk_block_ops = {
-    .resize_cb     = vhost_blk_resize,
-};
+    if (fstat(s->backend_fd, &st) < 0) {
+        error_setg_errno(errp, errno, "vhost-blk: unable to stat '%s'",
+                         s->conf.devpath);
+        goto fail;
+    }
+
+    if (!S_ISBLK(st.st_mode)) {
+        error_setg(errp, "vhost-blk: '%s' is not a block device",
+                   s->conf.devpath);
+        goto fail;
+    }
+
+    vhost_blk_update_size(s, errp);
+    if (*errp) {
+        goto fail;
+    }
+
+    conf->logical_block_size = BDRV_SECTOR_SIZE;
+    conf->physical_block_size = BDRV_SECTOR_SIZE;
+
+    return true;
+
+fail:
+    qemu_close(s->backend_fd);
+    s->backend_fd = -1;
+    return false;
+}
 
 static void vhost_blk_device_realize(DeviceState *dev, Error **errp)
 {
@@ -240,13 +277,8 @@ static void vhost_blk_device_realize(DeviceState *dev, Error **errp)
     struct vhost_virtqueue *vqs;
     int i, ret;
 
-    if (!conf->conf.blk) {
-        error_setg(errp, "vhost-blk: drive property not set");
-        return;
-    }
-
-    if (!blk_is_inserted(conf->conf.blk)) {
-        error_setg(errp, "vhost-blk: device needs media, but drive is empty");
+    if (!conf->devpath) {
+        error_setg(errp, "vhost-blk: devpath property must be set");
         return;
     }
 
@@ -274,17 +306,7 @@ static void vhost_blk_device_realize(DeviceState *dev, Error **errp)
         return;
     }
 
-    if (!blkconf_apply_backend_options(&conf->conf,
-                                       !blk_supports_write_perm(conf->conf.blk),
-                                       true, errp)) {
-        return;
-    }
-
-    if (!blkconf_geometry(&conf->conf, NULL, 65535, 255, 255, errp)) {
-        return;
-    }
-
-    if (!blkconf_blocksizes(&conf->conf, errp)) {
+    if (!vhost_blk_open_backend(s, errp)) {
         return;
     }
 
@@ -313,8 +335,6 @@ static void vhost_blk_device_realize(DeviceState *dev, Error **errp)
         goto cleanup;
     }
 
-    blk_set_dev_ops(s->conf.conf.blk, &vhost_blk_block_ops, s);
-
     ret = vhost_dev_init(&s->dev, (void *)((size_t)s->vhostfd),
                          VHOST_BACKEND_TYPE_KERNEL, 0, NULL);
     if (ret < 0) {
@@ -336,6 +356,10 @@ cleanup:
         close(s->vhostfd);
         s->vhostfd = -1;
     }
+    if (s->backend_fd >= 0) {
+        qemu_close(s->backend_fd);
+        s->backend_fd = -1;
+    }
     for (i = 0; i < conf->num_queues; i++) {
         virtio_del_queue(vdev, i);
     }
@@ -351,6 +375,10 @@ static void vhost_blk_device_unrealize(DeviceState *dev)
     qemu_del_vm_change_state_handler(s->mighand);
     vhost_blk_set_status(vdev, 0);
     vhost_dev_cleanup(&s->dev);
+    if (s->backend_fd >= 0) {
+        qemu_close(s->backend_fd);
+        s->backend_fd = -1;
+    }
     g_free(s->dev.vqs);
     virtio_cleanup(vdev);
 }
@@ -383,10 +411,6 @@ static uint64_t vhost_blk_get_features(VirtIODevice *vdev,
 
     virtio_add_feature(&features, VIRTIO_F_VERSION_1);
 
-    if (!blk_is_writable(s->conf.conf.blk)) {
-        virtio_add_feature(&features, VIRTIO_BLK_F_RO);
-    }
-
     if (s->conf.num_queues > 1) {
         virtio_add_feature(&features, VIRTIO_BLK_F_MQ);
     }
@@ -405,7 +429,9 @@ static void vhost_blk_update_config(VirtIODevice *vdev, uint8_t *config)
     int64_t length;
     int blk_size = conf->logical_block_size;
 
-    blk_get_geometry(s->conf.conf.blk, &capacity);
+    length = s->length;
+    capacity = length / BDRV_SECTOR_SIZE;
+
     memset(&blkcfg, 0, sizeof(blkcfg));
     virtio_stq_p(vdev, &blkcfg.capacity, capacity);
     virtio_stl_p(vdev, &blkcfg.seg_max, s->conf.queue_size - 2);
@@ -413,7 +439,6 @@ static void vhost_blk_update_config(VirtIODevice *vdev, uint8_t *config)
     virtio_stl_p(vdev, &blkcfg.blk_size, blk_size);
     blkcfg.geometry.heads = conf->heads;
 
-    length = blk_getlength(s->conf.conf.blk);
     if (length > 0 && length / conf->heads / conf->secs % blk_size) {
         unsigned short mask;
 
@@ -432,7 +457,7 @@ static void vhost_blk_update_config(VirtIODevice *vdev, uint8_t *config)
 }
 
 static const Property vhost_blk_properties[] = {
-    DEFINE_BLOCK_PROPERTIES(VHostBlk, conf.conf),
+    DEFINE_PROP_STRING("devpath", VHostBlk, conf.devpath),
     DEFINE_PROP_UINT16("num-queues", VHostBlk, conf.num_queues,
                        VHOST_BLK_AUTO_NUM_QUEUES),
     DEFINE_PROP_UINT16("queue-size", VHostBlk, conf.queue_size, 256),
@@ -478,6 +503,7 @@ static void vhost_blk_instance_init(Object *obj)
     VHostBlk *s = VHOST_BLK(obj);
 
     s->vhostfd = -1;
+    s->backend_fd = -1;
     device_add_bootindex_property(obj, &s->conf.conf.bootindex,
                                   "bootindex", "/disk@0,0",
                                   DEVICE(obj));
diff --git a/include/hw/virtio/vhost-blk.h b/include/hw/virtio/vhost-blk.h
index 0c7e212595..c194b421d9 100644
--- a/include/hw/virtio/vhost-blk.h
+++ b/include/hw/virtio/vhost-blk.h
@@ -14,7 +14,6 @@
 #include "standard-headers/linux/virtio_blk.h"
 #include "hw/block/block.h"
 #include "hw/virtio/vhost.h"
-#include "system/block-backend.h"
 
 #define TYPE_VHOST_BLK "vhost-blk"
 #define VHOST_BLK(obj) \
@@ -25,6 +24,7 @@
 
 typedef struct VhostBlkConf {
     BlockConf conf;
+    char *devpath;
     uint16_t num_queues;
     uint16_t queue_size;
     uint16_t num_threads;
@@ -37,10 +37,11 @@ typedef struct VHostBlk {
     VMChangeStateEntry *mighand;
     uint64_t host_features;
     uint64_t decided_features;
-    struct virtio_blk_config blkcfg;
     int vhostfd;
+    int backend_fd;
     struct vhost_dev dev;
     bool vhost_started;
+    uint64_t length;
 } VHostBlk;
 
 #endif
-- 
2.43.5

[QEMU HCI-8.0 PATCH v4 3/5] vhost-blk: add read-only flag

[Andrey Zhadchenko]

Also compare BLKROGET with the selected mode and reject r/w if
needed.

https://virtuozzo.atlassian.net/browse/VSTOR-143437
Reviewed-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>
Signed-off-by: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
---
v2:
 - check BLKROGET only if rw was requested like posix_file

 hw/block/vhost-blk.c          | 25 ++++++++++++++++++++++++-
 include/hw/virtio/vhost-blk.h |  1 +
 2 files changed, 25 insertions(+), 1 deletion(-)

diff --git a/hw/block/vhost-blk.c b/hw/block/vhost-blk.c
index a517c5e22b..dc21f9fbbf 100644
--- a/hw/block/vhost-blk.c
+++ b/hw/block/vhost-blk.c
@@ -234,8 +234,9 @@ static bool vhost_blk_open_backend(VHostBlk *s, Error **errp)
     ERRP_GUARD();
     BlockConf *conf = &s->conf.conf;
     struct stat st;
+    int open_flags = s->conf.readonly ? O_RDONLY : O_RDWR;
 
-    s->backend_fd = qemu_open(s->conf.devpath, O_RDWR, errp);
+    s->backend_fd = qemu_open(s->conf.devpath, open_flags, errp);
     if (s->backend_fd < 0) {
         error_prepend(errp, "vhost-blk: unable to open backend: ");
         return false;
@@ -253,6 +254,23 @@ static bool vhost_blk_open_backend(VHostBlk *s, Error **errp)
         goto fail;
     }
 
+    if (!s->conf.readonly) {
+        int readonly;
+
+        if (ioctl(s->backend_fd, BLKROGET, &readonly) < 0) {
+            error_setg_errno(errp, errno,
+                             "vhost-blk: unable to get read-only status of "
+                             "'%s'", s->conf.devpath);
+            goto fail;
+        }
+
+        if (readonly) {
+            error_setg_errno(errp, EROFS, "vhost-blk: '%s' is not writable",
+                             s->conf.devpath);
+            goto fail;
+        }
+    }
+
     vhost_blk_update_size(s, errp);
     if (*errp) {
         goto fail;
@@ -411,6 +429,10 @@ static uint64_t vhost_blk_get_features(VirtIODevice *vdev,
 
     virtio_add_feature(&features, VIRTIO_F_VERSION_1);
 
+    if (s->conf.readonly) {
+        virtio_add_feature(&features, VIRTIO_BLK_F_RO);
+    }
+
     if (s->conf.num_queues > 1) {
         virtio_add_feature(&features, VIRTIO_BLK_F_MQ);
     }
@@ -458,6 +480,7 @@ static void vhost_blk_update_config(VirtIODevice *vdev, uint8_t *config)
 
 static const Property vhost_blk_properties[] = {
     DEFINE_PROP_STRING("devpath", VHostBlk, conf.devpath),
+    DEFINE_PROP_BOOL("read-only", VHostBlk, conf.readonly, false),
     DEFINE_PROP_UINT16("num-queues", VHostBlk, conf.num_queues,
                        VHOST_BLK_AUTO_NUM_QUEUES),
     DEFINE_PROP_UINT16("queue-size", VHostBlk, conf.queue_size, 256),
diff --git a/include/hw/virtio/vhost-blk.h b/include/hw/virtio/vhost-blk.h
index c194b421d9..c6646f5845 100644
--- a/include/hw/virtio/vhost-blk.h
+++ b/include/hw/virtio/vhost-blk.h
@@ -25,6 +25,7 @@
 typedef struct VhostBlkConf {
     BlockConf conf;
     char *devpath;
+    bool readonly;
     uint16_t num_queues;
     uint16_t queue_size;
     uint16_t num_threads;
-- 
2.43.5

[QEMU HCI-8.0 PATCH v4 4/5] vhost-blk: watch the device for resize events

[Andrey Zhadchenko]

Resize was tied to block node, which we removed a few patches ago.
Luckily we can make resize automated: receive an uevent socket
from the management layer via the new "ueventfd" property (a
monitor fd name passed with the 'getfd' QMP command), watch it for
relevant netlink messages and call virtio_notify_config() if we
detect a capacity change.

The socket is set up (possibly with filter) by management layer.
We only need to check that it is netlink, subscribed to kernel
uevents, and do some message filtering.

When the property is not set, capacity changes are not detected.

https://virtuozzo.atlassian.net/browse/VSTOR-143437
Reviewed-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>
Signed-off-by: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
---
v2:
 - rework patch a bit: now we expect opened and filtered fd from
libvirt.

v3:
 - accept a monitor fd name passed with getfd instead of a
/dev/fdset path
 - verify that the socket is subscribed to kernel uevents
 - detach the socket after a permanent receive error
 - change buffer capacity to 4KiB

v4:
 - keep the uevent socket local until it has been validated

 hw/block/vhost-blk.c          | 202 ++++++++++++++++++++++++++++++++++
 include/hw/virtio/vhost-blk.h |   4 +
 2 files changed, 206 insertions(+)

diff --git a/hw/block/vhost-blk.c b/hw/block/vhost-blk.c
index dc21f9fbbf..ef46c55665 100644
--- a/hw/block/vhost-blk.c
+++ b/hw/block/vhost-blk.c
@@ -10,7 +10,10 @@
 
 #include "qemu/osdep.h"
 #include "qapi/error.h"
+#include "qemu/cutils.h"
 #include "qemu/error-report.h"
+#include "qemu/main-loop.h"
+#include "monitor/monitor.h"
 #include "qom/object.h"
 #include "hw/qdev-core.h"
 #include "hw/boards.h"
@@ -25,6 +28,7 @@
 #include "linux-headers/linux/vhost.h"
 #include <sys/ioctl.h>
 #include <linux/fs.h>
+#include <linux/netlink.h>
 #include "system/runstate.h"
 
 static int vhost_blk_start(VirtIODevice *vdev)
@@ -229,6 +233,193 @@ static bool vhost_blk_update_size(VHostBlk *s, Error **errp)
     return changed;
 }
 
+static void vhost_blk_resize_bh(void *opaque)
+{
+    VHostBlk *s = opaque;
+    Error *local_err = NULL;
+    bool changed;
+
+    changed = vhost_blk_update_size(s, &local_err);
+    if (local_err) {
+        error_report_err(local_err);
+        return;
+    }
+
+    if (changed) {
+        virtio_notify_config(VIRTIO_DEVICE(s));
+    }
+}
+
+static void vhost_blk_uevent_detach(VHostBlk *s)
+{
+    if (s->uevent_fd < 0) {
+        return;
+    }
+
+    qemu_set_fd_handler(s->uevent_fd, NULL, NULL, NULL);
+    qemu_bh_delete(s->resize_bh);
+    s->resize_bh = NULL;
+    qemu_close(s->uevent_fd);
+    s->uevent_fd = -1;
+}
+
+/*
+ * The uevent socket is created, bound and filtered by the management
+ * layer and passed to us via the "ueventfd" property.
+ */
+static void vhost_blk_uevent_read(void *opaque)
+{
+    VHostBlk *s = opaque;
+    char buffer[4096];
+
+    for (;;) {
+        struct sockaddr_nl source;
+        socklen_t source_len = sizeof(source);
+        uint64_t event_major = UINT64_MAX;
+        uint64_t event_minor = UINT64_MAX;
+        bool action_change = false;
+        bool subsystem_block = false;
+        bool resize = false;
+        char *field;
+        char *end;
+        ssize_t len;
+
+        memset(&source, 0, sizeof(source));
+        len = recvfrom(s->uevent_fd, buffer, sizeof(buffer) - 1,
+                       MSG_DONTWAIT, (struct sockaddr *)&source, &source_len);
+        if (len < 0) {
+            if (errno == EINTR) {
+                continue;
+            }
+            if (errno == ENOBUFS) {
+                /* Some events may be dropped, just re-check */
+                qemu_bh_schedule(s->resize_bh);
+                continue;
+            }
+            if (errno != EAGAIN && errno != EWOULDBLOCK) {
+                error_report("vhost-blk: detaching uevent after the error: %s",
+                             strerror(errno));
+                vhost_blk_uevent_detach(s);
+            }
+            return;
+        }
+
+        if (source.nl_family != AF_NETLINK || source.nl_pid != 0) {
+            continue;
+        }
+
+        buffer[len] = '\0';
+        field = buffer;
+        end = buffer + len;
+        while (field < end) {
+            size_t field_len = strnlen(field, end - field);
+
+            if (!strcmp(field, "ACTION=change")) {
+                action_change = true;
+            } else if (!strcmp(field, "SUBSYSTEM=block")) {
+                subsystem_block = true;
+            } else if (!strcmp(field, "RESIZE=1")) {
+                resize = true;
+            } else if (g_str_has_prefix(field, "MAJOR=")) {
+                uint64_t value;
+
+                if (!qemu_strtou64(field + strlen("MAJOR="), NULL, 10,
+                                   &value)) {
+                    event_major = value;
+                }
+            } else if (g_str_has_prefix(field, "MINOR=")) {
+                uint64_t value;
+
+                if (!qemu_strtou64(field + strlen("MINOR="), NULL, 10,
+                                   &value)) {
+                    event_minor = value;
+                }
+            }
+
+            if (field_len == end - field) {
+                break;
+            }
+            field += field_len + 1;
+        }
+
+        if (action_change && subsystem_block && resize &&
+            event_major == major(s->backend_rdev) &&
+            event_minor == minor(s->backend_rdev)) {
+            qemu_bh_schedule(s->resize_bh);
+        }
+    }
+}
+
+static bool vhost_blk_uevent_check(int fd, const char *src, Error **errp)
+{
+    struct sockaddr_nl addr;
+    socklen_t addrlen;
+    socklen_t optlen;
+    int domain;
+    int protocol;
+
+    optlen = sizeof(domain);
+    if (getsockopt(fd, SOL_SOCKET, SO_DOMAIN, &domain, &optlen) < 0) {
+        error_setg_errno(errp, errno, "vhost-blk: '%s' is not a socket", src);
+        return false;
+    }
+
+    optlen = sizeof(protocol);
+    if (getsockopt(fd, SOL_SOCKET, SO_PROTOCOL, &protocol, &optlen) < 0) {
+        error_setg_errno(errp, errno,
+                         "vhost-blk: unable to get protocol of '%s'", src);
+        return false;
+    }
+
+    if (domain != AF_NETLINK || protocol != NETLINK_KOBJECT_UEVENT) {
+        error_setg(errp,
+                   "vhost-blk: '%s' is not a NETLINK_KOBJECT_UEVENT socket",
+                   src);
+        return false;
+    }
+
+    addrlen = sizeof(addr);
+    if (getsockname(fd, (struct sockaddr *)&addr, &addrlen) < 0) {
+        error_setg_errno(errp, errno,
+                         "vhost-blk: unable to get address of '%s'", src);
+        return false;
+    }
+
+    if (addr.nl_groups != 1) {
+        error_setg(errp,
+                   "vhost-blk: '%s' is not subscribed to kernel uevents",
+                   src);
+        return false;
+    }
+
+    return true;
+}
+
+static bool vhost_blk_uevent_attach(VHostBlk *s, Error **errp)
+{
+    int fd;
+
+    if (!s->conf.ueventfd) {
+        return true;
+    }
+
+    fd = monitor_fd_param(monitor_cur(), s->conf.ueventfd, errp);
+    if (fd < 0) {
+        error_prepend(errp, "vhost-blk: unable to obtain uevent socket: ");
+        return false;
+    }
+
+    if (!vhost_blk_uevent_check(fd, s->conf.ueventfd, errp)) {
+        qemu_close(fd);
+        return false;
+    }
+
+    s->uevent_fd = fd;
+    s->resize_bh = qemu_bh_new(vhost_blk_resize_bh, s);
+    qemu_set_fd_handler(s->uevent_fd, vhost_blk_uevent_read, NULL, s);
+    return true;
+}
+
 static bool vhost_blk_open_backend(VHostBlk *s, Error **errp)
 {
     ERRP_GUARD();
@@ -253,6 +444,7 @@ static bool vhost_blk_open_backend(VHostBlk *s, Error **errp)
                    s->conf.devpath);
         goto fail;
     }
+    s->backend_rdev = st.st_rdev;
 
     if (!s->conf.readonly) {
         int readonly;
@@ -328,6 +520,12 @@ static void vhost_blk_device_realize(DeviceState *dev, Error **errp)
         return;
     }
 
+    if (!vhost_blk_uevent_attach(s, errp)) {
+        qemu_close(s->backend_fd);
+        s->backend_fd = -1;
+        return;
+    }
+
     s->dev.nvqs = conf->num_queues;
     s->dev.max_queues = conf->num_queues;
     vqs = g_new0(struct vhost_virtqueue, s->dev.nvqs);
@@ -378,6 +576,7 @@ cleanup:
         qemu_close(s->backend_fd);
         s->backend_fd = -1;
     }
+    vhost_blk_uevent_detach(s);
     for (i = 0; i < conf->num_queues; i++) {
         virtio_del_queue(vdev, i);
     }
@@ -390,6 +589,7 @@ static void vhost_blk_device_unrealize(DeviceState *dev)
     VirtIODevice *vdev = VIRTIO_DEVICE(dev);
     VHostBlk *s = VHOST_BLK(dev);
 
+    vhost_blk_uevent_detach(s);
     qemu_del_vm_change_state_handler(s->mighand);
     vhost_blk_set_status(vdev, 0);
     vhost_dev_cleanup(&s->dev);
@@ -480,6 +680,7 @@ static void vhost_blk_update_config(VirtIODevice *vdev, uint8_t *config)
 
 static const Property vhost_blk_properties[] = {
     DEFINE_PROP_STRING("devpath", VHostBlk, conf.devpath),
+    DEFINE_PROP_STRING("ueventfd", VHostBlk, conf.ueventfd),
     DEFINE_PROP_BOOL("read-only", VHostBlk, conf.readonly, false),
     DEFINE_PROP_UINT16("num-queues", VHostBlk, conf.num_queues,
                        VHOST_BLK_AUTO_NUM_QUEUES),
@@ -527,6 +728,7 @@ static void vhost_blk_instance_init(Object *obj)
 
     s->vhostfd = -1;
     s->backend_fd = -1;
+    s->uevent_fd = -1;
     device_add_bootindex_property(obj, &s->conf.conf.bootindex,
                                   "bootindex", "/disk@0,0",
                                   DEVICE(obj));
diff --git a/include/hw/virtio/vhost-blk.h b/include/hw/virtio/vhost-blk.h
index c6646f5845..e3ad92bc6e 100644
--- a/include/hw/virtio/vhost-blk.h
+++ b/include/hw/virtio/vhost-blk.h
@@ -25,6 +25,7 @@
 typedef struct VhostBlkConf {
     BlockConf conf;
     char *devpath;
+    char *ueventfd;
     bool readonly;
     uint16_t num_queues;
     uint16_t queue_size;
@@ -40,9 +41,12 @@ typedef struct VHostBlk {
     uint64_t decided_features;
     int vhostfd;
     int backend_fd;
+    int uevent_fd;
     struct vhost_dev dev;
     bool vhost_started;
     uint64_t length;
+    uint64_t backend_rdev;
+    QEMUBH *resize_bh;
 } VHostBlk;
 
 #endif
-- 
2.43.5

[QEMU HCI-8.0 PATCH v4 5/5] vhost-blk: preserve the uevent socket across cpr-exec

[Andrey Zhadchenko]

qemu-update uses cpr-exec migration: QEMU re-execs itself in place.
A plain monitor fdset descriptor like our uevent socket does not
survive that (unlike the cpr_save_fd()'d tap/vhost fds), so without
help the resized-device notifications would silently stop until the
next full VM start.

Preserve it the same way the other devices do: on cold boot
cpr_save_fd() the socket under the "<proxy id>_ueventfd" key, and
on the re-exec'd (incoming) side adopt it with cpr_find_fd()
instead of reopening a command line fd that no longer resolves.
The management layer passes a placeholder for the "ueventfd"
property in that case.  The restored socket is validated just like
a cold-boot one.

Block CPR migration modes when the proxy has no ID or we disabled
the socket after the error.

Drop the saved descriptor on teardown so it does not outlive the
device.

https://virtuozzo.atlassian.net/browse/VSTOR-143437
Signed-off-by: Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com>
---
v2:
 - old one dropped, setup moved to libvirt
 - added new patch to support uevent socket cpr

v3:
 - use the proxy ID for the CPR fd name and block CPR without an ID
 - validate restored sockets and improve failure cleanup

v4:
 - split cold-boot and restore socket setup paths
 - reject restored socket/property mismatches
 - block CPR after a permanent uevent receive error

 hw/block/vhost-blk.c          | 107 +++++++++++++++++++++++++++++++++-
 include/hw/virtio/vhost-blk.h |   1 +
 2 files changed, 105 insertions(+), 3 deletions(-)

diff --git a/hw/block/vhost-blk.c b/hw/block/vhost-blk.c
index ef46c55665..abddff02c6 100644
--- a/hw/block/vhost-blk.c
+++ b/hw/block/vhost-blk.c
@@ -29,6 +29,8 @@
 #include <sys/ioctl.h>
 #include <linux/fs.h>
 #include <linux/netlink.h>
+#include "migration/blocker.h"
+#include "migration/cpr.h"
 #include "system/runstate.h"
 
 static int vhost_blk_start(VirtIODevice *vdev)
@@ -250,12 +252,27 @@ static void vhost_blk_resize_bh(void *opaque)
     }
 }
 
+static char *vhost_blk_cpr_name(VHostBlk *s)
+{
+    DeviceState *proxy = qdev_get_parent_bus(DEVICE(s))->parent;
+
+    return proxy->id ? g_strdup_printf("%s_ueventfd", proxy->id) : NULL;
+}
+
 static void vhost_blk_uevent_detach(VHostBlk *s)
 {
+    g_autofree char *cpr_name = NULL;
+
+    migrate_del_blocker(&s->cpr_blocker);
+
     if (s->uevent_fd < 0) {
         return;
     }
 
+    cpr_name = vhost_blk_cpr_name(s);
+    if (cpr_name) {
+        cpr_delete_fd(cpr_name, 0);
+    }
     qemu_set_fd_handler(s->uevent_fd, NULL, NULL, NULL);
     qemu_bh_delete(s->resize_bh);
     s->resize_bh = NULL;
@@ -263,6 +280,19 @@ static void vhost_blk_uevent_detach(VHostBlk *s)
     s->uevent_fd = -1;
 }
 
+static void vhost_blk_block_cpr(VHostBlk *s)
+{
+    Error *local_err = NULL;
+
+    error_setg(&s->cpr_blocker,
+               "vhost-blk: the uevent socket was detached after an error");
+    if (migrate_add_blocker_modes(&s->cpr_blocker, &local_err,
+                                  MIG_MODE_CPR_TRANSFER,
+                                  MIG_MODE_CPR_EXEC, -1) < 0) {
+        error_report_err(local_err);
+    }
+}
+
 /*
  * The uevent socket is created, bound and filtered by the management
  * layer and passed to us via the "ueventfd" property.
@@ -300,6 +330,7 @@ static void vhost_blk_uevent_read(void *opaque)
                 error_report("vhost-blk: detaching uevent after the error: %s",
                              strerror(errno));
                 vhost_blk_uevent_detach(s);
+                vhost_blk_block_cpr(s);
             }
             return;
         }
@@ -395,7 +426,41 @@ static bool vhost_blk_uevent_check(int fd, const char *src, Error **errp)
     return true;
 }
 
-static bool vhost_blk_uevent_attach(VHostBlk *s, Error **errp)
+static bool vhost_blk_uevent_restore(VHostBlk *s, const char *cpr_name,
+                                     Error **errp)
+{
+    int fd = -1;
+
+    if (cpr_name) {
+        fd = cpr_find_fd(cpr_name, 0);
+    }
+
+    if (!s->conf.ueventfd) {
+        if (fd >= 0) {
+            error_setg(errp, "vhost-blk: a preserved uevent socket exists "
+                       "but the 'ueventfd' property is not configured");
+            return false;
+        }
+        return true;
+    }
+
+    if (fd < 0) {
+        error_setg(errp, "vhost-blk: no preserved uevent socket to restore");
+        return false;
+    }
+
+    if (!vhost_blk_uevent_check(fd, cpr_name, errp)) {
+        cpr_delete_fd(cpr_name, 0);
+        qemu_close(fd);
+        return false;
+    }
+
+    s->uevent_fd = fd;
+    return true;
+}
+
+static bool vhost_blk_uevent_open(VHostBlk *s, const char *cpr_name,
+                                  Error **errp)
 {
     int fd;
 
@@ -403,18 +468,54 @@ static bool vhost_blk_uevent_attach(VHostBlk *s, Error **errp)
         return true;
     }
 
+    if (!cpr_name) {
+        error_setg(&s->cpr_blocker,
+                   "vhost-blk: device ID is required for CPR migration");
+        if (migrate_add_blocker_modes(&s->cpr_blocker, errp,
+                                      MIG_MODE_CPR_TRANSFER,
+                                      MIG_MODE_CPR_EXEC, -1) < 0) {
+            return false;
+        }
+    }
+
     fd = monitor_fd_param(monitor_cur(), s->conf.ueventfd, errp);
     if (fd < 0) {
         error_prepend(errp, "vhost-blk: unable to obtain uevent socket: ");
-        return false;
+        goto fail;
     }
 
     if (!vhost_blk_uevent_check(fd, s->conf.ueventfd, errp)) {
         qemu_close(fd);
-        return false;
+        goto fail;
+    }
+
+    if (cpr_name) {
+        cpr_save_fd(cpr_name, 0, fd);
     }
 
     s->uevent_fd = fd;
+    return true;
+
+fail:
+    migrate_del_blocker(&s->cpr_blocker);
+    return false;
+}
+
+static bool vhost_blk_uevent_attach(VHostBlk *s, Error **errp)
+{
+    g_autofree char *cpr_name = vhost_blk_cpr_name(s);
+    bool ok;
+
+    ok = cpr_is_incoming() ? vhost_blk_uevent_restore(s, cpr_name, errp)
+                           : vhost_blk_uevent_open(s, cpr_name, errp);
+    if (!ok) {
+        return false;
+    }
+
+    if (s->uevent_fd < 0) {
+        return true;
+    }
+
     s->resize_bh = qemu_bh_new(vhost_blk_resize_bh, s);
     qemu_set_fd_handler(s->uevent_fd, vhost_blk_uevent_read, NULL, s);
     return true;
diff --git a/include/hw/virtio/vhost-blk.h b/include/hw/virtio/vhost-blk.h
index e3ad92bc6e..059f1ebc70 100644
--- a/include/hw/virtio/vhost-blk.h
+++ b/include/hw/virtio/vhost-blk.h
@@ -47,6 +47,7 @@ typedef struct VHostBlk {
     uint64_t length;
     uint64_t backend_rdev;
     QEMUBH *resize_bh;
+    Error *cpr_blocker;
 } VHostBlk;
 
 #endif
-- 
2.43.5

Re: [QEMU HCI-8.0 PATCH v4 0/5] vhost-blk change backend setup

[Andrey Drobyshev]

LGTM, applied to hci-8.0.

-- 
Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>

Re: [QEMU HCI-8.0 PATCH v4 2/5] vhost-blk: change backend setup

[Andrey Drobyshev]

On Tue, 08 Sep 2026 15:52:05 +0300, Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com> wrote:
> Previously we used very ugly and incapsulation-breaking assignment
> fd = blk_bs(s->conf.conf.blk)->file->bs->opaque;
> It is wrong in a many ways, so let's rework this.
> 
> Patch changes default `drive` to new `devpath` option so device fd
> is managed by vhost-blk itself. Unfortunately this way we need a
> bit more preparational work: finding out disk length, etc. Don't
> be too broad and just do the minimal work.
> Also drop generic block device properties along with the block
> node: the kernel module does all IO in terms of 512 sectors, so
> simply report 512 byte logical/physical block size to the guest.
> Existing vhost-blk setups will break, but it is expected.
> 
> [...]

Reviewed-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>

-- 
Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>

Re: [QEMU HCI-8.0 PATCH v4 5/5] vhost-blk: preserve the uevent socket across cpr-exec

[Andrey Drobyshev]

On Tue, 08 Sep 2026 15:52:08 +0300, Andrey Zhadchenko <andrey.zhadchenko@virtuozzo.com> wrote:
> qemu-update uses cpr-exec migration: QEMU re-execs itself in place.
> A plain monitor fdset descriptor like our uevent socket does not
> survive that (unlike the cpr_save_fd()'d tap/vhost fds), so without
> help the resized-device notifications would silently stop until the
> next full VM start.
> 
> Preserve it the same way the other devices do: on cold boot
> cpr_save_fd() the socket under the "<proxy id>_ueventfd" key, and
> on the re-exec'd (incoming) side adopt it with cpr_find_fd()
> instead of reopening a command line fd that no longer resolves.
> The management layer passes a placeholder for the "ueventfd"
> property in that case.  The restored socket is validated just like
> a cold-boot one.
> 
> [...]

Reviewed-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>

-- 
Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>