OpenVZ / Virtuozzo kernel development (devel@openvz.org)
 help / color / mirror / Atom feed
* [Devel] [PATCH hci-8.1 v3 0/2] drivers/vhost/blk: support VIRTIO_BLK_F_WRITE_ZEROES
@ 2026-11-17 10:09 Alexander Ivanov
  2026-11-17 10:09 ` [Devel] [PATCH hci-8.1 v3 1/2] drivers/vhost/blk: fix a flush hang on requests completed after worker kill Alexander Ivanov
                   ` (2 more replies)
  0 siblings, 3 replies; 8+ messages in thread
From: Alexander Ivanov @ 2026-11-17 10:09 UTC (permalink / raw)
  To: andrey.zhadchenko; +Cc: devel

Add VIRTIO_BLK_F_WRITE_ZEROES support to vhost-blk for parity with
virtio-blk.

Patch 1 releases requests completed after the vhost worker was killed,
which otherwise hang vhost_blk_flush() on release. Patch 2 handles
VIRTIO_BLK_T_WRITE_ZEROES.

v3:
- patch 1, commit message: the hang needs vhost_task workers; the
  reason for the work item; dropped the VHOST_RESET_OWNER sentence.
- patch 2, commit message: the VMM paragraph says what a large
  max_write_zeroes_sectors costs and that our QEMU does not advertise
  the feature yet.

v2:
- patch 1: new.
- patch 2: shorten the commit message, state parity with virtio-blk
  as the reason and what the VMM must advertise.
- patch 2: issue chunks of at most 16 WRITE ZEROES bios and 1 GiB,
  try them with BLKDEV_ZERO_NOFALLBACK and fall back to 4 MiB chunks
  of zero pages.
- patch 2: complete the request with VIRTIO_BLK_S_UNSUPP when
  VIRTIO_BLK_F_WRITE_ZEROES was not negotiated.

https://virtuozzo.atlassian.net/browse/VSTOR-141684

Alexander Ivanov (2):
  drivers/vhost/blk: fix a flush hang on requests completed after worker
    kill
  drivers/vhost/blk: support VIRTIO_BLK_F_WRITE_ZEROES

 drivers/vhost/blk.c | 214 ++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 209 insertions(+), 5 deletions(-)

-- 
2.53.0

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

^ permalink raw reply	[flat|nested] 8+ messages in thread

* [Devel] [PATCH hci-8.1 v3 1/2] drivers/vhost/blk: fix a flush hang on requests completed after worker kill
  2026-11-17 10:09 [Devel] [PATCH hci-8.1 v3 0/2] drivers/vhost/blk: support VIRTIO_BLK_F_WRITE_ZEROES Alexander Ivanov
@ 2026-11-17 10:09 ` Alexander Ivanov
  2026-11-17 10:14   ` no-reply
  2026-11-17 10:09 ` [Devel] [PATCH hci-8.1 v3 2/2] drivers/vhost/blk: support VIRTIO_BLK_F_WRITE_ZEROES Alexander Ivanov
  2026-11-17 13:50 ` [Devel] [PATCH hci-8.1 v3 0/2] " Konstantin Khorenko
  2 siblings, 1 reply; 8+ messages in thread
From: Alexander Ivanov @ 2026-11-17 10:09 UTC (permalink / raw)
  To: andrey.zhadchenko; +Cc: devel

With vhost_task workers (not the default kthread ones), killing the
owner makes vhost_worker_killed() detach the worker from the virtqueues
and run only the work already queued. A request that completes after
that is added to blk_vq->llhead, but vhost_vq_work_queue() returns
false and queues nothing, so the request is never forgotten.
req_inflight[] stays elevated and vhost_blk_flush() hangs
uninterruptibly in release when the fd is closed. A request in flight
at the kill is enough.

Release such requests from a work item when the worker cannot be
queued, as vhost-scsi does in vhost_scsi_drop_cmds(). The bio
completion may run in interrupt context, where a request cannot be
released: that takes vq->mutex, and flush_lock with plain spin_lock().
The guest is not answered, its memory is no longer reachable.

Once the completion publishes the request, the request can be
forgotten and vhost_blk_flush() can return while the completion is
still queueing work into blk, which release then frees. Queue under
rcu_read_lock() and wait for it with synchronize_rcu() in
vhost_blk_drop_backend(), then flush the drop work before freeing the
requests.

https://virtuozzo.atlassian.net/browse/VSTOR-141684

Fixes: 1629db092dd7 ("drivers/vhost: vhost-blk accelerator for virtio-blk guests")
Feature: vhost-blk: in-kernel accelerator for virtio-blk guests
Signed-off-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
---
 drivers/vhost/blk.c | 36 +++++++++++++++++++++++++++++++++---
 1 file changed, 33 insertions(+), 3 deletions(-)

diff --git a/drivers/vhost/blk.c b/drivers/vhost/blk.c
index b95358cdf3bc..eea8cca59cac 100644
--- a/drivers/vhost/blk.c
+++ b/drivers/vhost/blk.c
@@ -91,6 +91,7 @@ struct vhost_blk_vq {
 	struct iovec iov[UIO_MAXIOV];
 	struct llist_head llhead;
 	struct vhost_work work;
+	struct work_struct drop_work;
 };
 
 struct vhost_blk {
@@ -168,8 +169,13 @@ static void vhost_blk_req_done(struct bio *bio)
 	}
 
 	if (atomic_dec_and_test(&req->bio_nr)) {
-		llist_add(&req->llnode, &req->blk_vq->llhead);
-		vhost_vq_work_queue(&req->blk_vq->vq, &req->blk_vq->work);
+		struct vhost_blk_vq *blk_vq = req->blk_vq;
+
+		rcu_read_lock();
+		llist_add(&req->llnode, &blk_vq->llhead);
+		if (!vhost_vq_work_queue(&blk_vq->vq, &blk_vq->work))
+			schedule_work(&blk_vq->drop_work);
+		rcu_read_unlock();
 	}
 }
 
@@ -683,6 +689,24 @@ static void vhost_blk_handle_host_kick(struct vhost_work *work)
 	mutex_unlock(&vq->mutex);
 }
 
+static void vhost_blk_drop_reqs(struct work_struct *work)
+{
+	struct vhost_blk_vq *blk_vq;
+	struct vhost_blk_req *req, *tmp;
+	struct llist_node *llnode;
+
+	blk_vq = container_of(work, struct vhost_blk_vq, drop_work);
+
+	mutex_lock(&blk_vq->vq.mutex);
+	llnode = llist_del_all(&blk_vq->llhead);
+	llist_for_each_entry_safe(req, tmp, llnode, llnode) {
+		vhost_blk_req_cleanup(req);
+		req->inflight = false;
+		forget_request(req);
+	}
+	mutex_unlock(&blk_vq->vq.mutex);
+}
+
 /* It is forbidden to call more than one vhost_blk_flush() simultaneously */
 static void vhost_blk_flush(struct vhost_blk *blk)
 {
@@ -730,6 +754,10 @@ static void vhost_blk_drop_backend(struct vhost_blk *blk)
 
 	vhost_blk_flush(blk);
 
+	synchronize_rcu();
+	for (i = 0; i < VHOST_BLK_VQ_MAX; i++)
+		flush_work(&blk->vqs[i].drop_work);
+
 	for (i = 0; i < VHOST_BLK_VQ_MAX; i++) {
 		kvfree(blk->vqs[i].req);
 		blk->vqs[i].req = NULL;
@@ -803,8 +831,10 @@ static int vhost_blk_open(struct inode *inode, struct file *file)
 		       VHOST_DEV_WEIGHT, VHOST_DEV_PKT_WEIGHT, true, NULL);
 	file->private_data = blk;
 
-	for (i = 0; i < VHOST_BLK_VQ_MAX; i++)
+	for (i = 0; i < VHOST_BLK_VQ_MAX; i++) {
 		vhost_work_init(&blk->vqs[i].work, vhost_blk_handle_host_kick);
+		INIT_WORK(&blk->vqs[i].drop_work, vhost_blk_drop_reqs);
+	}
 
 	return ret;
 out_blk:
-- 
2.53.0

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

^ permalink raw reply	[flat|nested] 8+ messages in thread

* [Devel] [PATCH hci-8.1 v3 2/2] drivers/vhost/blk: support VIRTIO_BLK_F_WRITE_ZEROES
  2026-11-17 10:09 [Devel] [PATCH hci-8.1 v3 0/2] drivers/vhost/blk: support VIRTIO_BLK_F_WRITE_ZEROES Alexander Ivanov
  2026-11-17 10:09 ` [Devel] [PATCH hci-8.1 v3 1/2] drivers/vhost/blk: fix a flush hang on requests completed after worker kill Alexander Ivanov
@ 2026-11-17 10:09 ` Alexander Ivanov
  2026-11-17 10:18   ` no-reply
                     ` (2 more replies)
  2026-11-17 13:50 ` [Devel] [PATCH hci-8.1 v3 0/2] " Konstantin Khorenko
  2 siblings, 3 replies; 8+ messages in thread
From: Alexander Ivanov @ 2026-11-17 10:09 UTC (permalink / raw)
  To: andrey.zhadchenko; +Cc: devel

Handle VIRTIO_BLK_T_WRITE_ZEROES for parity with virtio-blk. The range
is zeroed with __blkdev_issue_zeroout(), which uses WRITE ZEROES of the
backend or falls back to writing zero pages. The UNMAP flag maps to
the absence of BLKDEV_ZERO_NOUNMAP. The range is issued in chunks of
at most 16 WRITE ZEROES bios and 1 GiB, or 4 MiB ones when zero pages
are written, since each chunk is built in the vhost worker.

The VMM must advertise blk_size equal to the backend logical block
size, max_write_zeroes_sectors aligned to it and at most 8388607, and
max_write_zeroes_seg = 1. A request is zeroed one chunk at a time and
holds vhost_blk_flush() until its last chunk, so a small
max_write_zeroes_sectors keeps vhost ioctls short on backends without
WRITE ZEROES. Our QEMU does not advertise the feature yet.

The guest gets VIRTIO_BLK_S_UNSUPP when VIRTIO_BLK_F_WRITE_ZEROES was
not negotiated, for a payload other than exactly one segment or an
unknown flag, and VIRTIO_BLK_S_IOERR for a read-only backend and for a
range that is unaligned, beyond the capacity or above 8388607 sectors.

https://virtuozzo.atlassian.net/browse/VSTOR-141684
Feature: vhost-blk: in-kernel accelerator for virtio-blk guests
Signed-off-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
---
 drivers/vhost/blk.c | 178 +++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 176 insertions(+), 2 deletions(-)

diff --git a/drivers/vhost/blk.c b/drivers/vhost/blk.c
index eea8cca59cac..e27320651469 100644
--- a/drivers/vhost/blk.c
+++ b/drivers/vhost/blk.c
@@ -29,7 +29,8 @@ enum {
 			     (1ULL << VIRTIO_RING_F_INDIRECT_DESC) |
 			     (1ULL << VIRTIO_RING_F_EVENT_IDX) |
 			     (1ULL << VIRTIO_BLK_F_MQ) |
-			     (1ULL << VIRTIO_BLK_F_FLUSH),
+			     (1ULL << VIRTIO_BLK_F_FLUSH) |
+			     (1ULL << VIRTIO_BLK_F_WRITE_ZEROES),
 };
 
 /*
@@ -75,6 +76,10 @@ struct vhost_blk_req {
 	struct iovec status[VHOST_MAX_METADATA_IOV];
 
 	sector_t sector;
+	sector_t nr_sects;
+	sector_t chunk_sects;
+	struct block_device *bdev;
+	unsigned int zeroout_flags;
 	int bi_opf;
 	u16 head;
 	size_t len;
@@ -271,6 +276,63 @@ static int vhost_blk_move_bb_to_req(struct vhost_blk_req *req)
 	return 0;
 }
 
+/*
+ * The range is zeroed in the vhost worker with vq->mutex held, so it is
+ * split into chunks of a few bios each, much smaller ones when zero pages
+ * are written. The chunk sizes are multiples of any logical block size.
+ */
+#define VHOST_BLK_MAX_WRITE_ZEROES_BIOS		16
+#define VHOST_BLK_MAX_WRITE_ZEROES_SECTORS	(SZ_1G >> SECTOR_SHIFT)
+#define VHOST_BLK_MAX_ZERO_PAGES_SECTORS	(SZ_4M >> SECTOR_SHIFT)
+
+/*
+ * A request holds vhost_blk_flush() until its last chunk completes, so the
+ * whole range is bounded too. No Linux guest sends more, a bio cannot carry
+ * it.
+ */
+#define VHOST_BLK_MAX_WRITE_ZEROES_REQ_SECTORS	(UINT_MAX >> SECTOR_SHIFT)
+
+static int vhost_blk_bio_make_zeroes(struct vhost_blk_req *req,
+				     struct block_device *bdev)
+{
+	sector_t bs_mask = (bdev_logical_block_size(bdev) >> SECTOR_SHIFT) - 1;
+	sector_t max_sects = (sector_t)bdev_write_zeroes_sectors(bdev) *
+			     VHOST_BLK_MAX_WRITE_ZEROES_BIOS & ~bs_mask;
+	struct bio *bio = NULL;
+	int ret = -EOPNOTSUPP;
+
+	req->chunk_sects = min3(req->nr_sects, max_sects,
+				(sector_t)VHOST_BLK_MAX_WRITE_ZEROES_SECTORS);
+	/* The limit may appear after it was read, and no bio is built then */
+	if (req->chunk_sects)
+		ret = __blkdev_issue_zeroout(bdev, req->sector,
+					     req->chunk_sects, GFP_KERNEL, &bio,
+					     req->zeroout_flags |
+					     BLKDEV_ZERO_NOFALLBACK);
+	if (ret == -EOPNOTSUPP) {
+		req->chunk_sects = min_t(sector_t, req->nr_sects,
+					 VHOST_BLK_MAX_ZERO_PAGES_SECTORS);
+		ret = __blkdev_issue_zeroout(bdev, req->sector,
+					     req->chunk_sects, GFP_KERNEL, &bio,
+					     req->zeroout_flags);
+	}
+	if (ret)
+		return ret;
+
+	bio->bi_private = req;
+	bio->bi_end_io  = vhost_blk_req_done;
+	req->bio = req->inline_bio;
+	req->bio[0] = bio;
+
+	/* Zero pages are plain writes, a failure of them is not retried */
+	if (bio_op(bio) != REQ_OP_WRITE_ZEROES)
+		req->bi_opf = REQ_OP_WRITE;
+
+	atomic_set(&req->bio_nr, 1);
+
+	return 0;
+}
+
 static int vhost_blk_bio_make(struct vhost_blk_req *req,
 			      struct block_device *bdev)
 {
@@ -284,6 +346,9 @@ static int vhost_blk_bio_make(struct vhost_blk_req *req,
 	if (unlikely(req->bi_opf & REQ_PREFLUSH))
 		return vhost_blk_bio_make_simple(req, bdev);
 
+	if (req->bi_opf == REQ_OP_WRITE_ZEROES)
+		return vhost_blk_bio_make_zeroes(req, bdev);
+
 	/*
 	 * Requests with unaligned io vectors are not supported by underlined
 	 * block device, let's use an aligned intermediary "bounce buffer" to
@@ -451,6 +516,50 @@ static bool vhost_blk_backend_ro(struct file *f)
 	       bdev_read_only(I_BDEV(f->f_mapping->host));
 }
 
+static int vhost_blk_parse_write_zeroes(struct vhost_blk_req *req, u8 *status)
+{
+	struct file *f = vhost_vq_get_backend(&req->blk_vq->vq);
+	struct block_device *bdev = I_BDEV(f->f_mapping->host);
+	sector_t bs_mask = (bdev_logical_block_size(bdev) >> SECTOR_SHIFT) - 1;
+	struct virtio_blk_discard_write_zeroes seg;
+	struct iov_iter iter;
+	u32 flags;
+
+	if (!vhost_has_feature(&req->blk_vq->vq, VIRTIO_BLK_F_WRITE_ZEROES) ||
+	    req->len != sizeof(seg)) {
+		*status = VIRTIO_BLK_S_UNSUPP;
+		return 0;
+	}
+
+	iov_iter_init(&iter, ITER_SOURCE, req->iov, req->iov_nr, req->len);
+	if (copy_from_iter(&seg, sizeof(seg), &iter) != sizeof(seg))
+		return -EFAULT;
+
+	flags = le32_to_cpu(seg.flags);
+	if (flags & ~VIRTIO_BLK_WRITE_ZEROES_FLAG_UNMAP) {
+		*status = VIRTIO_BLK_S_UNSUPP;
+		return 0;
+	}
+
+	req->sector = le64_to_cpu(seg.sector);
+	req->nr_sects = le32_to_cpu(seg.num_sectors);
+	req->bdev = bdev;
+	req->zeroout_flags = flags & VIRTIO_BLK_WRITE_ZEROES_FLAG_UNMAP ?
+			     0 : BLKDEV_ZERO_NOUNMAP;
+
+	if (vhost_blk_backend_ro(f) ||
+	    (req->sector | req->nr_sects) & bs_mask ||
+	    req->nr_sects > VHOST_BLK_MAX_WRITE_ZEROES_REQ_SECTORS ||
+	    req->sector > bdev_nr_sectors(bdev) ||
+	    req->nr_sects > bdev_nr_sectors(bdev) - req->sector) {
+		*status = VIRTIO_BLK_S_IOERR;
+		return 0;
+	}
+
+	*status = VIRTIO_BLK_S_OK;
+	return 0;
+}
+
 static int vhost_blk_req_handle(struct vhost_virtqueue *vq,
 				struct virtio_blk_outhdr *hdr,
 				u16 head, u16 out, u16 in)
@@ -475,6 +584,7 @@ static int vhost_blk_req_handle(struct vhost_virtqueue *vq,
 	req->head	= head;
 	req->blk	= blk;
 	req->sector	= vhost64_to_cpu(vq, hdr->sector);
+	req->nr_sects	= 0;
 	req->iov	= blk_vq->iov;
 	req->bio_err	= 0;
 	in_len		= iov_length(vq->iov + out, in);
@@ -490,7 +600,7 @@ static int vhost_blk_req_handle(struct vhost_virtqueue *vq,
 	req->status[0].iov_base = st->iov_base + st->iov_len;
 	req->status[0].iov_len = sizeof(status);
 
-	if (type == VIRTIO_BLK_T_OUT) {
+	if (type == VIRTIO_BLK_T_OUT || type == VIRTIO_BLK_T_WRITE_ZEROES) {
 		data_iov = vq->iov;
 		data_nr = out;
 	} else {
@@ -538,6 +648,22 @@ static int vhost_blk_req_handle(struct vhost_virtqueue *vq,
 		req->bi_opf = REQ_OP_WRITE | REQ_PREFLUSH;
 		ret = vhost_blk_req_submit(req);
 		break;
+	case VIRTIO_BLK_T_WRITE_ZEROES:
+		ret = vhost_blk_parse_write_zeroes(req, &status);
+		if (ret)
+			break;
+		if (status == VIRTIO_BLK_S_OK && req->nr_sects) {
+			req->bi_opf = REQ_OP_WRITE_ZEROES;
+			ret = vhost_blk_req_submit(req);
+			if (ret != -EPERM)
+				break;
+			status = VIRTIO_BLK_S_IOERR;
+		}
+		ret = vhost_blk_set_status(req, status);
+		if (ret)
+			break;
+		vhost_add_used_and_signal(&blk->dev, vq, head, sizeof(status));
+		break;
 	case VIRTIO_BLK_T_GET_ID:
 		len = min_t(size_t,
 			    strnlen(blk->serial, VIRTIO_BLK_ID_BYTES) + 1,
@@ -561,6 +687,51 @@ static int vhost_blk_req_handle(struct vhost_virtqueue *vq,
 	return ret;
 }
 
+/*
+ * Some devices clear their WRITE ZEROES limit only after a failed request,
+ * as blkdev_issue_write_zeroes() expects. Retry such a request with zero
+ * pages instead of failing it.
+ */
+static bool vhost_blk_retry_zeroes(struct vhost_blk_req *req)
+{
+	if (!req->bio_err || req->bi_opf != REQ_OP_WRITE_ZEROES)
+		return false;
+
+	if (bdev_write_zeroes_sectors(req->bdev))
+		return false;
+
+	/* The limit may come back before the bio is built, retry only once */
+	req->bi_opf = REQ_OP_WRITE;
+	if (vhost_blk_bio_make_zeroes(req, req->bdev))
+		return false;
+
+	req->bio_err = 0;
+	vhost_blk_bio_send(req);
+	return true;
+}
+
+/* Issue the next chunk of a write-zeroes range, if any is left */
+static bool vhost_blk_next_zeroes(struct vhost_blk_req *req)
+{
+	int ret;
+
+	if (req->bio_err || req->nr_sects <= req->chunk_sects)
+		return false;
+
+	req->sector += req->chunk_sects;
+	req->nr_sects -= req->chunk_sects;
+
+	req->bi_opf = REQ_OP_WRITE_ZEROES;
+	ret = vhost_blk_bio_make_zeroes(req, req->bdev);
+	if (ret) {
+		req->bio_err = ret;
+		return false;
+	}
+
+	vhost_blk_bio_send(req);
+	return true;
+}
+
 static void vhost_blk_handle_guest_kick(struct vhost_work *work)
 {
 	struct virtio_blk_outhdr hdr;
@@ -661,6 +832,9 @@ static void vhost_blk_handle_host_kick(struct vhost_work *work)
 		if (!blk)
 			blk = req->blk;
 
+		if (vhost_blk_retry_zeroes(req) || vhost_blk_next_zeroes(req))
+			continue;
+
 		if (!req->bio_err && req->bb && req->bi_opf == REQ_OP_READ) {
 			if (vhost_blk_move_bb_to_req(req))
 				req->bio_err = EINVAL;
-- 
2.53.0

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [Devel] [PATCH hci-8.1 v3 1/2] drivers/vhost/blk: fix a flush hang on requests completed after worker kill
  2026-11-17 10:09 ` [Devel] [PATCH hci-8.1 v3 1/2] drivers/vhost/blk: fix a flush hang on requests completed after worker kill Alexander Ivanov
@ 2026-11-17 10:14   ` no-reply
  0 siblings, 0 replies; 8+ messages in thread
From: no-reply @ 2026-11-17 10:14 UTC (permalink / raw)
  To: Alexander Ivanov, andrey.zhadchenko; +Cc: devel

> With vhost_task workers (not the default kthread ones), killing the
> owner makes vhost_worker_killed() detach the worker from the virtqueues
> and run only the work already queued. A request that completes after
> that is added to blk_vq->llhead, but vhost_vq_work_queue() returns
> false and queues nothing, so the request is never forgotten.
> req_inflight[] stays elevated and vhost_blk_flush() hangs
> uninterruptibly in release when the fd is closed. A request in flight
> at the kill is enough.
> 
> Release such requests from a work item when the worker cannot be
> queued, as vhost-scsi does in vhost_scsi_drop_cmds(). The bio
> completion may run in interrupt context, where a request cannot be
> released: that takes vq->mutex, and flush_lock with plain spin_lock().
> The guest is not answered, its memory is no longer reachable.
> 
> Once the completion publishes the request, the request can be
> forgotten and vhost_blk_flush() can return while the completion is
> still queueing work into blk, which release then frees. Queue under
> rcu_read_lock() and wait for it with synchronize_rcu() in
> vhost_blk_drop_backend(), then flush the drop work before freeing the
> requests.
> 
> https://virtuozzo.atlassian.net/browse/VSTOR-141684
> 
> Fixes: 1629db092dd7 ("drivers/vhost: vhost-blk accelerator for virtio-blk guests")
> Feature: vhost-blk: in-kernel accelerator for virtio-blk guests
> Signed-off-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.vzint.dev/#/patchset/20261117100904.3071306-1-alexander.ivanov@virtuozzo.com?part=1


_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [Devel] [PATCH hci-8.1 v3 2/2] drivers/vhost/blk: support VIRTIO_BLK_F_WRITE_ZEROES
  2026-11-17 10:09 ` [Devel] [PATCH hci-8.1 v3 2/2] drivers/vhost/blk: support VIRTIO_BLK_F_WRITE_ZEROES Alexander Ivanov
@ 2026-11-17 10:18   ` no-reply
  2026-11-17 11:32   ` Andrey Zhadchenko
  2026-11-17 13:50   ` Konstantin Khorenko
  2 siblings, 0 replies; 8+ messages in thread
From: no-reply @ 2026-11-17 10:18 UTC (permalink / raw)
  To: Alexander Ivanov, andrey.zhadchenko; +Cc: devel

> Handle VIRTIO_BLK_T_WRITE_ZEROES for parity with virtio-blk. The range
> is zeroed with __blkdev_issue_zeroout(), which uses WRITE ZEROES of the
> backend or falls back to writing zero pages. The UNMAP flag maps to
> the absence of BLKDEV_ZERO_NOUNMAP. The range is issued in chunks of
> at most 16 WRITE ZEROES bios and 1 GiB, or 4 MiB ones when zero pages
> are written, since each chunk is built in the vhost worker.
> 
> The VMM must advertise blk_size equal to the backend logical block
> size, max_write_zeroes_sectors aligned to it and at most 8388607, and
> max_write_zeroes_seg = 1. A request is zeroed one chunk at a time and
> holds vhost_blk_flush() until its last chunk, so a small
> max_write_zeroes_sectors keeps vhost ioctls short on backends without
> WRITE ZEROES. Our QEMU does not advertise the feature yet.
> 
> The guest gets VIRTIO_BLK_S_UNSUPP when VIRTIO_BLK_F_WRITE_ZEROES was
> not negotiated, for a payload other than exactly one segment or an
> unknown flag, and VIRTIO_BLK_S_IOERR for a read-only backend and for a
> range that is unaligned, beyond the capacity or above 8388607 sectors.
> 
> https://virtuozzo.atlassian.net/browse/VSTOR-141684
> Feature: vhost-blk: in-kernel accelerator for virtio-blk guests
> Signed-off-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>

Sashiko has reviewed this patch and found no issues.

-- 
Sashiko AI review · https://sashiko.vzint.dev/#/patchset/20261117100904.3071306-1-alexander.ivanov@virtuozzo.com?part=2


_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [Devel] [PATCH hci-8.1 v3 2/2] drivers/vhost/blk: support VIRTIO_BLK_F_WRITE_ZEROES
  2026-11-17 10:09 ` [Devel] [PATCH hci-8.1 v3 2/2] drivers/vhost/blk: support VIRTIO_BLK_F_WRITE_ZEROES Alexander Ivanov
  2026-11-17 10:18   ` no-reply
@ 2026-11-17 11:32   ` Andrey Zhadchenko
  2026-11-17 13:50   ` Konstantin Khorenko
  2 siblings, 0 replies; 8+ messages in thread
From: Andrey Zhadchenko @ 2026-11-17 11:32 UTC (permalink / raw)
  To: Alexander Ivanov; +Cc: devel

Generally speaking I think this is a wrong approach for our needs. This 
patch really overengineers the whole situation:
  - we should not build write requests if the device does not support 
zeroing. The correct way is to never advertise zeroes for such device. 
We can't do that in kernel because feature negotiation happens before 
backend is set. So we should check the limits in QEMU (which also should 
correctly set max_write_zeroes_sectors in config anyway, so the plumbing 
will be there)
  - vhost_blk_retry_zeroes() is probably nice but again not our case. 
This is a workaround when the device resets limits. We plan to use it 
with dm-qcow2 and we can just never reset the limit there (why would we
if writing zeroes is just a few bits in cluster tables)

So the whole patch shape should be
  - When we get zero request we just construct a simple bio with 
REQ_OP_WRITE_ZEROES and queue it. (maybe a little check to match 
bdev_write_zeroes_sectors)

That's it. Let's not add some convoluted code we will never use.
So first I would expect a QEMU patch that reads device zero support and 
sets max_write_zeroes_sectors limit. Then the simplified version of this 
patch.

On 11/17/26 11:09, Alexander Ivanov wrote:
> Handle VIRTIO_BLK_T_WRITE_ZEROES for parity with virtio-blk. The range
> is zeroed with __blkdev_issue_zeroout(), which uses WRITE ZEROES of the
> backend or falls back to writing zero pages. The UNMAP flag maps to
> the absence of BLKDEV_ZERO_NOUNMAP. The range is issued in chunks of
> at most 16 WRITE ZEROES bios and 1 GiB, or 4 MiB ones when zero pages
> are written, since each chunk is built in the vhost worker.
> 
> The VMM must advertise blk_size equal to the backend logical block
> size, max_write_zeroes_sectors aligned to it and at most 8388607, and
> max_write_zeroes_seg = 1. A request is zeroed one chunk at a time and
> holds vhost_blk_flush() until its last chunk, so a small
> max_write_zeroes_sectors keeps vhost ioctls short on backends without
> WRITE ZEROES. Our QEMU does not advertise the feature yet.
> 
> The guest gets VIRTIO_BLK_S_UNSUPP when VIRTIO_BLK_F_WRITE_ZEROES was
> not negotiated, for a payload other than exactly one segment or an
> unknown flag, and VIRTIO_BLK_S_IOERR for a read-only backend and for a
> range that is unaligned, beyond the capacity or above 8388607 sectors.
> 
> https://virtuozzo.atlassian.net/browse/VSTOR-141684
> Feature: vhost-blk: in-kernel accelerator for virtio-blk guests
> Signed-off-by: Alexander Ivanov <alexander.ivanov@virtuozzo.com>
> ---
>   drivers/vhost/blk.c | 178 +++++++++++++++++++++++++++++++++++++++++++-
>   1 file changed, 176 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/vhost/blk.c b/drivers/vhost/blk.c
> index eea8cca59cac..e27320651469 100644
> --- a/drivers/vhost/blk.c
> +++ b/drivers/vhost/blk.c
> @@ -29,7 +29,8 @@ enum {
>   			     (1ULL << VIRTIO_RING_F_INDIRECT_DESC) |
>   			     (1ULL << VIRTIO_RING_F_EVENT_IDX) |
>   			     (1ULL << VIRTIO_BLK_F_MQ) |
> -			     (1ULL << VIRTIO_BLK_F_FLUSH),
> +			     (1ULL << VIRTIO_BLK_F_FLUSH) |
> +			     (1ULL << VIRTIO_BLK_F_WRITE_ZEROES),
>   };
>   
>   /*
> @@ -75,6 +76,10 @@ struct vhost_blk_req {
>   	struct iovec status[VHOST_MAX_METADATA_IOV];
>   
>   	sector_t sector;
> +	sector_t nr_sects;
> +	sector_t chunk_sects;
> +	struct block_device *bdev;
Do we really need bdev? As far as I can see in the only place we use it, 
it can be derived from req->blk->backend.
In general we allocate a lot of vhost_blk_req beforehand and I would 
prefer to keeps this struct as tight as possible.
> +	unsigned int zeroout_flags;
>   	int bi_opf;
>   	u16 head;
>   	size_t len;
> @@ -271,6 +276,63 @@ static int vhost_blk_move_bb_to_req(struct vhost_blk_req *req)
>   	return 0;
>   }
>   
> +/*
> + * The range is zeroed in the vhost worker with vq->mutex held, so it is
> + * split into chunks of a few bios each, much smaller ones when zero pages
> + * are written. The chunk sizes are multiples of any logical block size.
> + */
> +#define VHOST_BLK_MAX_WRITE_ZEROES_BIOS		16
> +#define VHOST_BLK_MAX_WRITE_ZEROES_SECTORS	(SZ_1G >> SECTOR_SHIFT)
> +#define VHOST_BLK_MAX_ZERO_PAGES_SECTORS	(SZ_4M >> SECTOR_SHIFT)
> +
> +/*
> + * A request holds vhost_blk_flush() until its last chunk completes, so the
> + * whole range is bounded too. No Linux guest sends more, a bio cannot carry
> + * it.
> + */
> +#define VHOST_BLK_MAX_WRITE_ZEROES_REQ_SECTORS	(UINT_MAX >> SECTOR_SHIFT)
> +
> +static int vhost_blk_bio_make_zeroes(struct vhost_blk_req *req,
> +				     struct block_device *bdev)
> +{
> +	sector_t bs_mask = (bdev_logical_block_size(bdev) >> SECTOR_SHIFT) - 1;
> +	sector_t max_sects = (sector_t)bdev_write_zeroes_sectors(bdev) *
> +			     VHOST_BLK_MAX_WRITE_ZEROES_BIOS & ~bs_mask;
> +	struct bio *bio = NULL;
> +	int ret = -EOPNOTSUPP;
> +
> +	req->chunk_sects = min3(req->nr_sects, max_sects,
> +				(sector_t)VHOST_BLK_MAX_WRITE_ZEROES_SECTORS);
> +	/* The limit may appear after it was read, and no bio is built then */
> +	if (req->chunk_sects)
> +		ret = __blkdev_issue_zeroout(bdev, req->sector,
> +					     req->chunk_sects, GFP_KERNEL, &bio,
> +					     req->zeroout_flags |
> +					     BLKDEV_ZERO_NOFALLBACK);
> +	if (ret == -EOPNOTSUPP) {
> +		req->chunk_sects = min_t(sector_t, req->nr_sects,
> +					 VHOST_BLK_MAX_ZERO_PAGES_SECTORS);
> +		ret = __blkdev_issue_zeroout(bdev, req->sector,
> +					     req->chunk_sects, GFP_KERNEL, &bio,
> +					     req->zeroout_flags);
> +	}
> +	if (ret)
> +		return ret;
> +
> +	bio->bi_private = req;
> +	bio->bi_end_io  = vhost_blk_req_done;
> +	req->bio = req->inline_bio;
> +	req->bio[0] = bio;
> +
> +	/* Zero pages are plain writes, a failure of them is not retried */
> +	if (bio_op(bio) != REQ_OP_WRITE_ZEROES)
> +		req->bi_opf = REQ_OP_WRITE;
> +
> +	atomic_set(&req->bio_nr, 1);
> +
> +	return 0;
> +}
> +
>   static int vhost_blk_bio_make(struct vhost_blk_req *req,
>   			      struct block_device *bdev)
>   {
> @@ -284,6 +346,9 @@ static int vhost_blk_bio_make(struct vhost_blk_req *req,
>   	if (unlikely(req->bi_opf & REQ_PREFLUSH))
>   		return vhost_blk_bio_make_simple(req, bdev);
>   
> +	if (req->bi_opf == REQ_OP_WRITE_ZEROES)
> +		return vhost_blk_bio_make_zeroes(req, bdev);
> +
>   	/*
>   	 * Requests with unaligned io vectors are not supported by underlined
>   	 * block device, let's use an aligned intermediary "bounce buffer" to
> @@ -451,6 +516,50 @@ static bool vhost_blk_backend_ro(struct file *f)
>   	       bdev_read_only(I_BDEV(f->f_mapping->host));
>   }
>   
> +static int vhost_blk_parse_write_zeroes(struct vhost_blk_req *req, u8 *status)
> +{
> +	struct file *f = vhost_vq_get_backend(&req->blk_vq->vq);
> +	struct block_device *bdev = I_BDEV(f->f_mapping->host);
> +	sector_t bs_mask = (bdev_logical_block_size(bdev) >> SECTOR_SHIFT) - 1;
> +	struct virtio_blk_discard_write_zeroes seg;
> +	struct iov_iter iter;
> +	u32 flags;
> +
> +	if (!vhost_has_feature(&req->blk_vq->vq, VIRTIO_BLK_F_WRITE_ZEROES) ||
> +	    req->len != sizeof(seg)) {
> +		*status = VIRTIO_BLK_S_UNSUPP;
> +		return 0;
> +	}
> +
> +	iov_iter_init(&iter, ITER_SOURCE, req->iov, req->iov_nr, req->len);
> +	if (copy_from_iter(&seg, sizeof(seg), &iter) != sizeof(seg))
> +		return -EFAULT;
> +
> +	flags = le32_to_cpu(seg.flags);
> +	if (flags & ~VIRTIO_BLK_WRITE_ZEROES_FLAG_UNMAP) {
> +		*status = VIRTIO_BLK_S_UNSUPP;
> +		return 0;
> +	}
> +
> +	req->sector = le64_to_cpu(seg.sector);
> +	req->nr_sects = le32_to_cpu(seg.num_sectors);
> +	req->bdev = bdev;
> +	req->zeroout_flags = flags & VIRTIO_BLK_WRITE_ZEROES_FLAG_UNMAP ?
> +			     0 : BLKDEV_ZERO_NOUNMAP;
> +
> +	if (vhost_blk_backend_ro(f) ||
> +	    (req->sector | req->nr_sects) & bs_mask ||
> +	    req->nr_sects > VHOST_BLK_MAX_WRITE_ZEROES_REQ_SECTORS ||
> +	    req->sector > bdev_nr_sectors(bdev) ||
> +	    req->nr_sects > bdev_nr_sectors(bdev) - req->sector) {
> +		*status = VIRTIO_BLK_S_IOERR;
> +		return 0;
> +	}
> +
> +	*status = VIRTIO_BLK_S_OK;
> +	return 0;
> +}
> +
>   static int vhost_blk_req_handle(struct vhost_virtqueue *vq,
>   				struct virtio_blk_outhdr *hdr,
>   				u16 head, u16 out, u16 in)
> @@ -475,6 +584,7 @@ static int vhost_blk_req_handle(struct vhost_virtqueue *vq,
>   	req->head	= head;
>   	req->blk	= blk;
>   	req->sector	= vhost64_to_cpu(vq, hdr->sector);
> +	req->nr_sects	= 0;
>   	req->iov	= blk_vq->iov;
>   	req->bio_err	= 0;
>   	in_len		= iov_length(vq->iov + out, in);
> @@ -490,7 +600,7 @@ static int vhost_blk_req_handle(struct vhost_virtqueue *vq,
>   	req->status[0].iov_base = st->iov_base + st->iov_len;
>   	req->status[0].iov_len = sizeof(status);
>   
> -	if (type == VIRTIO_BLK_T_OUT) {
> +	if (type == VIRTIO_BLK_T_OUT || type == VIRTIO_BLK_T_WRITE_ZEROES) {
>   		data_iov = vq->iov;
>   		data_nr = out;
>   	} else {
> @@ -538,6 +648,22 @@ static int vhost_blk_req_handle(struct vhost_virtqueue *vq,
>   		req->bi_opf = REQ_OP_WRITE | REQ_PREFLUSH;
>   		ret = vhost_blk_req_submit(req);
>   		break;
> +	case VIRTIO_BLK_T_WRITE_ZEROES:
> +		ret = vhost_blk_parse_write_zeroes(req, &status);
> +		if (ret)
> +			break;
> +		if (status == VIRTIO_BLK_S_OK && req->nr_sects) {
> +			req->bi_opf = REQ_OP_WRITE_ZEROES;
> +			ret = vhost_blk_req_submit(req);
> +			if (ret != -EPERM)
> +				break;
> +			status = VIRTIO_BLK_S_IOERR;
> +		}
> +		ret = vhost_blk_set_status(req, status);
> +		if (ret)
> +			break;
> +		vhost_add_used_and_signal(&blk->dev, vq, head, sizeof(status));
> +		break;
>   	case VIRTIO_BLK_T_GET_ID:
>   		len = min_t(size_t,
>   			    strnlen(blk->serial, VIRTIO_BLK_ID_BYTES) + 1,
> @@ -561,6 +687,51 @@ static int vhost_blk_req_handle(struct vhost_virtqueue *vq,
>   	return ret;
>   }
>   
> +/*
> + * Some devices clear their WRITE ZEROES limit only after a failed request,
> + * as blkdev_issue_write_zeroes() expects. Retry such a request with zero
> + * pages instead of failing it.
> + */
> +static bool vhost_blk_retry_zeroes(struct vhost_blk_req *req)
> +{
> +	if (!req->bio_err || req->bi_opf != REQ_OP_WRITE_ZEROES)
> +		return false;
> +
> +	if (bdev_write_zeroes_sectors(req->bdev))
> +		return false;
> +
> +	/* The limit may come back before the bio is built, retry only once */
> +	req->bi_opf = REQ_OP_WRITE;
> +	if (vhost_blk_bio_make_zeroes(req, req->bdev))
> +		return false;
> +
> +	req->bio_err = 0;
> +	vhost_blk_bio_send(req);
> +	return true;
> +}
> +
> +/* Issue the next chunk of a write-zeroes range, if any is left */
> +static bool vhost_blk_next_zeroes(struct vhost_blk_req *req)
> +{
> +	int ret;
> +
> +	if (req->bio_err || req->nr_sects <= req->chunk_sects)
> +		return false;
> +
> +	req->sector += req->chunk_sects;
> +	req->nr_sects -= req->chunk_sects;
> +
> +	req->bi_opf = REQ_OP_WRITE_ZEROES;
> +	ret = vhost_blk_bio_make_zeroes(req, req->bdev);
> +	if (ret) {
> +		req->bio_err = ret;
> +		return false;
> +	}
> +
> +	vhost_blk_bio_send(req);
> +	return true;
> +}
> +
>   static void vhost_blk_handle_guest_kick(struct vhost_work *work)
>   {
>   	struct virtio_blk_outhdr hdr;
> @@ -661,6 +832,9 @@ static void vhost_blk_handle_host_kick(struct vhost_work *work)
>   		if (!blk)
>   			blk = req->blk;
>   
> +		if (vhost_blk_retry_zeroes(req) || vhost_blk_next_zeroes(req))
> +			continue;
> +
>   		if (!req->bio_err && req->bb && req->bi_opf == REQ_OP_READ) {
>   			if (vhost_blk_move_bb_to_req(req))
>   				req->bio_err = EINVAL;

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [Devel] [PATCH hci-8.1 v3 0/2] drivers/vhost/blk: support VIRTIO_BLK_F_WRITE_ZEROES
  2026-11-17 10:09 [Devel] [PATCH hci-8.1 v3 0/2] drivers/vhost/blk: support VIRTIO_BLK_F_WRITE_ZEROES Alexander Ivanov
  2026-11-17 10:09 ` [Devel] [PATCH hci-8.1 v3 1/2] drivers/vhost/blk: fix a flush hang on requests completed after worker kill Alexander Ivanov
  2026-11-17 10:09 ` [Devel] [PATCH hci-8.1 v3 2/2] drivers/vhost/blk: support VIRTIO_BLK_F_WRITE_ZEROES Alexander Ivanov
@ 2026-11-17 13:50 ` Konstantin Khorenko
  2 siblings, 0 replies; 8+ messages in thread
From: Konstantin Khorenko @ 2026-11-17 13:50 UTC (permalink / raw)
  To: Alexander Ivanov; +Cc: devel

I won't apply this version, please send v4 with the changes below.

1. Patch 2: take the shape Andrey asked for in his reply to 2/2: one
   REQ_OP_WRITE_ZEROES bio per request, no zero-page fallback, no
   chunks, no vhost_blk_retry_zeroes().
2. Patch 2: keep the request checks: the feature negotiated, exactly
   one segment, known flags only, alignment to the logical block size,
   a read-only backend, the range inside the device.
3. Patch 2: keep the cap at UINT_MAX >> SECTOR_SHIFT and also reject a
   range above bdev_write_zeroes_sectors() with VIRTIO_BLK_S_IOERR.
4. Patch 2: a backend without WRITE ZEROES (limit 0) gets
   VIRTIO_BLK_S_IOERR, not a fallback.
5. Patch 2 message: say that the VMM advertises the feature only for a
   backend with WRITE ZEROES and sets max_write_zeroes_sectors from it.
6. Patch 1 stays as it is.

On Tue, 17 Nov 2026 11:09:02 +0100, Alexander Ivanov wrote:
> Alexander Ivanov (2):
>   drivers/vhost/blk: fix a flush hang on requests completed after worker
>     kill
>   drivers/vhost/blk: support VIRTIO_BLK_F_WRITE_ZEROES

1. I agree with Andrey. Once the VMM advertises the feature only for a
backend that has WRITE ZEROES, the zero-page path and the retry are
never taken. My v1 ask for small zero-page chunks goes away with them.

3. num_sectors is 32 bits, so a guest can ask for almost 2 TiB, while
bi_size of one bio is an unsigned int in bytes. This check is needed,
not optional: without it such a request wraps bi_size.

4. A Linux guest then gets -EIO from BLKZEROOUT and does not fall back
to writing zeroes itself. So the VMM has to keep the condition true
when it sets another backend after feature negotiation.

Note on dm-qcow2: it has no REQ_OP_WRITE_ZEROES today. It does not set
num_write_zeroes_bios, so dm sets max_write_zeroes_sectors to 0 and the
VMM will not advertise the feature on it. The feature starts working
on dm-qcow2 only once dm-qcow2 handles WRITE ZEROES.

Thanks.

-- 
Konstantin Khorenko,
Virtuozzo Linux Kernel Team

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [Devel] [PATCH hci-8.1 v3 2/2] drivers/vhost/blk: support VIRTIO_BLK_F_WRITE_ZEROES
  2026-11-17 10:09 ` [Devel] [PATCH hci-8.1 v3 2/2] drivers/vhost/blk: support VIRTIO_BLK_F_WRITE_ZEROES Alexander Ivanov
  2026-11-17 10:18   ` no-reply
  2026-11-17 11:32   ` Andrey Zhadchenko
@ 2026-11-17 13:50   ` Konstantin Khorenko
  2 siblings, 0 replies; 8+ messages in thread
From: Konstantin Khorenko @ 2026-11-17 13:50 UTC (permalink / raw)
  To: Alexander Ivanov; +Cc: devel

I won't apply this version, please send v4 with the changes below.

On Tue, 17 Nov 2026 11:09:04 +0100, Alexander Ivanov wrote:
> +	req->nr_sects = le32_to_cpu(seg.num_sectors);

Items 1-5 of the cover letter reply: a single REQ_OP_WRITE_ZEROES bio,
as Andrey asked, with the request checks kept and the range also
limited by bdev_write_zeroes_sectors().

-- 
Konstantin Khorenko,
Virtuozzo Linux Kernel Team

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2026-11-17 13:52 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-17 10:09 [Devel] [PATCH hci-8.1 v3 0/2] drivers/vhost/blk: support VIRTIO_BLK_F_WRITE_ZEROES Alexander Ivanov
2026-11-17 10:09 ` [Devel] [PATCH hci-8.1 v3 1/2] drivers/vhost/blk: fix a flush hang on requests completed after worker kill Alexander Ivanov
2026-11-17 10:14   ` no-reply
2026-11-17 10:09 ` [Devel] [PATCH hci-8.1 v3 2/2] drivers/vhost/blk: support VIRTIO_BLK_F_WRITE_ZEROES Alexander Ivanov
2026-11-17 10:18   ` no-reply
2026-11-17 11:32   ` Andrey Zhadchenko
2026-11-17 13:50   ` Konstantin Khorenko
2026-11-17 13:50 ` [Devel] [PATCH hci-8.1 v3 0/2] " Konstantin Khorenko

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox