From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.openvz.org (mail.openvz.org [69.168.225.77]) by lore.virtuozzo.com (Postfix) with ESMTPS id EF4338017D for ; Mon, 2 Nov 2026 11:32:15 +0000 (UTC) Received: from mail.openvz.org (localhost [127.0.0.1]) by mail.openvz.org (8.14.4/8.14.4) with ESMTP id 6A2BUvWR024192; Mon, 2 Nov 2026 14:30:58 +0300 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.openvz.org 6A2BUvWR024192 Authentication-Results: mail.openvz.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=virtuozzo.com header.i=@virtuozzo.com header.b="bnKjww9W" Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by mail.openvz.org (8.14.4/8.14.4) with ESMTP id 6A2BUuiX024188 (version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128 verify=FAIL) for ; Mon, 2 Nov 2026 14:30:56 +0300 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.openvz.org 6A2BUuiX024188 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-4a1d1f26a63so27988975e9.1 for ; Mon, 02 Nov 2026 03:30:56 -0800 (PST) X-Forwarded-Encrypted: i=2; AKwUvBwbpMCz+VZqe3AlSiOvYsLaiNy8u3Xr+sVW/hdd/kqlwQz9BoqqlLtSZCC46IEDW0vyfH9auA==@openvz.org X-Gm-Message-State: AFq9FYIUXsrMG54Cdw806hIy4WaSLIUyjAXlhma3K24/MAEoighl9YPX rgyCr36cwrbpHZlD3NI0ln2cbzxTC11FfFVSnR8qsRHRuZ67yH9nohr3/OKdiXbjMu6ubohbD2B XAuUqMMlk6GJDCO146oniASL6tjCseJvaGwBEoGBbvmu6OPO7/I78yQ== X-Gm-Gg: AYBFou0uLJ+NlJzZOyg1aNZcEX62Y2qT71KjEj0TqsqoUpYrDvenQI3uGr2vBTj+vc/ czHtJGOCXgaGjicny7DtiioEeXkr7u+Igej0BoUwiq1qppEJtpomQCVtGARSyI/cMZ6prfiVk98 4Y10Y4udJeFp1KSFagTMxdpQ7eX/IqCva/iEJeN/5T3rDljkqVkMJy0cJA+w2ezhitz89N/4Io0 OJ4qw/CQoNnSOxp/nZBNn6jpisy3pIAZws9sLi3ZqkKHRvjCDqvgMto+BLl0FY/CRLj8kFvyIuM 6aZvxypV3YTh7gNZXEiuXUt6qTzJgczYnNeaAmq6xaYHKIBI0Qx4KzKf5VYRFURgMUtcGWWbfm6 f72KdcijXLWCUgpz5aOcR4wYCLiA= X-Received: by 2002:a05:600c:1c06:b0:4a2:fe11:9a8a with SMTP id 5b1f17b1804b1-4ac121940c8mr289911895e9.0.1793619056032; Mon, 02 Nov 2026 03:30:56 -0800 (PST) X-Received: by 2002:a05:600c:1c06:b0:4a2:fe11:9a8a with SMTP id 5b1f17b1804b1-4ac121940c8mr289910995e9.0.1793619055372; Mon, 02 Nov 2026 03:30:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1793619055; cv=none; d=google.com; s=arc-20260327; b=Iq/BoVXBDvcLTENQJIUH9nVJ08nYeGB/ESDulfkRJM+NBtXvcg1JP96z0vhRMfvXk3 OkIIyukx9wEk4pwQPCcOHWcX9KqEsTrU5vJezZ3cMe8n90IX+KvR+HQg+td90Pfm9Y/f fcNZJF4fRv+PoW25OsiMsCisnK/+raARa6VaDDyB43CYGGq4Y5b2vi5aotx/Z0NksqTL 1FcyMHrDu0VHnZfyqALEXZPka8aK+ijKaKcAZbXX/qhtB2Wy81qAcsb8wJlJ5zf7NZeh HCWBTrh3nZu+7+qpNT9lNdXRO5YVdavZTYmbAsOTs19Su6DOXoRkgZHvJ5asxofYqUxR SJsQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:dkim-signature; bh=T6VvR9EjwLFj2wfDlMqU/TpPF/g973fDOrjyPlGD3lQ=; fh=CMILZ+goaghq6B4UpCDhf0EQp6kYD0LAlmOkUiVCOvs=; b=T0KNeZ/lNpEvbEva0GaGvlRBPylfa93MbFkucG+M204EEsA1yJwFLku3e07ciuz0vs tHY6rIUxuc5TXNsAqymSnYzuXYyl4I119OwDmYfVOV58b6iiJ6f72wNWMa8X2X4DfJ26 GALyVZA151H28GpsHIJ/9VzjWNNiRRYsEWjza6mI5S3dB/NKWovQQAF+Qv+JqOcweBiM DKPvSwQ1z2KfbguUt6+XvYoVeybF/FxvMvnkG6a9eoW936KKRPEqxYVVFy53FEBcfF1P Z6793Z0jCRCrLk8xv0pX6ybu0MfKzA+aRP1SI3hsGds2la/L8Yyv8jNddcPzM6lCKLs4 3rGw==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@virtuozzo.com header.s=relay header.b=bnKjww9W; spf=pass (google.com: domain of alexander.ivanov@virtuozzo.com designates 130.117.225.111 as permitted sender) smtp.mailfrom=alexander.ivanov@virtuozzo.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=virtuozzo.com Received: from relay.virtuozzo.com (relay.virtuozzo.com. [130.117.225.111]) by mx.google.com with ESMTPS id 5b1f17b1804b1-4aeb4cb6a19si158816985e9.0.2026.11.02.03.30.55 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 02 Nov 2026 03:30:55 -0800 (PST) Received-SPF: pass (google.com: domain of alexander.ivanov@virtuozzo.com designates 130.117.225.111 as permitted sender) client-ip=130.117.225.111; Authentication-Results: mx.google.com; dkim=pass header.i=@virtuozzo.com header.s=relay header.b=bnKjww9W; spf=pass (google.com: domain of alexander.ivanov@virtuozzo.com designates 130.117.225.111 as permitted sender) smtp.mailfrom=alexander.ivanov@virtuozzo.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=virtuozzo.com DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=virtuozzo.com; s=relay; h=MIME-Version:Message-ID:Date:Subject:From: Content-Type; bh=T6VvR9EjwLFj2wfDlMqU/TpPF/g973fDOrjyPlGD3lQ=; b=bnKjww9Wis7T 6jgwtAKd7A/imPCJivbgBqiBbPZG2qBU+yKgqD8hNviqoXtuivMSUoRl13JDXvABFTBWh/8bb/u63 glnQqVfZ+3YolyKdYDNtAZxAidPFlc7gjEiIpSrsC8LzjppFAWl4hNltFZvtCs+sX7w77L8sx6oCS sLJQ9d0Pg2nB7oFV9CjJGGCxvnauWTsOgP9HC8GcpJMEtRfZKTgIPKAWsq0YVkM+jOF4wdtLI/Okp MYCiDpe371E+Df+cu15bd5e2914FhFB/7dZhxh7kFM83xsafkIjL8GQpn8hHm8XIKttNsv7XB+qz/ 9LyXbPbA3riOaGItBCOKIw==; Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=sky) by relay.virtuozzo.com with esmtps (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1xNqBh-006x3x-0g; Mon, 02 Nov 2026 12:30:46 +0100 From: Alexander Ivanov To: andrey.zhadchenko@virtuozzo.com Date: Mon, 2 Nov 2026 12:30:51 +0100 Message-ID: <20261102113051.4190522-1-alexander.ivanov@virtuozzo.com> X-Mailer: git-send-email 2.53.0 MIME-Version: 1.0 X-OZ-Fwd: true Cc: devel@openvz.org Subject: [Devel] [PATCH hci-8.1] drivers/vhost/blk: support VIRTIO_BLK_F_WRITE_ZEROES X-BeenThere: devel@openvz.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OpenVZ development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: devel-bounces@openvz.org Errors-To: devel-bounces@openvz.org QEMU virtio-blk advertises WRITE_ZEROES, so a guest switched to vhost-blk may issue such requests and gets VIRTIO_BLK_S_UNSUPP. Handle VIRTIO_BLK_T_WRITE_ZEROES with __blkdev_issue_zeroout(), which splits the range by the device limit and falls back to writing zero pages when the backend has no WRITE ZEROES support. The UNMAP flag maps to the absence of BLKDEV_ZERO_NOUNMAP. Only a single segment per request is accepted, as in QEMU, so the device config must report max_write_zeroes_seg = 1. The segment is taken from the device-readable part of the chain, where the virtio spec places it. Complete a range not aligned to the logical block size and a request to a read-only backend with VIRTIO_BLK_S_IOERR, as QEMU does. The backend is read-only as for VIRTIO_BLK_T_OUT: the fd lacks FMODE_WRITE or the bdev is read-only. BLKROSET or set_disk_ro() may switch the bdev read-only after the check, so -EPERM of __blkdev_issue_zeroout() is completed with VIRTIO_BLK_S_IOERR too. An error returned from the submit path is reported as a vq error while the request never completes, so the guest would hang on it. Complete a range beyond the device capacity with VIRTIO_BLK_S_IOERR. The range is zeroed in the vhost worker with vq->mutex held, so issue it in 1 GiB chunks, each one when the previous completes. The size is not left to the VMM: a guest reads a zero max_write_zeroes_sectors as no limit. The request stays in flight until its last chunk, holding vhost_blk_flush() with dev.mutex taken, so a range larger than a bio can carry is completed with VIRTIO_BLK_S_IOERR, as QEMU does above max_write_zeroes_sectors. A device may drop WRITE ZEROES support only after a request fails, e.g. sd clears the limit on a failed WRITE SAME. Like blkdev_issue_write_zeroes(), retry such a request with zero pages when the limit has become zero instead of reporting VIRTIO_BLK_S_IOERR. https://virtuozzo.atlassian.net/browse/VSTOR-141684 Feature: vhost-blk: in-kernel accelerator for virtio-blk guests Signed-off-by: Alexander Ivanov --- drivers/vhost/blk.c | 160 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 158 insertions(+), 2 deletions(-) diff --git a/drivers/vhost/blk.c b/drivers/vhost/blk.c index 731dbf58fb69..4393fe0d84eb 100644 --- a/drivers/vhost/blk.c +++ b/drivers/vhost/blk.c @@ -29,7 +29,8 @@ enum { (1ULL << VIRTIO_RING_F_INDIRECT_DESC) | (1ULL << VIRTIO_RING_F_EVENT_IDX) | (1ULL << VIRTIO_BLK_F_MQ) | - (1ULL << VIRTIO_BLK_F_FLUSH), + (1ULL << VIRTIO_BLK_F_FLUSH) | + (1ULL << VIRTIO_BLK_F_WRITE_ZEROES), }; /* @@ -75,6 +76,9 @@ struct vhost_blk_req { struct iovec status[VHOST_MAX_METADATA_IOV]; sector_t sector; + sector_t nr_sects; + struct block_device *bdev; + unsigned int zeroout_flags; int bi_opf; u16 head; size_t len; @@ -264,6 +268,47 @@ static int vhost_blk_move_bb_to_req(struct vhost_blk_req *req) return 0; } +/* + * The range is zeroed in the vhost worker with vq->mutex held, so it is + * split into chunks bounded like a data request. The chunk size is a + * multiple of any logical block size. + */ +#define VHOST_BLK_MAX_WRITE_ZEROES_SECTORS (SZ_1G >> SECTOR_SHIFT) + +/* + * A request holds vhost_blk_flush() until its last chunk completes, so the + * whole range is bounded too. No Linux guest sends more, a bio cannot carry + * it. + */ +#define VHOST_BLK_MAX_WRITE_ZEROES_REQ_SECTORS (UINT_MAX >> SECTOR_SHIFT) + +static int vhost_blk_bio_make_zeroes(struct vhost_blk_req *req, + struct block_device *bdev) +{ + sector_t nr_sects = min_t(sector_t, req->nr_sects, + VHOST_BLK_MAX_WRITE_ZEROES_SECTORS); + struct bio *bio = NULL; + int ret; + + ret = __blkdev_issue_zeroout(bdev, req->sector, nr_sects, + GFP_KERNEL, &bio, req->zeroout_flags); + if (ret) + return ret; + + bio->bi_private = req; + bio->bi_end_io = vhost_blk_req_done; + req->bio = req->inline_bio; + req->bio[0] = bio; + + /* Zero pages are plain writes, a failure of them is not retried */ + if (bio_op(bio) != REQ_OP_WRITE_ZEROES) + req->bi_opf = REQ_OP_WRITE; + + atomic_set(&req->bio_nr, 1); + + return 0; +} + static int vhost_blk_bio_make(struct vhost_blk_req *req, struct block_device *bdev) { @@ -277,6 +322,9 @@ static int vhost_blk_bio_make(struct vhost_blk_req *req, if (unlikely(req->bi_opf & REQ_PREFLUSH)) return vhost_blk_bio_make_simple(req, bdev); + if (req->bi_opf == REQ_OP_WRITE_ZEROES) + return vhost_blk_bio_make_zeroes(req, bdev); + /* * Requests with unaligned io vectors are not supported by underlined * block device, let's use an aligned intermediary "bounce buffer" to @@ -443,6 +491,49 @@ static bool vhost_blk_backend_ro(struct file *f) bdev_read_only(I_BDEV(f->f_mapping->host)); } +static int vhost_blk_parse_write_zeroes(struct vhost_blk_req *req, u8 *status) +{ + struct file *f = vhost_vq_get_backend(&req->blk_vq->vq); + struct block_device *bdev = I_BDEV(f->f_mapping->host); + sector_t bs_mask = (bdev_logical_block_size(bdev) >> SECTOR_SHIFT) - 1; + struct virtio_blk_discard_write_zeroes seg; + struct iov_iter iter; + u32 flags; + + if (req->len != sizeof(seg)) { + *status = VIRTIO_BLK_S_UNSUPP; + return 0; + } + + iov_iter_init(&iter, ITER_SOURCE, req->iov, req->iov_nr, req->len); + if (copy_from_iter(&seg, sizeof(seg), &iter) != sizeof(seg)) + return -EFAULT; + + flags = le32_to_cpu(seg.flags); + if (flags & ~VIRTIO_BLK_WRITE_ZEROES_FLAG_UNMAP) { + *status = VIRTIO_BLK_S_UNSUPP; + return 0; + } + + req->sector = le64_to_cpu(seg.sector); + req->nr_sects = le32_to_cpu(seg.num_sectors); + req->bdev = bdev; + req->zeroout_flags = flags & VIRTIO_BLK_WRITE_ZEROES_FLAG_UNMAP ? + 0 : BLKDEV_ZERO_NOUNMAP; + + if (vhost_blk_backend_ro(f) || + (req->sector | req->nr_sects) & bs_mask || + req->nr_sects > VHOST_BLK_MAX_WRITE_ZEROES_REQ_SECTORS || + req->sector > bdev_nr_sectors(bdev) || + req->nr_sects > bdev_nr_sectors(bdev) - req->sector) { + *status = VIRTIO_BLK_S_IOERR; + return 0; + } + + *status = VIRTIO_BLK_S_OK; + return 0; +} + static int vhost_blk_req_handle(struct vhost_virtqueue *vq, struct virtio_blk_outhdr *hdr, u16 head, u16 out, u16 in) @@ -462,6 +553,7 @@ static int vhost_blk_req_handle(struct vhost_virtqueue *vq, req->head = head; req->blk = blk; req->sector = vhost64_to_cpu(vq, hdr->sector); + req->nr_sects = 0; req->iov = blk_vq->iov; req->bio_err = 0; in_len = iov_length(vq->iov + out, in); @@ -477,7 +569,7 @@ static int vhost_blk_req_handle(struct vhost_virtqueue *vq, req->status[0].iov_base = st->iov_base + st->iov_len; req->status[0].iov_len = sizeof(status); - if (type == VIRTIO_BLK_T_OUT) { + if (type == VIRTIO_BLK_T_OUT || type == VIRTIO_BLK_T_WRITE_ZEROES) { data_iov = vq->iov; data_nr = out; } else { @@ -525,6 +617,22 @@ static int vhost_blk_req_handle(struct vhost_virtqueue *vq, req->bi_opf = REQ_OP_WRITE | REQ_PREFLUSH; ret = vhost_blk_req_submit(req); break; + case VIRTIO_BLK_T_WRITE_ZEROES: + ret = vhost_blk_parse_write_zeroes(req, &status); + if (ret) + break; + if (status == VIRTIO_BLK_S_OK && req->nr_sects) { + req->bi_opf = REQ_OP_WRITE_ZEROES; + ret = vhost_blk_req_submit(req); + if (ret != -EPERM) + break; + status = VIRTIO_BLK_S_IOERR; + } + ret = vhost_blk_set_status(req, status); + if (ret) + break; + vhost_add_used_and_signal(&blk->dev, vq, head, sizeof(status)); + break; case VIRTIO_BLK_T_GET_ID: len = min_t(size_t, strnlen(blk->serial, VIRTIO_BLK_ID_BYTES) + 1, @@ -548,6 +656,51 @@ static int vhost_blk_req_handle(struct vhost_virtqueue *vq, return ret; } +/* + * Some devices clear their WRITE ZEROES limit only after a failed request, + * as blkdev_issue_write_zeroes() expects. Retry such a request with zero + * pages instead of failing it. + */ +static bool vhost_blk_retry_zeroes(struct vhost_blk_req *req) +{ + if (!req->bio_err || req->bi_opf != REQ_OP_WRITE_ZEROES) + return false; + + if (bdev_write_zeroes_sectors(req->bdev)) + return false; + + /* The limit may come back before the bio is built, retry only once */ + req->bi_opf = REQ_OP_WRITE; + if (vhost_blk_bio_make_zeroes(req, req->bdev)) + return false; + + req->bio_err = 0; + vhost_blk_bio_send(req); + return true; +} + +/* Issue the next chunk of a write-zeroes range, if any is left */ +static bool vhost_blk_next_zeroes(struct vhost_blk_req *req) +{ + int ret; + + if (req->bio_err || req->nr_sects <= VHOST_BLK_MAX_WRITE_ZEROES_SECTORS) + return false; + + req->sector += VHOST_BLK_MAX_WRITE_ZEROES_SECTORS; + req->nr_sects -= VHOST_BLK_MAX_WRITE_ZEROES_SECTORS; + + req->bi_opf = REQ_OP_WRITE_ZEROES; + ret = vhost_blk_bio_make_zeroes(req, req->bdev); + if (ret) { + req->bio_err = ret; + return false; + } + + vhost_blk_bio_send(req); + return true; +} + static void vhost_blk_handle_guest_kick(struct vhost_work *work) { struct virtio_blk_outhdr hdr; @@ -648,6 +801,9 @@ static void vhost_blk_handle_host_kick(struct vhost_work *work) if (!blk) blk = req->blk; + if (vhost_blk_retry_zeroes(req) || vhost_blk_next_zeroes(req)) + continue; + if (!req->bio_err && req->bb && req->bi_opf == REQ_OP_READ) { if (vhost_blk_move_bb_to_req(req)) req->bio_err = EINVAL; -- 2.53.0 _______________________________________________ Devel mailing list Devel@openvz.org https://lists.openvz.org/mailman/listinfo/devel