From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.openvz.org (mail.openvz.org [69.168.225.77]) by lore.virtuozzo.com (Postfix) with ESMTPS id 576AE817C6 for ; Tue, 17 Nov 2026 10:10:32 +0000 (UTC) Received: from mail.openvz.org (localhost [127.0.0.1]) by mail.openvz.org (8.14.4/8.14.4) with ESMTP id 6AHA9KT9007721; Tue, 17 Nov 2026 13:09:21 +0300 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.openvz.org 6AHA9KT9007721 Authentication-Results: mail.openvz.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=virtuozzo.com header.i=@virtuozzo.com header.b="sfCRzcIB" Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by mail.openvz.org (8.14.4/8.14.4) with ESMTP id 6AHA98Dm007699 (version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128 verify=FAIL) for ; Tue, 17 Nov 2026 13:09:09 +0300 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.openvz.org 6AHA98Dm007699 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-49d8ce126f6so2800668f8f.1 for ; Tue, 17 Nov 2026 02:09:08 -0800 (PST) X-Forwarded-Encrypted: i=2; AL+hErH15BW554kBWWLa7LJC6cleaz1DSNYc6C4pee8Tr4F35PscxkOMiRz8hotH5U6ZwWVdSKv/jA==@openvz.org X-Gm-Message-State: APuU4Z/ezVoptZyt9lENDuzYbI4ihB6YSMtk8tUDyV+jCAGvcpZOgng5 6gmUnqDKyVVF82Z9pqrWQ4FG3fNOFsWpzI6C6pSvsg1KEIk+BQ6iEdYah3kkR9HVH1vApMgqadM z1Jy4b0KoEaOQlV2hqFUmfcIX6BHHLrn3UFk0RR6GbzUZigsa8aSg1g== X-Gm-Gg: AZFdTZcYA9OieRBy39rMr5oHyGUDCAsfqjqVPSqY5yV0FKqpU80TufvRMXxNpwAcjny IOsRkFsHRlo3ricwD8rwGgOjRowyFwjKU/j6xT8okRtoYTf2EOBv/D+z8VjZ8IAiPZpqrHmd4Ju /Xjp1p1tQq7I/mAc34W13u0MulrhvJO6ZWkmCfxtjkJCiHS5+tv3uM6aol8Y7nxqeZpFHRyDqPD Cge2iVJYqjeAp6iGUo3dlMoQT1hrQ+jb4FJdzOrY040zBRUN4QdMu4LRutyvN4HvVf/g6XBVjkL Xr8NBo7lbKOJ/8gSZvtaN5mGCZTIbzzPARxvdL/Kf04BNpWagv9MGNeupWVDkItuM1Wi3E/OveF FZr7Kq/2hUiNlBIsOoI2fDai9VBqR X-Received: by 2002:a5d:59c9:0:b0:491:68e2:bf2f with SMTP id ffacd0b85a97d-49c72ba7408mr28128787f8f.52.1794910148385; Tue, 17 Nov 2026 02:09:08 -0800 (PST) X-Received: by 2002:a5d:59c9:0:b0:491:68e2:bf2f with SMTP id ffacd0b85a97d-49c72ba7408mr28128725f8f.52.1794910147875; Tue, 17 Nov 2026 02:09:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1794910147; cv=none; d=google.com; s=arc-20260327; b=rjDUp5UePPhS2lm4EH8mUJ5WKRew2er/kF4dsdvVKLa+ONbrKHz2yEfDD+hnK6GFmB VjMckAsANRohf2VbAI3ia0FV6jNHjip8h4dqC207bXRuh6P6qcSf8Mj5R6RTD0U/v3qj iJFSvsDbbzPwjGb2RRy1b5ZNBAklRY9cU/zhtU1P9jj1keCkfUGPQZKtTn/Nzm4fEUDC 3h6XUULQNKWrNK0yYy9SHdSkaS9MGG/xxF9NjWt5kEP42dVY+9eKRc/yIcY7B3/3ER9Q KlVIEDWucRi577RAccSuZhRMQWXi7zJKJGFD+jimSfxGaIYuuSPvQx+QceoTNAtgklCH GUBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=bNpwOe85uXbLcPLlJ/GGbc7ZFMJv7RcYMg4ZvqmsX8k=; fh=CMILZ+goaghq6B4UpCDhf0EQp6kYD0LAlmOkUiVCOvs=; b=famg9VuzBiRxzhrcIHebemwSVaDvJ47pN4u5EtpHNHDFWcH1HNsA2g15ZCgYwRJJTq vZvb1AHnya3ZIO9XZ69ubCysCkp3R06OZcrFyMqEcoEU3Jh7GwwRloEkYKU5T+SPn5Q2 2xWvZX+Onwp3u3SvnA3tKIA3yM9szt1nlRycF9gYdc6GKF+TJHdjqg1XL0/tBmiu/nP/ 4vqyVORuRO/Txf90DAzouGJ89GEDhyMIch8S3+AfNNrLEB6kYQiRanc1q3Akk0FbJ+kh jDHMvoOUzFCVqamWe5ei8Apyj/9+CWem/LkvQE21HCtgHUcn/K8JqyfDTKZDedaVnbIx JjpQ==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@virtuozzo.com header.s=relay header.b=sfCRzcIB; spf=pass (google.com: domain of alexander.ivanov@virtuozzo.com designates 130.117.225.111 as permitted sender) smtp.mailfrom=alexander.ivanov@virtuozzo.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=virtuozzo.com Received: from relay.virtuozzo.com (relay.virtuozzo.com. [130.117.225.111]) by mx.google.com with ESMTPS id ffacd0b85a97d-49c600213f9si25056926f8f.22.2026.11.17.02.09.07 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 17 Nov 2026 02:09:07 -0800 (PST) Received-SPF: pass (google.com: domain of alexander.ivanov@virtuozzo.com designates 130.117.225.111 as permitted sender) client-ip=130.117.225.111; Authentication-Results: mx.google.com; dkim=pass header.i=@virtuozzo.com header.s=relay header.b=sfCRzcIB; spf=pass (google.com: domain of alexander.ivanov@virtuozzo.com designates 130.117.225.111 as permitted sender) smtp.mailfrom=alexander.ivanov@virtuozzo.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=virtuozzo.com DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=virtuozzo.com; s=relay; h=MIME-Version:Message-ID:Date:Subject:From: Content-Type; bh=bNpwOe85uXbLcPLlJ/GGbc7ZFMJv7RcYMg4ZvqmsX8k=; b=sfCRzcIBRbKt 7FZqniK79T4/MNlGhe/Nux4YxU6YyGWJf6Dll0kQiNvlo8bFSKlnk3MY4YCLDr3m8LSkEidHN+O0A Z5BnpWRGgy7lVUJVvIXrjx99MouVvpuDqO/7qnpRUx1yT7A0Sw6qEikDrSL8a7/fnEi1kRsvD6U/N QCbqGbLddIcX/09NuE4Atqm/2AQk2sXoqaKZq7rcTEhM4AAIio6+Sk4je/b/GIMIT2RU4iuUzozVf 7zWoszfNtZyJyu7bwHuAIVRLONfjtvoqNDNbsxy1bb3lIQ0wdoSU+GfAFygi06hJnFhjOa2fT4TPB 8ldDHMe9DW06ruR4gan39Q==; Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=sky) by relay.virtuozzo.com with esmtps (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1xTG3P-009Hhf-0k; Tue, 17 Nov 2026 11:08:57 +0100 From: Alexander Ivanov To: andrey.zhadchenko@virtuozzo.com Date: Tue, 17 Nov 2026 11:09:04 +0100 Message-ID: <20261117100904.3071306-3-alexander.ivanov@virtuozzo.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20261117100904.3071306-1-alexander.ivanov@virtuozzo.com> References: <20261117100904.3071306-1-alexander.ivanov@virtuozzo.com> MIME-Version: 1.0 X-OZ-Fwd: true Cc: devel@openvz.org Subject: [Devel] [PATCH hci-8.1 v3 2/2] drivers/vhost/blk: support VIRTIO_BLK_F_WRITE_ZEROES X-BeenThere: devel@openvz.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OpenVZ development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: devel-bounces@openvz.org Errors-To: devel-bounces@openvz.org Handle VIRTIO_BLK_T_WRITE_ZEROES for parity with virtio-blk. The range is zeroed with __blkdev_issue_zeroout(), which uses WRITE ZEROES of the backend or falls back to writing zero pages. The UNMAP flag maps to the absence of BLKDEV_ZERO_NOUNMAP. The range is issued in chunks of at most 16 WRITE ZEROES bios and 1 GiB, or 4 MiB ones when zero pages are written, since each chunk is built in the vhost worker. The VMM must advertise blk_size equal to the backend logical block size, max_write_zeroes_sectors aligned to it and at most 8388607, and max_write_zeroes_seg = 1. A request is zeroed one chunk at a time and holds vhost_blk_flush() until its last chunk, so a small max_write_zeroes_sectors keeps vhost ioctls short on backends without WRITE ZEROES. Our QEMU does not advertise the feature yet. The guest gets VIRTIO_BLK_S_UNSUPP when VIRTIO_BLK_F_WRITE_ZEROES was not negotiated, for a payload other than exactly one segment or an unknown flag, and VIRTIO_BLK_S_IOERR for a read-only backend and for a range that is unaligned, beyond the capacity or above 8388607 sectors. https://virtuozzo.atlassian.net/browse/VSTOR-141684 Feature: vhost-blk: in-kernel accelerator for virtio-blk guests Signed-off-by: Alexander Ivanov --- drivers/vhost/blk.c | 178 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 176 insertions(+), 2 deletions(-) diff --git a/drivers/vhost/blk.c b/drivers/vhost/blk.c index eea8cca59cac..e27320651469 100644 --- a/drivers/vhost/blk.c +++ b/drivers/vhost/blk.c @@ -29,7 +29,8 @@ enum { (1ULL << VIRTIO_RING_F_INDIRECT_DESC) | (1ULL << VIRTIO_RING_F_EVENT_IDX) | (1ULL << VIRTIO_BLK_F_MQ) | - (1ULL << VIRTIO_BLK_F_FLUSH), + (1ULL << VIRTIO_BLK_F_FLUSH) | + (1ULL << VIRTIO_BLK_F_WRITE_ZEROES), }; /* @@ -75,6 +76,10 @@ struct vhost_blk_req { struct iovec status[VHOST_MAX_METADATA_IOV]; sector_t sector; + sector_t nr_sects; + sector_t chunk_sects; + struct block_device *bdev; + unsigned int zeroout_flags; int bi_opf; u16 head; size_t len; @@ -271,6 +276,63 @@ static int vhost_blk_move_bb_to_req(struct vhost_blk_req *req) return 0; } +/* + * The range is zeroed in the vhost worker with vq->mutex held, so it is + * split into chunks of a few bios each, much smaller ones when zero pages + * are written. The chunk sizes are multiples of any logical block size. + */ +#define VHOST_BLK_MAX_WRITE_ZEROES_BIOS 16 +#define VHOST_BLK_MAX_WRITE_ZEROES_SECTORS (SZ_1G >> SECTOR_SHIFT) +#define VHOST_BLK_MAX_ZERO_PAGES_SECTORS (SZ_4M >> SECTOR_SHIFT) + +/* + * A request holds vhost_blk_flush() until its last chunk completes, so the + * whole range is bounded too. No Linux guest sends more, a bio cannot carry + * it. + */ +#define VHOST_BLK_MAX_WRITE_ZEROES_REQ_SECTORS (UINT_MAX >> SECTOR_SHIFT) + +static int vhost_blk_bio_make_zeroes(struct vhost_blk_req *req, + struct block_device *bdev) +{ + sector_t bs_mask = (bdev_logical_block_size(bdev) >> SECTOR_SHIFT) - 1; + sector_t max_sects = (sector_t)bdev_write_zeroes_sectors(bdev) * + VHOST_BLK_MAX_WRITE_ZEROES_BIOS & ~bs_mask; + struct bio *bio = NULL; + int ret = -EOPNOTSUPP; + + req->chunk_sects = min3(req->nr_sects, max_sects, + (sector_t)VHOST_BLK_MAX_WRITE_ZEROES_SECTORS); + /* The limit may appear after it was read, and no bio is built then */ + if (req->chunk_sects) + ret = __blkdev_issue_zeroout(bdev, req->sector, + req->chunk_sects, GFP_KERNEL, &bio, + req->zeroout_flags | + BLKDEV_ZERO_NOFALLBACK); + if (ret == -EOPNOTSUPP) { + req->chunk_sects = min_t(sector_t, req->nr_sects, + VHOST_BLK_MAX_ZERO_PAGES_SECTORS); + ret = __blkdev_issue_zeroout(bdev, req->sector, + req->chunk_sects, GFP_KERNEL, &bio, + req->zeroout_flags); + } + if (ret) + return ret; + + bio->bi_private = req; + bio->bi_end_io = vhost_blk_req_done; + req->bio = req->inline_bio; + req->bio[0] = bio; + + /* Zero pages are plain writes, a failure of them is not retried */ + if (bio_op(bio) != REQ_OP_WRITE_ZEROES) + req->bi_opf = REQ_OP_WRITE; + + atomic_set(&req->bio_nr, 1); + + return 0; +} + static int vhost_blk_bio_make(struct vhost_blk_req *req, struct block_device *bdev) { @@ -284,6 +346,9 @@ static int vhost_blk_bio_make(struct vhost_blk_req *req, if (unlikely(req->bi_opf & REQ_PREFLUSH)) return vhost_blk_bio_make_simple(req, bdev); + if (req->bi_opf == REQ_OP_WRITE_ZEROES) + return vhost_blk_bio_make_zeroes(req, bdev); + /* * Requests with unaligned io vectors are not supported by underlined * block device, let's use an aligned intermediary "bounce buffer" to @@ -451,6 +516,50 @@ static bool vhost_blk_backend_ro(struct file *f) bdev_read_only(I_BDEV(f->f_mapping->host)); } +static int vhost_blk_parse_write_zeroes(struct vhost_blk_req *req, u8 *status) +{ + struct file *f = vhost_vq_get_backend(&req->blk_vq->vq); + struct block_device *bdev = I_BDEV(f->f_mapping->host); + sector_t bs_mask = (bdev_logical_block_size(bdev) >> SECTOR_SHIFT) - 1; + struct virtio_blk_discard_write_zeroes seg; + struct iov_iter iter; + u32 flags; + + if (!vhost_has_feature(&req->blk_vq->vq, VIRTIO_BLK_F_WRITE_ZEROES) || + req->len != sizeof(seg)) { + *status = VIRTIO_BLK_S_UNSUPP; + return 0; + } + + iov_iter_init(&iter, ITER_SOURCE, req->iov, req->iov_nr, req->len); + if (copy_from_iter(&seg, sizeof(seg), &iter) != sizeof(seg)) + return -EFAULT; + + flags = le32_to_cpu(seg.flags); + if (flags & ~VIRTIO_BLK_WRITE_ZEROES_FLAG_UNMAP) { + *status = VIRTIO_BLK_S_UNSUPP; + return 0; + } + + req->sector = le64_to_cpu(seg.sector); + req->nr_sects = le32_to_cpu(seg.num_sectors); + req->bdev = bdev; + req->zeroout_flags = flags & VIRTIO_BLK_WRITE_ZEROES_FLAG_UNMAP ? + 0 : BLKDEV_ZERO_NOUNMAP; + + if (vhost_blk_backend_ro(f) || + (req->sector | req->nr_sects) & bs_mask || + req->nr_sects > VHOST_BLK_MAX_WRITE_ZEROES_REQ_SECTORS || + req->sector > bdev_nr_sectors(bdev) || + req->nr_sects > bdev_nr_sectors(bdev) - req->sector) { + *status = VIRTIO_BLK_S_IOERR; + return 0; + } + + *status = VIRTIO_BLK_S_OK; + return 0; +} + static int vhost_blk_req_handle(struct vhost_virtqueue *vq, struct virtio_blk_outhdr *hdr, u16 head, u16 out, u16 in) @@ -475,6 +584,7 @@ static int vhost_blk_req_handle(struct vhost_virtqueue *vq, req->head = head; req->blk = blk; req->sector = vhost64_to_cpu(vq, hdr->sector); + req->nr_sects = 0; req->iov = blk_vq->iov; req->bio_err = 0; in_len = iov_length(vq->iov + out, in); @@ -490,7 +600,7 @@ static int vhost_blk_req_handle(struct vhost_virtqueue *vq, req->status[0].iov_base = st->iov_base + st->iov_len; req->status[0].iov_len = sizeof(status); - if (type == VIRTIO_BLK_T_OUT) { + if (type == VIRTIO_BLK_T_OUT || type == VIRTIO_BLK_T_WRITE_ZEROES) { data_iov = vq->iov; data_nr = out; } else { @@ -538,6 +648,22 @@ static int vhost_blk_req_handle(struct vhost_virtqueue *vq, req->bi_opf = REQ_OP_WRITE | REQ_PREFLUSH; ret = vhost_blk_req_submit(req); break; + case VIRTIO_BLK_T_WRITE_ZEROES: + ret = vhost_blk_parse_write_zeroes(req, &status); + if (ret) + break; + if (status == VIRTIO_BLK_S_OK && req->nr_sects) { + req->bi_opf = REQ_OP_WRITE_ZEROES; + ret = vhost_blk_req_submit(req); + if (ret != -EPERM) + break; + status = VIRTIO_BLK_S_IOERR; + } + ret = vhost_blk_set_status(req, status); + if (ret) + break; + vhost_add_used_and_signal(&blk->dev, vq, head, sizeof(status)); + break; case VIRTIO_BLK_T_GET_ID: len = min_t(size_t, strnlen(blk->serial, VIRTIO_BLK_ID_BYTES) + 1, @@ -561,6 +687,51 @@ static int vhost_blk_req_handle(struct vhost_virtqueue *vq, return ret; } +/* + * Some devices clear their WRITE ZEROES limit only after a failed request, + * as blkdev_issue_write_zeroes() expects. Retry such a request with zero + * pages instead of failing it. + */ +static bool vhost_blk_retry_zeroes(struct vhost_blk_req *req) +{ + if (!req->bio_err || req->bi_opf != REQ_OP_WRITE_ZEROES) + return false; + + if (bdev_write_zeroes_sectors(req->bdev)) + return false; + + /* The limit may come back before the bio is built, retry only once */ + req->bi_opf = REQ_OP_WRITE; + if (vhost_blk_bio_make_zeroes(req, req->bdev)) + return false; + + req->bio_err = 0; + vhost_blk_bio_send(req); + return true; +} + +/* Issue the next chunk of a write-zeroes range, if any is left */ +static bool vhost_blk_next_zeroes(struct vhost_blk_req *req) +{ + int ret; + + if (req->bio_err || req->nr_sects <= req->chunk_sects) + return false; + + req->sector += req->chunk_sects; + req->nr_sects -= req->chunk_sects; + + req->bi_opf = REQ_OP_WRITE_ZEROES; + ret = vhost_blk_bio_make_zeroes(req, req->bdev); + if (ret) { + req->bio_err = ret; + return false; + } + + vhost_blk_bio_send(req); + return true; +} + static void vhost_blk_handle_guest_kick(struct vhost_work *work) { struct virtio_blk_outhdr hdr; @@ -661,6 +832,9 @@ static void vhost_blk_handle_host_kick(struct vhost_work *work) if (!blk) blk = req->blk; + if (vhost_blk_retry_zeroes(req) || vhost_blk_next_zeroes(req)) + continue; + if (!req->bio_err && req->bb && req->bi_opf == REQ_OP_READ) { if (vhost_blk_move_bb_to_req(req)) req->bio_err = EINVAL; -- 2.53.0 _______________________________________________ Devel mailing list Devel@openvz.org https://lists.openvz.org/mailman/listinfo/devel