From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from DU2PR03CU002.outbound.protection.outlook.com (mail-northeuropeazon11021117.outbound.protection.outlook.com [52.101.65.117]) by lore.virtuozzo.com (Postfix) with ESMTPS id E10BC80CB2 for ; Tue, 27 Oct 2026 11:39:06 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=M0PYF2UtRh3i1isa9Bdd6ohTEuuBXQtudPkdYeb2ay2Ql/uB+AdSWN+wZCABrKqFMv0wLM6HfcxNjoDkppE+Pymgw3rRin5j9rsXnXL3kFTz7DOUm2CQ4t+Wye6w+Y89z2rioWV6d2an0WJkEy4dpW0uvfgUB1wS5MTm5Bn3kn5tWq1ynia2huPwhnL84TIYVIqsORJKaK5FDDZcUHjis5eN5Kc1/yDOGjE0C6eSayJuuTDBPifTtsHI1HJfLxuvlAtmIYNuIfH0m2QbYxfluYQE21Lsv6lgG2r0Ix6HPzTxmrh0zyFjat5I17tJL1ke9mbV+tj5lb0EVclWc9erRg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=gcK9xCkTZlXoH6cGyYSaKeY6YNI1qfcF444NnpjjFf0=; b=SQGTVB286FlG9KJjrHMvJUtKcw9MA5a2bexIQKfDcimzkFavEoOM+Sf3stUR4oeDMnpNV2p22fnVgt4WmzV1Zj3FaOSIIt1pctbux5NQRWJvWjcr6lCdpbtTbD/N35RwNr1o7ZvApkbyaOVfQQzkULCw3iFSe2giycK7ooT0swRLVxfjU9vZuPxlbyTPUoJ9HyljkwS+O6VPOuVSIk8YT9P97DsDCN57gD+v33wMthJ2RrYu2IRkWOD7OMeFVr2WqKhx7/102WG4m/R0cK18f57OG+CIfX3EZ3l06YtRq3AV0L4Roqj9ODSdUTLe2ibbzZOzhBlsNSYWhqZ0fRkjaA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 130.117.225.111) smtp.rcpttodomain=virtuozzo.com smtp.mailfrom=virtuozzo.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=virtuozzo.com; dkim=pass (signature was verified) header.d=virtuozzo.com; arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gcK9xCkTZlXoH6cGyYSaKeY6YNI1qfcF444NnpjjFf0=; b=LyvjWMLhPlKBMIWdWhCXhrGYiSSVddH25wZI/mP2OvZqx7+cNqQaX24uFCG4H+PruJuPdxmDOy59RzkbhqTOZJJr8cK4wWivje3S/qfOTxr+m+dQRcg096Xu1T6ZzctcvJUqk1a3tY2dMPxPBmJEzPtv6TMRN/+v0vflFrqYgUavWrQrMY8BGkdM6zDOWz64CNgDvDpB2i2f3Bn6iU7Dfus3DY2m03FybSt2j/ovFksB9a4PU14/lYNsmOtdahIFeEaCzxYpEvlum+aDBn/Y5C8gr1507l2bVDMxLx1VEbpKJIF1RJh7OcyKhNF/hLPhjLnsKEtGx4RWLqItxL5lOw== Received: from ZR0P278CA0180.CHEP278.PROD.OUTLOOK.COM (2603:10a6:910:45::7) by GV1PR08MB11026.eurprd08.prod.outlook.com (2603:10a6:150:1ed::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.546.22; Tue, 27 Oct 2026 11:39:01 +0000 Received: from ZR1PEPF0000E6B7.eurprd07.prod.outlook.com (2603:10a6:910:45:cafe::59) by ZR0P278CA0180.outlook.office365.com (2603:10a6:910:45::7) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.546.22 via Frontend Transport; Tue, 27 Oct 2026 11:39:01 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 130.117.225.111) smtp.mailfrom=virtuozzo.com; dkim=pass (signature was verified) header.d=virtuozzo.com;dmarc=pass action=none header.from=virtuozzo.com; Received-SPF: Pass (protection.outlook.com: domain of virtuozzo.com designates 130.117.225.111 as permitted sender) receiver=protection.outlook.com; client-ip=130.117.225.111; helo=relay.virtuozzo.com; pr=C Received: from relay.virtuozzo.com (130.117.225.111) by ZR1PEPF0000E6B7.mail.protection.outlook.com (10.167.241.70) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.569.4 via Frontend Transport; Tue, 27 Oct 2026 11:39:00 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=virtuozzo.com; s=relay; h=MIME-Version:Message-ID:Date:Subject:From: Content-Type; bh=gcK9xCkTZlXoH6cGyYSaKeY6YNI1qfcF444NnpjjFf0=; b=EadticBvnkFo pwKKOTM7S2lpyiHdI3/uNKCjQxaBboGF3WyPmiS2iSPeqnVnzXhUt1QIGRVG3ZOOYCMpe9VWzA0+l 7FacBM8zHjqJ2hoxunHJ3Sj+KVzCcJzrfnPWdkgeeeRCtZgan3rRZBM8bvgwDKaHpSTebbjAbiqxu PjAV6G8rGl42JRcmCB1VrCG4kFIUZQrJWpYGKwynKkSXL44/QtsCrUiXV3+08kc6jXJZmuBUWkCSs pATVXc8rn9ZjbGz0GgPvavbp8+Yk478NTTzBmkRoZcODYaWGLR26xbQOZeSNeZllGVhZuMcPc729M L9oNFL02DhOMr5fhF1v3XQ==; Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=sky) by relay.virtuozzo.com with esmtps (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1xLfSN-00BIbh-1o; Tue, 27 Oct 2026 12:38:50 +0100 From: Alexander Ivanov To: andrey.zhadchenko@virtuozzo.com Cc: andrey.drobyshev@virtuozzo.com, svt-core@virtuozzo.com Subject: [PATCH hci-8.1 1/2] vhost-blk: create kernel threads with the mainstream worker ioctls Date: Tue, 27 Oct 2026 12:38:56 +0100 Message-ID: <20261027113857.2928048-2-alexander.ivanov@virtuozzo.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20261027113857.2928048-1-alexander.ivanov@virtuozzo.com> References: <20261027113857.2928048-1-alexander.ivanov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: ZR1PEPF0000E6B7:EE_|GV1PR08MB11026:EE_ X-MS-Office365-Filtering-Correlation-Id: 48df9d63-55ff-494e-e809-08df341ee4f5 X-LD-Processed: 0bc7f26d-0264-416e-a6fc-8352af79c58f,ExtAddr List-Id: svt-core@virtuozzo.com X-MS-Exchange-Transport-Rules-Loop: 1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|36860700016|1800799024|376014|23010399003|5113699003|10067099003|18002099003|22082099003|18092099006|261026140227099003|261009223027099003|56012099006|5023799004|261009221227099003|261022105027099003|261022075727099003|261021121427099003|261026121527099003|261021123027099003; X-Microsoft-Antispam-Message-Info: vhX6NMLSu1KfayAV5RcwZ14QGqk/vqGHsVuENsnnR6jz9V2ozMhM0GCdW/40hzN8L1Bzm3dL7zczumn1pZPSo9wGYQ+t2xk7763cGmjwRSsRgXq8lvh2lixYzrSqEYfYHwHCZbIFNmK1tIrhgLkX3jEeo6Aq8RNyZ5GzWJsuKaIQHMYCNq1dgy37jQT/dmiq0lLTnRn8IFU0X34FoVsPHYn81nAZlmIV3x2iTe1ilcliCVg19ZzNHSnrBjdtCo4Krz/Atm8tPE+ioiNSeN2sqXo4GC6jW0QODOVeg0s2USOhUmaDu9DF4iz3MhowL72/D8s44r9O7MlaZ9OOqB84W87rhhzWdl7TmHP7X1DVaKHdmVAPXXHLqhniKQli7uI2UfQ8JgsSxG7a3rJjRGdBxq4g7659lPR31KqSdUe5zF75qv+x2+Ob6x2zHF59C343PVazWg41KzdDjK43LW7s4QBjURZLTg2cxPx6ADumeGCYsG3sWg/cQc9SfIxtkMkXYQUaQ+NQ3toWyIcH4uOsYCV61e9jBhtJJ94wrIwTvtU+th2lWnZqWJl5qSqhYs4SmrdeQqnghblOgNfAETrY5EpS3hUkKSvAL3cHI8Vrt/XLYQ4K0D2rkM75rFQTExdjUdBG5aTU6fUHavloPRVIQcKXEKGm8+sl5WbaNhWKd84= X-Forefront-Antispam-Report: CIP:130.117.225.111;CTRY:CH;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:relay.virtuozzo.com;PTR:relay.virtuozzo.com;CAT:NONE;SFS:(13230040)(82310400026)(36860700016)(1800799024)(376014)(23010399003)(5113699003)(10067099003)(18002099003)(22082099003)(18092099006)(261026140227099003)(261009223027099003)(56012099006)(5023799004)(261009221227099003)(261022105027099003)(261022075727099003)(261021121427099003)(261026121527099003)(261021123027099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: PM4kLyJiwXK0kFnvzZ2uanm80U2d3sno0SWz7ixgxCsiYQTRL0e4E/PKxLZTVPUohfPHpuNAJfaUVCE+kn8zHBdLkK3fr2LgkXX2lpfAUVeJEuJbnrtoKGtrTK8CILE2rkWUQGv85tb+ATKR8sy5HBFAS6d+MW53NxYBnSZ3O9GzN9goZNLwZIXZtOGnkTi+olnXOP3S0dPEeNFsh8yUF75lpOEcbXi87uELXgBuq9iAtUS9I5d9ZYuyaBaD87+JkzxnuzVAa9FQbeM/7ifmR0JyB0wI2u5x4FiLFfUcE6Si4knDBU8bZLUq3m2sQuHOw9FW+5C/fli0Te1IuSqGfg9ckGsnDV2UF/sVzqLyq9sAj0w3QxZZR0T8jwW0h4txworugCOZVV9MR6JSq8xKyQpckEH9SEFsNKo/JEx0vS3g1BYz9/PVxBb9mmJrzf7O X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Oct 2026 11:39:00.4466 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 48df9d63-55ff-494e-e809-08df341ee4f5 X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=0bc7f26d-0264-416e-a6fc-8352af79c58f;Ip=[130.117.225.111];Helo=[relay.virtuozzo.com] X-MS-Exchange-CrossTenant-AuthSource: ZR1PEPF0000E6B7.eurprd07.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV1PR08MB11026 num-threads was passed to our own VHOST_SET_NWORKERS kernel ioctl. The kernel is dropping it in favour of the mainstream worker ioctls, which vhost-blk now supports, so use those: create num-threads - 1 workers with VHOST_NEW_WORKER and attach them to the virtqueues round-robin with VHOST_ATTACH_VRING_WORKER, the default worker being the first one. This is the layout VHOST_SET_NWORKERS produced. VHOST_NEW_WORKER works only in vhost_task mode, while the kernel defaults to kthreads, so switch the device to vhost_task mode with VHOST_SET_FORK_FROM_OWNER before VHOST_SET_OWNER. VHOST_RESET_OWNER restores the default, hence this is done on every start. With num-threads=1 nothing changes: no worker ioctls, kthread mode. On a kernel without these ioctls warn and run a single thread rather than fail to start vhost-blk. https://virtuozzo.atlassian.net/browse/VSTOR-107537 Signed-off-by: Alexander Ivanov --- hw/block/vhost-blk.c | 75 +++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 71 insertions(+), 4 deletions(-) diff --git a/hw/block/vhost-blk.c b/hw/block/vhost-blk.c index abddff02c6d..99fec9533a9 100644 --- a/hw/block/vhost-blk.c +++ b/hw/block/vhost-blk.c @@ -33,6 +33,51 @@ #include "migration/cpr.h" #include "system/runstate.h" +/* + * Spread the virtqueues round-robin over num-threads workers, the first one + * being the default worker that VHOST_SET_OWNER created. + */ +static int vhost_blk_set_workers(VHostBlk *s, int nworkers) +{ + uint32_t ids[VHOST_BLK_MAX_QUEUES]; + struct vhost_worker_state worker; + struct vhost_vring_worker vq_worker; + int ret, i; + + for (i = 1; i < nworkers; i++) { + memset(&worker, 0, sizeof(worker)); + if (ioctl(s->vhostfd, VHOST_NEW_WORKER, &worker)) { + ret = -errno; + if (ret == -ENOTTY && i == 1) { + warn_report_once("vhost-blk: kernel does not support worker " + "ioctls, using a single kernel thread"); + return 0; + } + error_report("vhost-blk: unable to create kernel thread: %s", + strerror(-ret)); + return ret; + } + ids[i] = worker.worker_id; + } + + for (i = 0; i < s->dev.nvqs; i++) { + if (i % nworkers == 0) { + continue; + } + memset(&vq_worker, 0, sizeof(vq_worker)); + vq_worker.index = i; + vq_worker.worker_id = ids[i % nworkers]; + if (ioctl(s->vhostfd, VHOST_ATTACH_VRING_WORKER, &vq_worker)) { + ret = -errno; + error_report("vhost-blk: unable to attach kernel thread to " + "queue %d: %s", i, strerror(-ret)); + return ret; + } + } + + return 0; +} + static int vhost_blk_start(VirtIODevice *vdev) { VHostBlk *s = VHOST_BLK(vdev); @@ -51,15 +96,37 @@ static int vhost_blk_start(VirtIODevice *vdev) return 0; } + nworkers = MIN(s->conf.num_threads, s->dev.nvqs); + if (nworkers > 1) { + /* + * VHOST_NEW_WORKER works only in vhost_task mode, and + * VHOST_RESET_OWNER brings the kthread default back. + */ + uint8_t fork_owner = VHOST_FORK_OWNER_TASK; + + if (ioctl(s->vhostfd, VHOST_SET_FORK_FROM_OWNER, &fork_owner)) { + ret = -errno; + if (ret != -ENOTTY) { + error_report("vhost-blk: unable to set vhost task mode: %s", + strerror(-ret)); + return ret; + } + warn_report_once("vhost-blk: kernel does not support vhost " + "tasks, using a single kernel thread"); + nworkers = 1; + } + } + if (ioctl(s->vhostfd, VHOST_SET_OWNER, NULL)) { error_report("vhost-blk: unable to set owner"); return -ENOSYS; } - nworkers = s->conf.num_threads; - if (nworkers != 1 && ioctl(s->vhostfd, VHOST_SET_NWORKERS, &nworkers)) { - error_report("vhost-blk: unable to set number of kernel threads"); - return -EINVAL; + if (nworkers > 1) { + ret = vhost_blk_set_workers(s, nworkers); + if (ret < 0) { + return ret; + } } if (s->conf.serial) { -- 2.53.0