From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from PA4PR04CU001.outbound.protection.outlook.com (mail-francecentralazon11023092.outbound.protection.outlook.com [40.107.162.92]) by lore.virtuozzo.com (Postfix) with ESMTPS id A78B780F25 for ; Thu, 29 Oct 2026 09:23:14 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=wHiZonS8+kcyqmCuHF8wCWnmX4ec4ciQZLc1Q0I1H0xti2V2rODew8jRS4B8uD4mDF8NzXfx4gslvPSqEe6I1AP+I+2TjX7wcyK/ZBqxWIwGA2bFciclur2CBtKHWJ6gPnrx3l/vw/Lwui9LmnJtONiSlwyTEtFYgu2yxTYeFQyNlh0iinnMw0f02nYhou+Sp2RTCPSK/tjsL6b5HCKEU4988WpIokKByaCa844iDE9EvUz6d4Wc48pAh+8x1C11T0zsQcOWWG+AtuXIsSkHN9+tHgLKMiZksOFfD6YnlCeipkpC+rDvfzzDS7SHauzNZnpBaF8Mz4sH4lmFNgF7+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1DmeoD0XlqyeT3uAEcThSIGOqCbs3n2ciRxtx5NDIOM=; b=YBygYQou18ca/A5b7Su7fWplSHnOXOFTdGh/O6E7z+H7LWw0Sefx97CDR6xC2Zv9BJz44RBLfeNnEh4gltiGc586ewArf48ZLWexr1sUGv2o2q8XEg7Bb46q3y4a8oEbUdK1QKppUn9CpAYkjNIpC4cX2KlRiF0s4D+tmwSMOsv6Gc9MEWQ42w20HB3qd4PFZxELPW3OjckyDjCnGTi/3YGZ6wjlruUCc2s7jBSjF8cwsnFKxpd5ezXYd8nM8WI0Nk+AS5hxcZHdLgdKI4/tgbpXKxqbmKxz69W/Ju4bPe8kS0t4OxzoO9EvR13CttHBKPEp82H7siLYoVDt2edT4w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 130.117.225.111) smtp.rcpttodomain=virtuozzo.com smtp.mailfrom=virtuozzo.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=virtuozzo.com; dkim=pass (signature was verified) header.d=virtuozzo.com; arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1DmeoD0XlqyeT3uAEcThSIGOqCbs3n2ciRxtx5NDIOM=; b=RcMl5SAyeVwjuqrf8n+UfmE7sI7uOyDw46HWLBw5c8DuaLzrSnGgSDt81tPKxdLWdPVuevIBOSbsU00j5hs4a+/sdwFpN5a/sGPdZf95ORTXztYpcjbYznjNblY83OTWa/vgIFcZ53InkIFWUJ2fTL5JrRQHd4jhbz7zc9Zg98EAJhfVV7MmNnmcWwBQgx9hWxv0GGepThR73HxAqAxITaDNtpA2Q0RfPgbR4nLAn8yaQKHstD3FMvkvs+iObaVmzMSQSlw6f6/1xTQtsOnHGTByBBaU0P2ENWXTpdec2wvvIWF9r77LFRd5v8QJkQ/qbHGq8OkscvWrQE9A/dJdxg== Received: from AS9PR05CA0044.eurprd05.prod.outlook.com (2603:10a6:20b:489::34) by AS8PR08MB7861.eurprd08.prod.outlook.com (2603:10a6:20b:52c::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.569.8; Thu, 29 Oct 2026 09:23:08 +0000 Received: from AM2PEPF00070CF9.eurprd02.prod.outlook.com (2603:10a6:20b:489:cafe::a1) by AS9PR05CA0044.outlook.office365.com (2603:10a6:20b:489::34) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.569.7 via Frontend Transport; Thu, 29 Oct 2026 09:23:08 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 130.117.225.111) smtp.mailfrom=virtuozzo.com; dkim=pass (signature was verified) header.d=virtuozzo.com;dmarc=pass action=none header.from=virtuozzo.com; Received-SPF: Pass (protection.outlook.com: domain of virtuozzo.com designates 130.117.225.111 as permitted sender) receiver=protection.outlook.com; client-ip=130.117.225.111; helo=relay.virtuozzo.com; pr=C Received: from relay.virtuozzo.com (130.117.225.111) by AM2PEPF00070CF9.mail.protection.outlook.com (10.167.242.11) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.569.4 via Frontend Transport; Thu, 29 Oct 2026 09:23:07 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=virtuozzo.com; s=relay; h=MIME-Version:Message-ID:Date:Subject:From: Content-Type; bh=1DmeoD0XlqyeT3uAEcThSIGOqCbs3n2ciRxtx5NDIOM=; b=FsMWuIqnxbsw v4r20O8sM95oI++QSoONyVZqdwkBoG79Lr2P01OIn7Y8JDh/rwuSRAQdKo1Cp6+ue0LGAxyr8EUW9 teho8MnQbrK6e2Cp2cLRrWP57VuDmSvUd+CF4YemLowAbCV9aZtSlX/fHXjH0sR1XKYf+dTsi1jGE QOu5vi/cT8NgGgj5C3nEiSNFwXBgoA/9DEYHbz1IcCZy+KOLAJrbZrFOf7UBW1qFXkOWFjMLUHd4G nwCUdfevNjsY1CQqWLbnSC4gsMziYICB2wpoMsG2xQoqyYDtLswlwkCnJLX3TScRX9Vzqkg13tdHv 3NKI9nAuF6fMAT+kSML2wg==; Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=sky) by relay.virtuozzo.com with esmtps (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.96) (envelope-from ) id 1xMMHv-000QvD-1N; Thu, 29 Oct 2026 10:23:05 +0100 From: Alexander Ivanov To: andrey.zhadchenko@virtuozzo.com Cc: andrey.drobyshev@virtuozzo.com, svt-core@virtuozzo.com Subject: [PATCH hci-8.1 v4 4/5] vhost-blk: create kernel threads with the mainstream worker ioctls Date: Thu, 29 Oct 2026 10:23:00 +0100 Message-ID: <20261029092301.3286535-5-alexander.ivanov@virtuozzo.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20261029092301.3286535-1-alexander.ivanov@virtuozzo.com> References: <20261029092301.3286535-1-alexander.ivanov@virtuozzo.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM2PEPF00070CF9:EE_|AS8PR08MB7861:EE_ X-MS-Office365-Filtering-Correlation-Id: 5d5133ca-1188-487c-0d95-08df359e3e1b X-LD-Processed: 0bc7f26d-0264-416e-a6fc-8352af79c58f,ExtAddr List-Id: svt-core@virtuozzo.com X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|23010399003|376014|36860700016|82310400026|261028144727099003|56012099006|5023799004|261009223027099003|261009221227099003|261026121527099003|261021121427099003|261021123027099003|261026140227099003|261022105027099003|10067099003|261022075727099003|22082099003|18002099003|5113699003|13003099007; X-Microsoft-Antispam-Message-Info: fW9+l6ug4bMOOg/ems9kWAJCrV2s8earYhE4Wr8mmIfFlGVFB7BHkLCw+KXjQU7vN7MvgtmI6fjifHuMtOZhNjr506VMimeqi+UI1BQDnLZCI7+cbP8MtehGyJiH1cSzkRO9aRbpHQnnlJra2EEh0g/crVtCuTFvY74YA6+6i9j72YFdb+Xcu3WCHaB3SldBtCNAEk9ptF3z09YJIvolMcAUPytUMa4uNZovZaEs3vvq0tQcIY6mO8R9pV+aWUfH6DbQ5aSCQ/znCApSh13h/q/mWW7c2Za1lAxnc1AYBpdCv2NOrrE2e+BwbnaLyzgSv1PBqddvLjd3q4hbbbccyAh/kTgK4H3IZVVVV4yYgz9hW6npB6peo0SbrlgvOgDKlcilcav9rpNqpAdM2PB9zN050uu10NgGol+l60oCZR2e8OuVrz3hADZ9IdfloL0xv0yYkCsOVErKvUHYcGaxgKcu2Sg/jQyxeoRfefcpAoXsQoVfxAwiLIGgnvgYEl/dLVwR5gRpa1HT0WftkX7MkjIQJ0Y8+bCtZ6v9k7yO951IU/zFSwuW1ArlSMj4eq3ovW1l568rhrpLjXK16Ulqoo980JIjVbJBL1ixAlADCENXbpOy45a7VL+OMfylBFxp4W4l93X56B1zquE846hJWOhz8bguQKWrsNBiHwVk2jM= X-Forefront-Antispam-Report: CIP:130.117.225.111;CTRY:CH;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:relay.virtuozzo.com;PTR:relay.virtuozzo.com;CAT:NONE;SFS:(13230040)(1800799024)(23010399003)(376014)(36860700016)(82310400026)(261028144727099003)(56012099006)(5023799004)(261009223027099003)(261009221227099003)(261026121527099003)(261021121427099003)(261021123027099003)(261026140227099003)(261022105027099003)(10067099003)(261022075727099003)(22082099003)(18002099003)(5113699003)(13003099007);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: KjSruQFPImOX5fQWGRwKe5T2z5q97ZMnmIplSGDhFJBt/v0L59AlwEK+hyxIS5cNtUB4Wuh9Z0oEWjlWgg6UXRYl3iCFO/kDTHA1JcSqsK+CmJkKZcNfNLQPD4RIifpqXGmBz6IecTrAhZhvvHNsBPf81KH/vuIt3UVMDKaAYHb2P35g3X5dqmbQfoTJjF3ZlkaaQ+Jl+lxTwsLeNYqakLgeXUgdnLNYVJZ30gZNwa2UXfh2AmYzuF/slGJinMLSgVWoHjFcqTnd/+3MjkFABHJEuoxYkaZmY2EjuGkmadJVKDVIzoo6EmOS9TfQNkSaEh3IZI7CfPU/DXky30gc3UThMqdumqR1Xh4dvt1y//rexjHqVg1oB6Q74gkvkshPN7u+osVZZsNeB/UXoFqr/0LNUUz96ViuM6nWUiGPSF7uwdKDUk0LzLA//thL6BX3 X-Auto-Response-Suppress: DR, OOF, AutoReply X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Oct 2026 09:23:07.2596 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 5d5133ca-1188-487c-0d95-08df359e3e1b X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=0bc7f26d-0264-416e-a6fc-8352af79c58f;Ip=[130.117.225.111];Helo=[relay.virtuozzo.com] X-MS-Exchange-CrossTenant-AuthSource: AM2PEPF00070CF9.eurprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB7861 num-threads was passed to our own VHOST_SET_NWORKERS kernel ioctl. The kernel drops it with the revert of "drivers/vhost: add ioctl to increase the number of workers" in favour of the mainstream worker ioctls, which vhost-blk now supports. Create num-threads - 1 workers with VHOST_NEW_WORKER and attach them to the virtqueues round-robin with VHOST_ATTACH_VRING_WORKER, the layout VHOST_SET_NWORKERS produced. Our kernel, unlike mainline, defaults to kthread mode, where VHOST_NEW_WORKER fails with -EFAULT rather than -ENOTTY. So switch the device to vhost_task mode with VHOST_SET_FORK_FROM_OWNER before VHOST_SET_OWNER. With num-threads=1 nothing changes: no worker ioctls, kthread mode. Released vz10 kernels have VHOST_SET_FORK_FROM_OWNER, but their vhost-blk does not handle the worker ioctls and fails VHOST_NEW_WORKER with -ENOTTY. Other errors are possible as well, so do not key on the errno: if VHOST_SET_FORK_FROM_OWNER fails, keep all virtqueues on the default thread; if VHOST_NEW_WORKER fails partway, spread the remaining virtqueues over the threads created so far; if attaching a virtqueue fails, it stays on the default thread. Either way warn rather than fail to start vhost-blk. Unless VHOST_SET_FORK_FROM_OWNER failed, the default thread stays a vhost_task, as the mode cannot be changed after VHOST_SET_OWNER. https://virtuozzo.atlassian.net/browse/VSTOR-107537 Signed-off-by: Alexander Ivanov --- hw/block/vhost-blk.c | 64 +++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 60 insertions(+), 4 deletions(-) diff --git a/hw/block/vhost-blk.c b/hw/block/vhost-blk.c index b3eb181ceae..9416c3ac07e 100644 --- a/hw/block/vhost-blk.c +++ b/hw/block/vhost-blk.c @@ -33,6 +33,53 @@ #include "migration/cpr.h" #include "system/runstate.h" +/* + * Spread the virtqueues round-robin over nworkers kernel threads. Thread 0 is + * the default worker VHOST_SET_OWNER created; vq 0 and every nworkers-th vq + * after it stay on it. + */ +static void vhost_blk_set_workers(struct vhost_dev *hdev, int nworkers) +{ + uint32_t ids[VHOST_BLK_MAX_QUEUES]; + int i, ret, err = 0, nfailed = 0; + + assert(nworkers <= ARRAY_SIZE(ids)); + + for (i = 1; i < hdev->nvqs; i++) { + int w = i % nworkers; + struct vhost_vring_worker vq_worker = { .index = i }; + + if (w == 0) { + continue; + } + + if (i < nworkers) { + struct vhost_worker_state worker = { }; + + ret = hdev->vhost_ops->vhost_new_worker(hdev, &worker); + if (ret < 0) { + warn_report("vhost-blk: unable to create kernel thread: %s", + strerror(-ret)); + nworkers = i; + continue; + } + ids[w] = worker.worker_id; + } + + vq_worker.worker_id = ids[w]; + ret = hdev->vhost_ops->vhost_attach_vring_worker(hdev, &vq_worker); + if (ret < 0) { + err = ret; + nfailed++; + } + } + + if (nfailed) { + warn_report("vhost-blk: unable to attach kernel threads to %d " + "queues: %s", nfailed, strerror(-err)); + } +} + static int vhost_blk_start(VirtIODevice *vdev) { VHostBlk *s = VHOST_BLK(vdev); @@ -52,15 +99,24 @@ static int vhost_blk_start(VirtIODevice *vdev) return 0; } + nworkers = MIN(s->conf.num_threads, hdev->nvqs); + if (nworkers > 1) { + ret = hdev->vhost_ops->vhost_set_fork_from_owner(hdev, + VHOST_FORK_OWNER_TASK); + if (ret < 0) { + warn_report("vhost-blk: unable to use multiple kernel threads: " + "%s", strerror(-ret)); + nworkers = 1; + } + } + if (ioctl(s->vhostfd, VHOST_SET_OWNER, NULL)) { error_report("vhost-blk: unable to set owner"); return -ENOSYS; } - nworkers = s->conf.num_threads; - if (nworkers != 1 && ioctl(s->vhostfd, VHOST_SET_NWORKERS, &nworkers)) { - error_report("vhost-blk: unable to set number of kernel threads"); - return -EINVAL; + if (nworkers > 1) { + vhost_blk_set_workers(hdev, nworkers); } if (s->conf.serial) { -- 2.53.0