From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from PA4PR04CU001.outbound.protection.outlook.com (mail-francecentralazon11023116.outbound.protection.outlook.com [40.107.162.116]) by lore.virtuozzo.com (Postfix) with ESMTPS id 0C5228045D for ; Mon, 2 Nov 2026 16:26:48 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=kUlyMjfGc4GWnRP8zbCrSt5E642bkt1ld3Sgmjs3s3qsjlT+k9U7Yt/6oIgM9XcL2mWTRHKzaMNe+V1OfDQVZHkz/uxTdG7s2ursBZbS2F368+DtYqJO7DvGYGD4+BOsnRpAN6dVqFZwIjSTapLvRINRjiLTGK7iUXwIcLqF7+qtQFu4bUXxKXNZg7J7cPnCEHQXIJAnMxEJEZv6GWJ7spDk/vTTLEQLtQBoAq2RW4EJJ0MrjHNwOVmloR7o9HSe9BOJEsyZK/fdiILU1rvs6GcDujL+0jZrYKWVu26Kx9j6Ikhez3zPaPu3iABIXHpcCoyqP09zHgxuf85ONnMUPQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Fo/oW4DxZZHUd0UHiMWqS//RmxhOxhUCmwTWmFSfdpo=; b=oZMmr1MipIe+psrkHQ/Lzy6HdimTA9N+Wl2jGayTMEHnoWFiwTlo8bQDRJX8eCV9k5Z1oIb5NDrSd8lIWeAg9kdSrJqs0OvXKP8JlFO1XriG0k1KeEz80Nn3FdSghaOiVNKdjn7yCeC72uF6HoLtk8YZeEa0xuXIw+8vHxzMpssF53yPd5ckxtgq5fGEAF9TkWmReAGZbNu60RiOHt3pFahZxrgfBkCyzsDU9ddPT7r7mFwRXOEUbf9S6XNUkzg/X7+UDaQhRVHmWwBDpPiJDQFtQRNuxGTdTl82zQuNCKBqO9Y5S1QdXioTdHhfr6zD2AL98zbBkogxNS0khoccCw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Fo/oW4DxZZHUd0UHiMWqS//RmxhOxhUCmwTWmFSfdpo=; b=V3HCST3Y6uAt7cqVwjDr4irgBIfi7iufwdvahlU5wfU/IsqJmYQrJ4Zf86E3gnR1FhrgA2Tug47Dw5OgkQWiSXY+lSC2Rn2DEG2YWKeM2iwCtBKYF0DC1MFs/IMovRvIsxzEY7YS3AHK7h7EcXOLchvCfAfVCgAlNwykhi6zJccoctC9MEzL/4aE8Dm7+Zf1IuWQf5SXpMN87Ce14w3OXg9dCpNufxdIG2LsRmcq/v1i8HgPJNWirpXnoTmUtq4fAAQ6jkmzjnE7u9QU2M1/Nxa/kzqa0EaRsr7KRukf+Dnj5wHxu7XqKfgytE7YLgMDnM2mYIDA4gvGD2xNggCD1Q== Authentication-Results: mx.microsoft.com 1; dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=virtuozzo.com; Received: from VI0PR08MB10656.eurprd08.prod.outlook.com (2603:10a6:800:20a::12) by MI7PR08MB837237.eurprd08.prod.outlook.com (2603:10a6:290:102::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.569.12; Mon, 2 Nov 2026 16:26:46 +0000 Received: from VI0PR08MB10656.eurprd08.prod.outlook.com ([fe80::4e37:b189:ddcd:3dd8]) by VI0PR08MB10656.eurprd08.prod.outlook.com ([fe80::4e37:b189:ddcd:3dd8%7]) with mapi id 15.21.0569.011; Mon, 2 Nov 2026 16:26:45 +0000 Message-ID: <129c45a3-2a29-44ea-b8b8-3de43fd9ea28@virtuozzo.com> Date: Mon, 2 Nov 2026 18:26:44 +0200 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH hci-8.1 v5 4/5] vhost-blk: create kernel threads with the mainstream worker ioctls To: Alexander Ivanov , andrey.zhadchenko@virtuozzo.com Cc: svt-core@virtuozzo.com References: <20261031153528.3857823-1-alexander.ivanov@virtuozzo.com> <20261031153528.3857823-5-alexander.ivanov@virtuozzo.com> Content-Language: en-US From: Andrey Drobyshev In-Reply-To: <20261031153528.3857823-5-alexander.ivanov@virtuozzo.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-ClientProxiedBy: FR3P281CA0112.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:a3::15) To VI0PR08MB10656.eurprd08.prod.outlook.com (2603:10a6:800:20a::12) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: VI0PR08MB10656:EE_|MI7PR08MB837237:EE_ X-MS-Office365-Filtering-Correlation-Id: 4f8a6652-6497-4652-e9d8-08df38fe1617 X-LD-Processed: 0bc7f26d-0264-416e-a6fc-8352af79c58f,ExtAddr List-Id: svt-core@virtuozzo.com X-MS-Exchange-Transport-Rules-Loop: 1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016|23010399003|5113699003|10067099003|18092099006|18002099003|22082099003|261102123227099003|56012099006|4143699003|261026121527099003|261022105027099003|261022075727099003|261026140227099003|261009223027099003|261009221227099003|261028144727099003|261021123027099003|261021121427099003; X-Microsoft-Antispam-Message-Info: Ht9JCkGHfUdlQnN2vHl94GX5CWrgdiPN4IqzZ28jlxt6PHDk3SgpvdDdhg65Y8MRN9rjSJs7x5ROKV15uttAguhZ+GkitrZBPBXf+ZZexorNXUN+5ZvJSM2QIlfuGPOd2DBZMXwMFOZ0rq4TbmmIFO2NK5jSX5A3mZN9LFiW8WMFp0PFJ46E8Koxdn4h+t+bJTmHsmu/Qd+m9QoY79LG/kW0tO7xSUQxBfIwaTI0gN6AL/6zgm+kRqNGAjMjShZkgxelNFr3VPR0t+MzxOX5GVAgnw39WSPEz0npX3UBL4s3SVNuEa2+fmwSnuyPYrE+D92K/zI3fw9Xe/FJwaSnLx/wHu3j9GszMjxcG7HM50HjINTp18IJHBLqj/AUcK4d+Kgk+Zb/JhcEagdD+pyfIPNgTqHGYD/+hSj7V0yF6U8CvOQS0flAdchEErnbx1S+v1fYTCt4sVS0ru/PD92E5X0H1CDkih0kM8RIMg+AgPWcmpOXdWIHsqOVsoryp+gk4c/tl3gfs07qK6fWBoKKEYmWfyD6JR061ihUutpftJmgC4GA68q1r22EReoCasn91JgJuAyz/aCeiwaESyBjsd+guRhQUHKvfSZZTChFEYj4Je7eaSspS3gZkpRzUVoL X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI0PR08MB10656.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016)(23010399003)(5113699003)(10067099003)(18092099006)(18002099003)(22082099003)(261102123227099003)(56012099006)(4143699003)(261026121527099003)(261022105027099003)(261022075727099003)(261026140227099003)(261009223027099003)(261009221227099003)(261028144727099003)(261021123027099003)(261021121427099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RDg5ck5jWlEwN2MvcWhtNERBMzFGdkVaV01LTWFyZXZ5OHdsRjJHaDUrQTNi?= =?utf-8?B?Mjc2ZTBhNm14ZnJsbTZDcEFIUllKUGxrSGRkc2MyK3hVS0UyQVE3WGVkNzR0?= =?utf-8?B?NGlVNG12cERLOTFMbk9LVlVvZ3FPM3JFbSthdkgzUXY4dHduRXVEbUQ5QWlQ?= =?utf-8?B?UWJTakNYMUpWSlR2dW9ZWUJoUmRlQzE0YndKcWNRRnB2aEZDbXliL01hZk9j?= =?utf-8?B?bDlmQTNQcDdzSys3MS9sdkJVSGoyS0U2ck51T1JlbnpqQnNmZEo1ZHF0SC9N?= =?utf-8?B?ZDFQM05meUlEbndTemJ6WGpLNEk0NW1tZUU1MFA0dXdMNFNyT1FKRmFpU2lX?= =?utf-8?B?aFZ0SUtGa0FhTElSYlFQUnNraklSRW0wYzUwMGxiZGlENmZ6UUhUOTZUWGp5?= =?utf-8?B?NEJFMWNoZHlLcjlhMTJwRndYVDlEV203d3l5UEt3dlhKYTRodHlROTh4V2JT?= =?utf-8?B?Z1R0U2RBQXFvNWRYWGlhNXhTRnhwZXByeXRieHpoRlhGR0w4cU83Q0hsUldp?= =?utf-8?B?ckZQSUVkQlBRdkRBbExWWVo3MzM2Lzltcmw0cUQrc0FIYXZLd2x1dVdhMzdv?= =?utf-8?B?aC8ycU04SGUreEVGcll2MWpoeWpFejhTaHNiM3lDZmNHckhDa1gvWmt3dTBZ?= =?utf-8?B?ci9xY2JiNmRtc2g1Qm55ZW9NT2FTSnFBR2prMFI1RUoydjc4eHZDd2JBSG5o?= =?utf-8?B?QXBKRGtrZGJ5c3dKYnVISTduejVUMVBETFhaNE9rV0dqS3BBUGJtOXVoNDRm?= =?utf-8?B?eXRjQmF0YUllNGJndXBianlDbWJQOWp5ZVJsVzN5SXVCOVNnWW1Ydi9tNERE?= =?utf-8?B?cVVSclpJV0xRYVFuTnpxN3prQ2ZWRW1yVUtYWEQ1RklObmlXRWNMbGFMVnBT?= =?utf-8?B?UVNxaDROY0RSNFlpdUpKcjl2RmJ2VjZJbGtZekJUREc0LytGSXV5Yy9NQ3RF?= =?utf-8?B?Q0Jxa0YxK1hDcG1DN0ZyUE5DbVBSQmIybExvV3JIT1dycFNZUFVJYVAzcHha?= =?utf-8?B?WGVMQmRybU52Z2tDaWZlQ2dPeHBkdG5SaWhpdkgwMU83VktJeGwvNll2cURy?= =?utf-8?B?WEZGVjBRenkySW85YldYQ3ZiSFJzN2xINmVrTXhyR2dkZTF4c3oxK1F0RFNw?= =?utf-8?B?UDlVNjhVSkM0QXVnM3NnaHQ5VnpPY0NrMlJzdTl2dlFsOGRVRGNnT0dzRDVw?= =?utf-8?B?OWNkVlNpOWwxN2crRlZYVGcwanFqUHJleW94ODkwaVU5RTNKeFp0TkVESEV5?= =?utf-8?B?OFkya2hpTDZPbVYxMFU1WEowemlsdDQrM0cwTVVQenltalZuSnpPVkNuc1hi?= =?utf-8?B?TmF0cmp4ekloY1BlQTJPT3FZNUZNenlGSEVtZFMxNDBFRDVwVEZKVkJSL21q?= =?utf-8?B?K050ZzJub21WWWJOQTMyNGE3b0VVNjJlYXdNd1ArcGZVdG9rdy9YY2tGYjdl?= =?utf-8?B?RmJXVVp3TW93QU03QWNpY3lIQ2dOc3dYdnF2cm4zUWE0VGd4RWJ4QjhNNzBY?= =?utf-8?B?bXFlTmFDdUV6d3dIMXlvb3pBT3hTdHdYUTZjbWdzSG1EendEdUpmZnpWQTdL?= =?utf-8?B?cWdLY2FnUHkrazcvcW54d29FUmJkOWtIeUhFY0ZFSU9Cb0o5ODBnNVVKK1Jh?= =?utf-8?B?Sm5MU3dMM2ViZE5BdmthNTdXN3RQSHlWc2xiYjl1bFQwczFBeEZhMWo5bno0?= =?utf-8?B?RU5nKzhqYngzTEJHV0hyaHdTMW83QTVnZjBZT2ZQU2dabWR0OHZHcDRmaHRG?= =?utf-8?B?eDE5cEV3cHFjdk5xcm5DSFR6MlVoaisyR3ZhaFFtRzRldE1MUDVhRzBab2pK?= =?utf-8?B?Tno3YlpiVWtCdVRhVVMyWHpzd2dmR3RxT3pIeTNqeDFHcW53a09yYUdtMExX?= =?utf-8?B?MXVuQVBKb2xZRjV1emN3WU5KMlFxdHlieEloMGhqN0tpSEZ0Z0NidFMxblA3?= =?utf-8?B?cnBtVmNiNStCM0ExQ0xJN1J5aFBtOEhpRzVISEVOTXdxT1ZvQVBvZi9NdXRh?= =?utf-8?B?cXBnTTYrUHdVTUJwb3NNWEVYWGRNZi9jdzdDM21aVnI0cEwvVS92R1dieHMw?= =?utf-8?B?NnQ3Umx5N3hlamdoRzJ0bDEvSFBhWjdJaFBlUTNaMTlCOHBLRk5IcmRPVFRZ?= =?utf-8?B?NDVxT2lmN1ROMWJsMll0alNEbG9FVElqaW1iaVdvY1NoT0h0cWxkZSt3NW8x?= =?utf-8?B?MkQwdmRIdlFJZWdadzZqYVM5YndrZ25wRFF2RGRkN2xlQko5Z1VDVkd0MER5?= =?utf-8?B?b3VjNkV2aTkxZm9nQmYwb1BVS2JBQ21sZVpKYW9LN2lMWkc0bUVuYjV6K0s3?= =?utf-8?B?Z0ZqNGluZmlnSWducWM2aHJIQW5lcG4ramt2RE5RZDkwdk9rNExTOWtmOWo3?= =?utf-8?Q?LDgzyoD11RXXyiNQ=3D?= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4f8a6652-6497-4652-e9d8-08df38fe1617 X-MS-Exchange-CrossTenant-AuthSource: VI0PR08MB10656.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Nov 2026 16:26:45.4679 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 2ah+FF3acVKbzQVOCDvv6PNlkLMDQbxLtnZsCcNTi5JiZal0uxdvfApQQQA9upgoPzUBDLxwjHfiQwnPnhstwsbEcn+hl6SWOICFMFZaLL8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: MI7PR08MB837237 On 10/31/26 5:35 PM, Alexander Ivanov wrote: > num-threads was passed to our own VHOST_SET_NWORKERS kernel ioctl. The > kernel drops it with the revert of "drivers/vhost: add ioctl to > increase the number of workers" in favour of the mainstream worker > ioctls, which vhost-blk now supports. Create num-threads - 1 workers > with VHOST_NEW_WORKER and attach them to the virtqueues round-robin > with VHOST_ATTACH_VRING_WORKER, the layout VHOST_SET_NWORKERS produced. > > Our kernel, unlike mainline, defaults to kthread mode, where > VHOST_NEW_WORKER fails with -EFAULT rather than -ENOTTY. So switch the > device to vhost_task mode with VHOST_SET_FORK_FROM_OWNER before > VHOST_SET_OWNER. Do it for num-threads=1 as well, so the default > thread is of the same kind whatever the number of workers. > > Fail to start vhost-blk if any of these ioctls fails rather than > silently run on fewer threads than requested. Released vz10 kernels > have VHOST_SET_FORK_FROM_OWNER, but their vhost-blk does not handle > the worker ioctls, so num-threads above 1 now requires a kernel that > does. > > https://virtuozzo.atlassian.net/browse/VSTOR-107537 > > Signed-off-by: Alexander Ivanov > --- > hw/block/vhost-blk.c | 62 +++++++++++++++++++++++++++++++++++++++++--- > 1 file changed, 58 insertions(+), 4 deletions(-) > > diff --git a/hw/block/vhost-blk.c b/hw/block/vhost-blk.c > index b3eb181ceae..3cde12051fb 100644 > --- a/hw/block/vhost-blk.c > +++ b/hw/block/vhost-blk.c > @@ -33,6 +33,50 @@ > #include "migration/cpr.h" > #include "system/runstate.h" > > +/* > + * Spread the virtqueues round-robin over nworkers kernel threads. Thread 0 is > + * the default worker VHOST_SET_OWNER created; vq 0 and every nworkers-th vq > + * after it stay on it. > + */ > +static int vhost_blk_set_workers(struct vhost_dev *hdev, int nworkers) > +{ > + uint32_t ids[VHOST_BLK_MAX_QUEUES]; > + int i, ret; > + > + assert(nworkers <= ARRAY_SIZE(ids)); > + > + for (i = 1; i < hdev->nvqs; i++) { > + int w = i % nworkers; > + struct vhost_vring_worker vq_worker = { .index = i }; > + > + if (w == 0) { > + continue; > + } > + > + if (i < nworkers) { > + struct vhost_worker_state worker = { }; > + > + ret = hdev->vhost_ops->vhost_new_worker(hdev, &worker); > + if (ret < 0) { > + error_report("vhost-blk: unable to create kernel thread: %s", > + strerror(-ret)); > + return ret; > + } > + ids[w] = worker.worker_id; > + } > + > + vq_worker.worker_id = ids[w]; > + ret = hdev->vhost_ops->vhost_attach_vring_worker(hdev, &vq_worker); > + if (ret < 0) { > + error_report("vhost-blk: unable to attach kernel thread to " > + "queue %d: %s", i, strerror(-ret)); > + return ret; > + } > + } > + > + return 0; > +} > + > static int vhost_blk_start(VirtIODevice *vdev) > { > VHostBlk *s = VHOST_BLK(vdev); > @@ -52,15 +96,25 @@ static int vhost_blk_start(VirtIODevice *vdev) > return 0; > } > > + ret = hdev->vhost_ops->vhost_set_fork_from_owner(hdev, > + VHOST_FORK_OWNER_TASK); > + if (ret < 0) { > + error_report("vhost-blk: unable to set vhost_task mode: %s", > + strerror(-ret)); > + return ret; > + } Note, with this newly introduced strict retval checking: > long vhost_dev_ioctl() > { > if (ioctl == VHOST_SET_OWNER) { ... } > > #ifdef CONFIG_VHOST_ENABLE_FORK_OWNER_CONTROL > if (ioctl == VHOST_SET_FORK_FROM_OWNER) { ... } That means any vhost-blk disk will fail to start on an (older) kernel which lacks VHOST_SET_FORK_FROM_OWNER. This shouldn't be an issue for us, but still worth keeping in mind. E.g. our vzkernel 6.12.0-55.* didn't have this setting AFAICT. > + > if (ioctl(s->vhostfd, VHOST_SET_OWNER, NULL)) { > error_report("vhost-blk: unable to set owner"); > return -ENOSYS; > } > > - nworkers = s->conf.num_threads; > - if (nworkers != 1 && ioctl(s->vhostfd, VHOST_SET_NWORKERS, &nworkers)) { > - error_report("vhost-blk: unable to set number of kernel threads"); > - return -EINVAL; > + nworkers = MIN(s->conf.num_threads, hdev->nvqs); > + if (nworkers > 1) { > + ret = vhost_blk_set_workers(hdev, nworkers); > + if (ret < 0) { > + return ret; Pre-existing, but each return after VHOST_SET_OWNER above leaves the device in a stale state. This might deserve another fix, separately from this patch. > + } > } > > if (s->conf.serial) { LGTM overall. Reviewed-by: Andrey Drobyshev