From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from GVXPR05CU001.outbound.protection.outlook.com (mail-swedencentralazon11023089.outbound.protection.outlook.com [52.101.83.89]) by lore.virtuozzo.com (Postfix) with ESMTPS id 42E1180E7E for ; Thu, 29 Oct 2026 10:12:56 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=zBVtMFn3u8aagdjAMGGuqimVm6lu97ERlRoYKf3gjF5mAJ2acC8QIBOA3KQOrARqY7VLtXHWzrioSX45zhS82P9QqcLc8zFtGwhhglzlJkMHK6j3AkIGaDwWfnIgZ58NjjTIq5XHsWfLIZfKfl4d/5tHG2GmdzY53bOXqKwotYOceXBNlmwXHQKrnVJixVNNPwCAR//NqV+4IkJsgw8kio1jyR5707c2gIyn13CMolitD8gso6QVp1iaj8xKoGK7BU5wVGn8vD6Xl9XoNjaKsurfz3y2vZuu9kKJYE1cE/rbesI/2Nh39sqD388JgJsHp5ZX1J9MeM0Hp76eDJwz6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=GfsNyJHhWELDPplNzmWt/pfkO1+uHbQECShTbsMefdM=; b=RDcAdLg0WcQIXkK+J0e9u3lj4RFthpEdsxKpSmahVNDBeKAS5GRGlX4219hbOqDBvDdO5mAtPfWY4vKf3D6LErkeYGr1WlrfA5fq1f9HmfE7e87eGwA2y41HCdwIbAeh/0LCMzYk4DPAsQistQcAB0jqV2xCq35ZLFLyJRVXFpvUpb4nY4d+ARAvCp3EAUY272qUkonXvP68Z8EI7W6n7qZdbTBQJ4UJWrPvYghtuqRB6BB+N5nvBDeI59x4qXCyVhEj5gR29TJ1gYNVXh2KUed2c9Zl1i4Wso9F7B9NpIziDHYs2WISm37e3q8uv9iYAxRCtSSFeCUNXjCcABGxSA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GfsNyJHhWELDPplNzmWt/pfkO1+uHbQECShTbsMefdM=; b=bhNq2ca0mE3r8PSWT68py1sghYALBW/Nd9jgc3e7X1kCFpkkDST+qXaEcgEmAaHd/5qpkPLDE73YJg1CIfrBS1EOmZZA/cEmct/RyA+egmw3MLAPu9VpfUwhXM/ztUUkUW+PqzqSsJUQv85fPI+cE6POvbjfKRnGMNx0yoSOkrBtZqFsBllvM4f3ViQNDwU+rQnjGbAsMQ3W1PsSgEjazvz2Hkxl3qiLoYK4jxj+XuwS5h5c9kJW1AMLwM4e/B6nBo8848ANUKeJR2jP8R4tpwVpR1S76xLlvbPyYpggxIZAGadmAwXfRgGfR3omCpxujf9eSRDDyK3jq43l6wJpLw== Authentication-Results: mx.microsoft.com 1; dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=virtuozzo.com; Received: from PA4PR08MB6061.eurprd08.prod.outlook.com (2603:10a6:102:e9::13) by AS8PR08MB6294.eurprd08.prod.outlook.com (2603:10a6:20b:29a::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.569.8; Thu, 29 Oct 2026 10:12:52 +0000 Received: from PA4PR08MB6061.eurprd08.prod.outlook.com ([fe80::c3bf:e107:b1d2:6a00]) by PA4PR08MB6061.eurprd08.prod.outlook.com ([fe80::c3bf:e107:b1d2:6a00%3]) with mapi id 15.21.0569.007; Thu, 29 Oct 2026 10:12:52 +0000 Message-ID: <9d7c0998-6825-421b-b110-1c7d6df4ff17@virtuozzo.com> Date: Thu, 29 Oct 2026 11:12:51 +0100 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH hci-8.1 v4 4/5] vhost-blk: create kernel threads with the mainstream worker ioctls To: Alexander Ivanov Cc: andrey.drobyshev@virtuozzo.com, svt-core@virtuozzo.com References: <20261029092301.3286535-1-alexander.ivanov@virtuozzo.com> <20261029092301.3286535-5-alexander.ivanov@virtuozzo.com> Content-Language: en-US From: Andrey Zhadchenko In-Reply-To: <20261029092301.3286535-5-alexander.ivanov@virtuozzo.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-ClientProxiedBy: FR0P281CA0212.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:ac::11) To PA4PR08MB6061.eurprd08.prod.outlook.com (2603:10a6:102:e9::13) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: PA4PR08MB6061:EE_|AS8PR08MB6294:EE_ X-MS-Office365-Filtering-Correlation-Id: 83ed5026-a03b-468c-bab1-08df35a53168 X-LD-Processed: 0bc7f26d-0264-416e-a6fc-8352af79c58f,ExtAddr List-Id: svt-core@virtuozzo.com X-MS-Exchange-Transport-Rules-Loop: 1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|366016|1800799024|23010399003|5113699003|18002099003|18092099006|22082099003|5023799004|56012099006|4143699003|261026140227099003|261021123027099003|261021121427099003|10067099003|261026121527099003|261009223027099003|261022075727099003|261022105027099003|261009221227099003|261028144727099003; X-Microsoft-Antispam-Message-Info: wMVv/vLY0ORSkXh+Vjd2EhV5KUM7Cen4m/3gaHifN4W3bkjbhJY8VB73YN2aNbdk/9kU0ewmTyLNZBFeGt15VurhnP4UKy2NUsHvSO9zIbhyU+7/tagZlhJ9ZGRr2YLu2ggvnemRDCuLQHkXenfug/K3RlUK8Dz23qiZqasxDcesUVu6KKwDJ41lU2EaMXirc3rlSCPuVGVSjXEE/YE6NL2nkDZLcD8ire7DkpuvIz2cFW6k6qhENsG1KYzbCE93wP8RffkmmKyfsi/M1v1KmOma9CFYuI3xa/KuNMVbmjtZ3rZuY3I31J+qiuJ3qckQwLnnozK4B3MrqtPUajWwPmyjP4Vyr/rvLhpRLcndSCyxfsZBuxjz4hgCC+EaFtSg27IUuBupCWEdNRb0WZyc8sRDdXG3mT/4ueGWKHaoTcyREQlkKM9miL1KUFdVTq4534KSAOFiwCzfb+AOqowE5JCXbLUkzDZ04XkM5FLxTRW4LFGf8deOjeDiQ0L3bhkGuWQ0tvbvYNA93pg2YCm8OeoEKITZ6btD1TeswTY2C3bqvgvqNuu2NiFTzb5tXDKE66XwbANtrZrGw0KBJleVAB9Ruta6+DPibE4TluDebc+1of16zck9GDsSfUvRcxVm X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:PA4PR08MB6061.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(366016)(1800799024)(23010399003)(5113699003)(18002099003)(18092099006)(22082099003)(5023799004)(56012099006)(4143699003)(261026140227099003)(261021123027099003)(261021121427099003)(10067099003)(261026121527099003)(261009223027099003)(261022075727099003)(261022105027099003)(261009221227099003)(261028144727099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?a0FTelI4MHZNSkxhd3FGSWZObXJnZ3ZVamdZZzVRUllqZ0NBYk1LVWxOWDBz?= =?utf-8?B?RU1JeUZrZ05UMjhtcTdiVVVVazk0Z2xwV1lud1NqN213bzh6dmEvWHdjQ0FP?= =?utf-8?B?dXlyQWZ4NmhvVFc1UWtyVTBNcnFWS0o0cFAyU1NKZjUzci84WWRqVEZqaEF4?= =?utf-8?B?bVZ2dW1nMWl0S1JkejUzSEdITzRUZWRzY0RNZE0vSHVPQUxodTNLanJXZUlv?= =?utf-8?B?QjdZRjgwelFUTXdyWllCSFVBTm9SSThsNGk2R05wMTZhQUFIa2xMeUdLeHB0?= =?utf-8?B?ZWRsUUZnQ1BKOWpma1QrYjNQU2tJOG9QTEhmSS9tR3kzT0hNR1d2bFYvbHdS?= =?utf-8?B?djcwc2lkbjArSlYreHRWeDFsTVhRUzRWcWVxYUhnWGlZT0lnQWViYmRSZ3FZ?= =?utf-8?B?cUV1dDVqYWw4K1JYYnFYYTlPb3IyM0tRR0tFbWN6ZmNQZmJGZEV0WGNTTlhm?= =?utf-8?B?MXFKbGs2dDNuVmYzUHhlNjMxdHNJYkNna25VVk45MThPeTNGWGRQNW1nYzNI?= =?utf-8?B?ekdzOVpOcS9lY1lDSElJNjBKaU5XY3ArSVorMTd5ODc5WXlUcHIyYkMvTWxs?= =?utf-8?B?VnllZEpVdkhVbHh5bXpPL2d4R093YjNsUDU3dlFJbzNVWTlmOHBjNVJ2dTdH?= =?utf-8?B?RHROUWNnMTNLQzdmaXZrRGFGZFdaTDBhVUpXb3RzblltNGllY0cvQ2tob3A4?= =?utf-8?B?R215OSt6Vkk3ZnF2ejRkeXZiRmMrbXgxM3NMTHNZQmR1aDZ6eVk3eTdjQXd4?= =?utf-8?B?N216bk90SGRjRmlVeDBFRDVPdEFMWTlodlBVMXRIeTJhSFpyT293SEdSdXU5?= =?utf-8?B?SzgydmtZcE91d052U3ZWSDVHUjZmSi92MWJSa3ArZVdOZFM4N3EwWTBZUlYz?= =?utf-8?B?Rk9VcTZvKzdlWXpBTThkS0pUWStsYy8zdmlUNDhMUGV3TTRjZzVLcTd4WTNm?= =?utf-8?B?OHpTWUVhUHlvZFBpdGNuTWQreVdZMTlUTCt5NUc1WTNXclRvZCtFUTZsQkZH?= =?utf-8?B?b3VOOGdNYnFTbUE4V0hQd3p2N1RrT05pYVR6OStNbWk1a2txeXRBSFd2TDhv?= =?utf-8?B?bkdCbTRYTkVvaGZza0FUKys5bmtQUEtMWTZyVGh3cElCWWVhOFIyMElQY3U1?= =?utf-8?B?TlFGLzJGWnk3eTM4Wjg0TnZmdjZwUUhOdHZHZkp1SmdwRFlLZU9Sa00raXhm?= =?utf-8?B?TWs5NjRUcG1leFhETkdJd1FmQXcxRXN1eWdnT0FrTmowUE9uaElWcDNIUXhP?= =?utf-8?B?L1dwTWh1Vk1EaVZrdi9KRVppNlloaWdsZytmaEFBRDUyNUtGUnpmUzF5ekE2?= =?utf-8?B?Y0ZhQXVyWW00UGZKWmwxTnlXN1lrNVl5dUcxV2VHRTc0dTB1KzNWeDJVMk40?= =?utf-8?B?Yk1udW16NTh3ZTFmd0wwYXBxM1U2WVVoYTRLdU1vV0VDOXp1bnhEM25CTVpW?= =?utf-8?B?T0VTZFg4aURHMzhEUC94clp3Znc3UUMzdi9wajFCdTJOaks1czY2UFRqRU1W?= =?utf-8?B?emxQSmFLN3VTdXVFRGhrU3YwSnZrYzIvNUhFUjZPbEt3bW03OUU5dnRybmYv?= =?utf-8?B?UDlPK3Fzd0JhM2ZUci9pYnp6YnVFcDJOZGxtR2srMGNxVlZscGxDdGNFdmRT?= =?utf-8?B?MkJOWGxGSlBGNzhJM2w2aXhNYVNpSWp3bWRiWDdud21pdVBvUGlZSHdSc1NQ?= =?utf-8?B?RUxNRVgrM1VGQnorOTdqZjIvVFYvMzRZZG11Y2pQeENnRUQ1RXpXVjArNDNI?= =?utf-8?B?OE9iWHpnMlFPMUE1WWV1cHRtTEg0T0pMWnhMM05kTDAyZ2hVeXZUOHE0VkNJ?= =?utf-8?B?cy9ucHpFMlRkWXFLOUFqUFhYYm4xaXZIS3lKOU9jVE8zY1NSYnp6S3djUlpv?= =?utf-8?B?UzJ2ZFB3OGRXK2QrY2VoT3UwN2RqRzd6LytkZGI0U0UvalVjaXpydmxnOTRU?= =?utf-8?B?S1NTdjcwRXdrcnduckZhTURwYzZaNC8zcTdtYThqazFhUVIvWXdOZGcrUlNm?= =?utf-8?B?REpCZklnRVNDdWV4TmY1VlFXall0VU0xTFU1ME9XZlV5SmtKTG4zbk84OTNK?= =?utf-8?B?QnRQMVRhazlXMXIwUklYUTd0ajFLTU0vY085WnZibXVwUiswR2wzOVVvTWJW?= =?utf-8?B?WWwvaDNjV29wMHdnb24yY2hpc1BXTk9Rc3NXQWcySmZOU0lyRlhJNlk1Y2Qx?= =?utf-8?B?TnFzYmYvZkJxc1NoT09aWTlVbVBCUUp2Q2lJT3FjQVgrZEduM0N4UlU1eGpP?= =?utf-8?B?UUZ3ZStuTnhWV0trYzIrRjNtaWVmdlh4dk9VTGdtWW14VHFkckd1VU1CeTdp?= =?utf-8?B?NVB0NmdOQjRpK3pZbmkvN2dMc1YvS2NKNWdjb0VDQlBXMHdSbU1JUUh1Zzlk?= =?utf-8?Q?JzyN4MPp4/3xFUgM=3D?= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 83ed5026-a03b-468c-bab1-08df35a53168 X-MS-Exchange-CrossTenant-AuthSource: PA4PR08MB6061.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Oct 2026 10:12:52.6121 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: fUFhR4Vw28DLkAGw2NGr+Wz5h09Oyd0Ac7sTfLYFv6w9EBlUZ8cXfeB5ls+8hXDcVFIa6fJLH8RIrCaZTaCNXtW0IudjUf8P4d979Eo5AAc= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB6294 I like VHOST_FORK_OWNER_TASK, but I would rather avoid treating worker creation as non-fatal error. This will go on silently and we will never know. Let's just fail when any of the IOCTLs do. Also let's set this mode for any amount of workers. The difference is more or less negligible but we will have uniformity across nworkers and better worker isolation. On 10/29/26 10:23, Alexander Ivanov wrote: > num-threads was passed to our own VHOST_SET_NWORKERS kernel ioctl. The > kernel drops it with the revert of "drivers/vhost: add ioctl to > increase the number of workers" in favour of the mainstream worker > ioctls, which vhost-blk now supports. Create num-threads - 1 workers > with VHOST_NEW_WORKER and attach them to the virtqueues round-robin > with VHOST_ATTACH_VRING_WORKER, the layout VHOST_SET_NWORKERS produced. > > Our kernel, unlike mainline, defaults to kthread mode, where > VHOST_NEW_WORKER fails with -EFAULT rather than -ENOTTY. So switch the > device to vhost_task mode with VHOST_SET_FORK_FROM_OWNER before > VHOST_SET_OWNER. With num-threads=1 nothing changes: no worker ioctls, > kthread mode. > > Released vz10 kernels have VHOST_SET_FORK_FROM_OWNER, but their > vhost-blk does not handle the worker ioctls and fails VHOST_NEW_WORKER > with -ENOTTY. Other errors are possible as well, so do not key on the > errno: if VHOST_SET_FORK_FROM_OWNER fails, keep all virtqueues on the > default thread; if VHOST_NEW_WORKER fails partway, spread the > remaining virtqueues over the threads created so far; if attaching a > virtqueue fails, it stays on the default thread. Either way warn rather > than fail to start vhost-blk. Unless VHOST_SET_FORK_FROM_OWNER failed, > the default thread stays a vhost_task, as the mode cannot be changed > after VHOST_SET_OWNER. > > https://virtuozzo.atlassian.net/browse/VSTOR-107537 > > Signed-off-by: Alexander Ivanov > --- > hw/block/vhost-blk.c | 64 +++++++++++++++++++++++++++++++++++++++++--- > 1 file changed, 60 insertions(+), 4 deletions(-) > > diff --git a/hw/block/vhost-blk.c b/hw/block/vhost-blk.c > index b3eb181ceae..9416c3ac07e 100644 > --- a/hw/block/vhost-blk.c > +++ b/hw/block/vhost-blk.c > @@ -33,6 +33,53 @@ > #include "migration/cpr.h" > #include "system/runstate.h" > > +/* > + * Spread the virtqueues round-robin over nworkers kernel threads. Thread 0 is > + * the default worker VHOST_SET_OWNER created; vq 0 and every nworkers-th vq > + * after it stay on it. > + */ > +static void vhost_blk_set_workers(struct vhost_dev *hdev, int nworkers) > +{ > + uint32_t ids[VHOST_BLK_MAX_QUEUES]; > + int i, ret, err = 0, nfailed = 0; > + > + assert(nworkers <= ARRAY_SIZE(ids)); > + > + for (i = 1; i < hdev->nvqs; i++) { > + int w = i % nworkers; > + struct vhost_vring_worker vq_worker = { .index = i }; > + > + if (w == 0) { > + continue; > + } > + > + if (i < nworkers) { > + struct vhost_worker_state worker = { }; > + > + ret = hdev->vhost_ops->vhost_new_worker(hdev, &worker); > + if (ret < 0) { > + warn_report("vhost-blk: unable to create kernel thread: %s", > + strerror(-ret)); > + nworkers = i; > + continue; > + } > + ids[w] = worker.worker_id; > + } > + > + vq_worker.worker_id = ids[w]; > + ret = hdev->vhost_ops->vhost_attach_vring_worker(hdev, &vq_worker); > + if (ret < 0) { > + err = ret; > + nfailed++; > + } > + } > + > + if (nfailed) { > + warn_report("vhost-blk: unable to attach kernel threads to %d " > + "queues: %s", nfailed, strerror(-err)); > + } > +} > + > static int vhost_blk_start(VirtIODevice *vdev) > { > VHostBlk *s = VHOST_BLK(vdev); > @@ -52,15 +99,24 @@ static int vhost_blk_start(VirtIODevice *vdev) > return 0; > } > > + nworkers = MIN(s->conf.num_threads, hdev->nvqs); > + if (nworkers > 1) { > + ret = hdev->vhost_ops->vhost_set_fork_from_owner(hdev, > + VHOST_FORK_OWNER_TASK); > + if (ret < 0) { > + warn_report("vhost-blk: unable to use multiple kernel threads: " > + "%s", strerror(-ret)); > + nworkers = 1; > + } > + } > + > if (ioctl(s->vhostfd, VHOST_SET_OWNER, NULL)) { > error_report("vhost-blk: unable to set owner"); > return -ENOSYS; > } > > - nworkers = s->conf.num_threads; > - if (nworkers != 1 && ioctl(s->vhostfd, VHOST_SET_NWORKERS, &nworkers)) { > - error_report("vhost-blk: unable to set number of kernel threads"); > - return -EINVAL; > + if (nworkers > 1) { > + vhost_blk_set_workers(hdev, nworkers); > } > > if (s->conf.serial) {