From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from AM0PR02CU008.outbound.protection.outlook.com (mail-westeuropeazon11023104.outbound.protection.outlook.com [52.101.72.104]) by lore.virtuozzo.com (Postfix) with ESMTPS id 89CCC8045D for ; Wed, 28 Oct 2026 15:31:11 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=pUJ+tyn7KKE51cVmDZur5gK3VyNq+2ZkioEe0g5qTCWJw6qR5jOQd8QJSjmdH6Ob+ZwmSVD29/dp8cIWBhZ1wyuW3kSmiKP9rMnb4Fcz0PTI+lKR6t6yN48Q199vQtQx6Rv0P6zTxLKdaC29SHGQdOCdSpHcitun1L8N3Zabyje6w4Jl9Nhsv7AVfu8ryVSRkFcDtogCE9CHc3NlvodtrzPtWTms9QhG0SnESw8ykJBU3XYqZDfw6gKTiWiUoTy3pDydFPfxT1Eb+xIA0o8GXRIYennIOGtOgw0s9kvxeT3Cl0kr+wTQqEi9s+Dph9yrOuQJE7KGPzGPDpt4fiNrVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Vn34QfrFH6c/R1aoxJvMOXej0Q64jTgjLwbZBqD+a+k=; b=GAJ6K/jEk9GOugKAI+EW1cUiHwWCYLo8zT1hIinNOeigVMVJbdsAt1QKSWSWDloa1eZxcEZh0Ms2qBLPdfACuWeItkJ2mdeNPP7fW0neNMBZHlAMi6FMzRprmxt/YjqNR00d5/W/dfAr3dIQkBoIv6vfmDwSlgo5PbUNXeLoYKkYBUi6etG8DIHgDe0njlQGTRmi9MlPoN/0PvUxImwls6wkkzm94xtvDM9o0PRR6xI2dnAvWxYFstuVk/F9u9N+syJZ67ER6L8+zokbcImJQwJ6jqB4ozNaiKvncSAO0EvdvJv9TnVvuubRAvZjMA5P1cDF3ZGl3tQYDr+vQaY/hA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Vn34QfrFH6c/R1aoxJvMOXej0Q64jTgjLwbZBqD+a+k=; b=v74hXKY+XDHx/pKYmwavTNJWxC4xgs+URAdmn4KdXRciOMwYJwNy80RjyjrNPA0BaQTau4xXK9SruD88cL3iHPCUFxalB+UN0D+n+0k3ueRM7KWM0WT7V2/81LC7XWuWTc2OT3pST9uu3x/X6gUW4dJCeKs0G/vxSBP2jOGDj213iu4z3bHfsMBLGpN9+EO0/BsgOZmREA7l5ye0j9yPEiIVmhp3owGJXXkP6ZrtdHy8dVV4uDUhHGwwu5KW145wWS9+KaEeI+XzdZL+VENOZ5pOkyNDnXgHmu2DNNzW6p1Gyai9m5MO/4FbfcJ4rNk2Mg4rfhksM+2E9n0pTiZLgQ== Authentication-Results: mx.microsoft.com 1; dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=virtuozzo.com; Received: from VI0PR08MB10656.eurprd08.prod.outlook.com (2603:10a6:800:20a::12) by BESPR08MB489008.eurprd08.prod.outlook.com (2603:10a6:b10:12d::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.546.22; Wed, 28 Oct 2026 15:31:10 +0000 Received: from VI0PR08MB10656.eurprd08.prod.outlook.com ([fe80::4e37:b189:ddcd:3dd8]) by VI0PR08MB10656.eurprd08.prod.outlook.com ([fe80::4e37:b189:ddcd:3dd8%7]) with mapi id 15.21.0546.021; Wed, 28 Oct 2026 15:31:10 +0000 Message-ID: <6cef8611-7c90-4a9d-9a0b-de65c559be89@virtuozzo.com> Date: Wed, 28 Oct 2026 17:31:10 +0200 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH hci-8.1 v3 3/4] vhost-blk: create kernel threads with the mainstream worker ioctls To: Alexander Ivanov , andrey.zhadchenko@virtuozzo.com Cc: svt-core@virtuozzo.com References: <20261028123333.3145676-1-alexander.ivanov@virtuozzo.com> <20261028123333.3145676-4-alexander.ivanov@virtuozzo.com> Content-Language: en-US From: Andrey Drobyshev In-Reply-To: <20261028123333.3145676-4-alexander.ivanov@virtuozzo.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-ClientProxiedBy: FR0P281CA0116.DEUP281.PROD.OUTLOOK.COM (2603:10a6:d10:a8::18) To VI0PR08MB10656.eurprd08.prod.outlook.com (2603:10a6:800:20a::12) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: VI0PR08MB10656:EE_|BESPR08MB489008:EE_ X-MS-Office365-Filtering-Correlation-Id: 9074b9ea-41ec-48d5-de45-08df35087e31 X-LD-Processed: 0bc7f26d-0264-416e-a6fc-8352af79c58f,ExtAddr List-Id: svt-core@virtuozzo.com X-MS-Exchange-Transport-Rules-Loop: 1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014|23010399003|261026140227099003|5023799004|4143699003|56012099006|261009223027099003|261009221227099003|261022075727099003|261022105027099003|261021121427099003|261021123027099003|261026121527099003|22082099003|5113699003|18092099006|18002099003|261028144727099002|10067099003; X-Microsoft-Antispam-Message-Info: Pa9wKt6azph8gL2aLj00Pi9Hqvo5rG88HumqFjG4R7S+tyBwlMZFoXWXG0z6v0D086xlC2gX61kQHUwlyYWsRt1/G2h/a6G6zOzxeaEHAANRWMVeE1YXtPtIDFCHVAG72b5o3tGj57HI/k7vv44UY9FwrYooaWb/sX544DLyfY5ynq4O14baHrBhLk8Ii8wlzFJNN5pVVrnZBchYM+4U2vrbur5q2aUgy7i7qvQRdFVIoNI9/lJ5DKHsfTYhKrmTlbyE578cJmbBWcLqgs462MJg83Gcd3vI8c84e+pWGtuOcs1kDw2TEaYyuDoghWtWAc6UPfI2rClJjDsTvQHfa4SEgS5QSdQl+PuLHMcMwuIi62kJAQEPE5oy0usp8+wXuLlac/MtupSUFdqBLXPxGFqFP2qcRw+/KbyFBZJ1vdJzIvXQ2rQLUBSAdoJGSp0dP4B+zh9lbZ6L2ki7viBUbAEoAMVnQRfTGsbtdT52GxX2SNYgTT2giPfzcCpwv85R2EuL0y/RgF7bcxFiPFBcfnObiHPUXxI1vqQK4qC050OGBnKt9VTX0zKLtHJz8ctJ83cdWrFWuuT1q+YAoksUkotyKtns4SLHBKovhmLWKsEcZSdsbRfT6tvcolTg62d7 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI0PR08MB10656.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014)(23010399003)(261026140227099003)(5023799004)(4143699003)(56012099006)(261009223027099003)(261009221227099003)(261022075727099003)(261022105027099003)(261021121427099003)(261021123027099003)(261026121527099003)(22082099003)(5113699003)(18092099006)(18002099003)(261028144727099002)(10067099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?WjBNdlllYXVzTTNNNnBGM1hYbVZ1d2JheUJoNWtRRUNpY01uNWlqOXNXYWpn?= =?utf-8?B?djM5OFlSWC8yWkFkU0NwbXM3RDNGYzlQZFBwMmplTm5pUG1MUXhsSFVFcjBI?= =?utf-8?B?THU2Z0VYT3p3TTRVcnFiRThTVXNFWGVqL0NtMlJidGNuQkU3RGJTYVgzVGls?= =?utf-8?B?MVVXSVN2QVYxM25nZU12enFnSFFBZ1ZSM1JmaFBjaTJnZHNXNUJMV1I4WmhX?= =?utf-8?B?dnkrNERrS3JuRVl6STdlRlowaVVGb3ZuMXlMZzZRQkJsWitMRFVqTzZ1aGIy?= =?utf-8?B?NHliVitsakdDdk5tbmd3YmZIaHpkM2hDR1dsWGFNUWhqSUNzeGJqU3d1Q3RD?= =?utf-8?B?c3RFbi9yMUd0a3hDYkVaUFMwandqdjZGNTFxQUNzOFVhdllMRHgrVktGbHVY?= =?utf-8?B?UTEwLzZNMGgwRDJYcUVGNFhyTFM5SGVhT2cwNGtnQTNaWVBtNXVjVDhLUkVI?= =?utf-8?B?aW9oMWJQV2I2QTFSUEVvdS9GMDkzZUhBeGI0d0dUekp0QmVpaFROcEFWcms1?= =?utf-8?B?OEI3K29xYXdSTzlmWnVDWXA2NWFqTzhiNkZJSDNJZDh3UllqcHpLWEcyWXcw?= =?utf-8?B?SHJWUXJOY01KZDlscnJ5OTZBZ25YQTdRNTYrOUxTemNZMjhWQ1VGd21PdkZS?= =?utf-8?B?SDhHZlBpLzZkcWxMSE1SanUyVjJLdXNIN2ZTVWlldUFQL0MvdTd5cnNDVDQr?= =?utf-8?B?eWdENEF1U0hKWjZnKzhFVzZBa1BiL3MvRWk2VHJ3RDhzcndOWG9QM2Qrak4v?= =?utf-8?B?OEZGZXE3TEZheWtadEFuL1VBbXlORitCeitmY0podm9CZ1BpeXErd2Nvamp5?= =?utf-8?B?Q0VqVGR2dlpudGdGaEh4WEFYcXhHRk5va1k0a0JHZFRxakxWd2VvSVV0TFhl?= =?utf-8?B?NzNCRFlReFZWWnRmYndhWEQzRHBlb1FUWVdKV1JGOXNVMkdva2pCMW9qSCtj?= =?utf-8?B?emUxbDZFeDNzVEM4UnpGNGZPQ2lwUlZxVno3UnJlZUJZK0VJS2V0ZlV6SkZ2?= =?utf-8?B?WnpsVlRTZXVwTGlYazZ2M1MrN0VRamwrOW1FbStJOTVrQjNrNlA5QVJWNi9Y?= =?utf-8?B?NnBIbjFjSHpWU3NMMjVGUHBjOXBjZXRlR0hlUnU5ZGk4WlVaZlJ5Tis0S2RK?= =?utf-8?B?eGU2azdNY2hYZys3b0FVaFNZVEVxcnd2ZnpibFIyNG92WTNKajQxTVQwL24y?= =?utf-8?B?QjVGT0pyQXJvS2JDdFpIRDVtckl2RDlWVkJnU0dJQkZrOGlxR3lTbXhvelg0?= =?utf-8?B?V1FMeUQxMjJnMVFhZ2QzbHRvaFRzNUd2eS83Y0R2Z0ZKcVFGQUVNMExpelVo?= =?utf-8?B?KzNZU2thdGFCbDBPL3VVNVJ6UmI0YWQzY3h3YWxtU0NlSUpIbE1Fc2NyNzY2?= =?utf-8?B?MTl6b0pJMkJ5OXBTTG1JUDhqMndzTGpmSlR1bm1CWnpVSFRLdTVFTkVGQnYr?= =?utf-8?B?b2d2cU00RFVab1ZJelFUWm9Yb2NqaXlTSSszTTROM20xbHpobEtMc3AzckJL?= =?utf-8?B?bEZpejA3MXNOUXZ0VXVWMldkT1Y1VjlRTTFHMFNqYWpEaFl5NUgybUJDaC8y?= =?utf-8?B?T21abFphdFZGaldJb21GeXFjNVFLK3Yxc0ovc2lzd0YyUERvbU9HL2MzQmY3?= =?utf-8?B?RWp6SEEwOTNsa2h0RU5zTjRNWW5PZVhQcDRjNzBwaTNVV1p5My9HY1EvMDJl?= =?utf-8?B?bkZONVJnNS8weTJhYWUzNFFLenczeFJ3SzR4d0xEbHJqRVhvYWpnU3FtUjRl?= =?utf-8?B?cnhyTlV6Nlk0WkVJUnpSZUNTZ1JaV1p0blBRbTFFT3NIL1A3QUx6YlJIcTZu?= =?utf-8?B?b01hMWgrKzFwSUtWd3BXcmsxY2ZhMXFSd29uZCtIVitGYUoyekE2UEdXRHFP?= =?utf-8?B?VmFRVkwxaE9TRVNTemJhcmE0OFMxTFRXTWt4cFptdWh1clIwbWxVcGVoaDVO?= =?utf-8?B?cUpiNWphVnNJRnU0WUVIcXNkUlFETlZQdkkwdk1HV1N4ODhOWWp4K3ZaaVhv?= =?utf-8?B?cHdVeGpidUhzM3pwMGU0dkZEWnRyV0YyRVlDcTVJT2RHazlKUjgrOW5ObzlX?= =?utf-8?B?cDVsVnFYdXUwN29vTG1IdjNsQWhadnltUU1XUzVDc0FscU5MRlVGSmdGN0E4?= =?utf-8?B?bzNuUEVoYzFxMEJzaXVFdk5tZEo1UlF6UzFGMVFVb2IwR0VJVnBpNm1Zam5z?= =?utf-8?B?Rnk4RSs3VW1sckNzejJtbVpUKzA5UnlTSytKYndBdVFqay9SQ01haUZrdEpY?= =?utf-8?B?Y25rZG1KSEk3eWZReGF2OFVXSi9wNFRxc2tCdDhENkk1NzNOVTFhNXhiNDBE?= =?utf-8?B?UFhCV2FnNnlrcGF4eDFCSFlYTjFPbEZnWnE3SEhwaFIyM2FlNHYxS1J1Yndi?= =?utf-8?Q?PgC7QnGpGIApfbRU=3D?= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9074b9ea-41ec-48d5-de45-08df35087e31 X-MS-Exchange-CrossTenant-AuthSource: VI0PR08MB10656.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Oct 2026 15:31:10.4078 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: RKLgUwGA1reNPrnN38x84HngNXJKE+Uza+6A/Y2RTO97Ry89YUW8xKTLm6v81HC+wW4Lwl2T2I/odNphRLo2LBsFoU53OBGkQUOwJiD84QE= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BESPR08MB489008 On 10/28/26 2:33 PM, Alexander Ivanov wrote: > num-threads was passed to our own VHOST_SET_NWORKERS kernel ioctl. The > kernel drops it with the revert of "drivers/vhost: add ioctl to > increase the number of workers" in favour of the mainstream worker > ioctls, which vhost-blk now supports. Create num-threads - 1 workers > with VHOST_NEW_WORKER and attach them to the virtqueues round-robin > with VHOST_ATTACH_VRING_WORKER, the layout VHOST_SET_NWORKERS produced. > > Our kernel, unlike mainline, defaults to kthread mode, where > VHOST_NEW_WORKER fails with -EFAULT rather than -ENOTTY. So switch the > device to vhost_task mode with VHOST_SET_FORK_FROM_OWNER before > VHOST_SET_OWNER. With num-threads=1 nothing changes: no worker ioctls, > kthread mode. > > Released vz10 kernels have VHOST_SET_FORK_FROM_OWNER, but their > vhost-blk does not handle the worker ioctls and fails VHOST_NEW_WORKER > with -ENOTTY. On such a kernel, or one without > VHOST_SET_FORK_FROM_OWNER, warn and run a single thread rather than > fail to start vhost-blk. > > https://virtuozzo.atlassian.net/browse/VSTOR-107537 > > Signed-off-by: Alexander Ivanov > --- > hw/block/vhost-blk.c | 100 +++++++++++++++++++++++++++++++++++++------ > 1 file changed, 86 insertions(+), 14 deletions(-) > > diff --git a/hw/block/vhost-blk.c b/hw/block/vhost-blk.c > index 8a015d54fa7..cc05eb057df 100644 > --- a/hw/block/vhost-blk.c > +++ b/hw/block/vhost-blk.c > @@ -33,9 +33,60 @@ > #include "migration/cpr.h" > #include "system/runstate.h" > > +/* > + * Spread the virtqueues round-robin over nworkers kernel threads. Thread 0 is > + * the default worker VHOST_SET_OWNER created; vq 0 and every nworkers-th vq > + * after it stay on it. > + */ > +static int vhost_blk_set_workers(struct vhost_dev *hdev, int nworkers) > +{ > + uint32_t ids[VHOST_BLK_MAX_QUEUES]; > + int i, ret; > + > + assert(nworkers <= ARRAY_SIZE(ids)); > + > + for (i = 1; i < hdev->nvqs; i++) { > + int w = i % nworkers; > + struct vhost_vring_worker vq_worker = { .index = i }; > + > + if (w == 0) { > + continue; > + } > + > + /* vq i < nworkers is the first user of thread i: create it here */ > + if (i < nworkers) { > + struct vhost_worker_state worker = { }; > + > + ret = hdev->vhost_ops->vhost_new_worker(hdev, &worker); > + if (ret == -ENOTTY) { > + /* vhost-blk predating worker ioctls, fails on vq 1 only */ > + warn_report_once("vhost-blk: kernel does not support worker " > + "ioctls, using a single kernel thread"); > + return 0; AFAIU Sashiko is right about failure not always being indicated by -ENOTTY. We might get EPERM, ENOBUFS, EINVAL, EFAULT. Maybe let's just stop keying on exact error: > ret = hdev->vhost_ops->vhost_set_fork_from_owner(hdev, VHOST_FORK_OWNER_TASK); > if (ret < 0) { > warn_report_once("vhost-blk: unable to use multiple kernel threads: %s", > strerror(-ret)); > nworkers = 1; > } And the same for vhost_new_worker() inside the loop. @Andrey Zhadchenko can you confirm that's the right error-handling approach here? > + } else if (ret < 0) { > + error_report("vhost-blk: unable to create kernel thread: %s", > + strerror(-ret)); > + return ret; > + } > + ids[w] = worker.worker_id; > + } > + > + vq_worker.worker_id = ids[w]; > + ret = hdev->vhost_ops->vhost_attach_vring_worker(hdev, &vq_worker); > + if (ret < 0) { > + error_report("vhost-blk: unable to attach kernel thread to " > + "queue %d: %s", i, strerror(-ret)); > + return ret; > + } > + } > + > + return 0; > +} > + > static int vhost_blk_start(VirtIODevice *vdev) > { > VHostBlk *s = VHOST_BLK(vdev); > + struct vhost_dev *hdev = &s->dev; > struct vhost_vring_file backend; > int ret, i, nworkers; > BusState *qbus = BUS(qdev_get_parent_bus(DEVICE(vdev))); > @@ -51,15 +102,36 @@ static int vhost_blk_start(VirtIODevice *vdev) > return 0; > } > > + nworkers = MIN(s->conf.num_threads, hdev->nvqs); > + if (nworkers > 1) { > + /* > + * VHOST_NEW_WORKER needs vhost_task mode; our kernel defaults to > + * kthreads and VHOST_RESET_OWNER brings that default back, so ask > + * on every start. Must precede VHOST_SET_OWNER. > + */ > + ret = hdev->vhost_ops->vhost_set_fork_from_owner(hdev, > + VHOST_FORK_OWNER_TASK); > + if (ret == -ENOTTY) { > + warn_report_once("vhost-blk: kernel does not support vhost tasks, " > + "using a single kernel thread"); > + nworkers = 1; > + } else if (ret < 0) { > + error_report("vhost-blk: unable to set vhost task mode: %s", > + strerror(-ret)); > + return ret; > + } > + } > + > if (ioctl(s->vhostfd, VHOST_SET_OWNER, NULL)) { > error_report("vhost-blk: unable to set owner"); > return -ENOSYS; > } > > - nworkers = s->conf.num_threads; > - if (nworkers != 1 && ioctl(s->vhostfd, VHOST_SET_NWORKERS, &nworkers)) { > - error_report("vhost-blk: unable to set number of kernel threads"); > - return -EINVAL; > + if (nworkers > 1) { > + ret = vhost_blk_set_workers(hdev, nworkers); > + if (ret < 0) { > + return ret; > + } > } > > if (s->conf.serial) { > @@ -70,22 +142,22 @@ static int vhost_blk_start(VirtIODevice *vdev) > } > } > > - ret = k->set_guest_notifiers(qbus->parent, s->dev.nvqs, true); > + ret = k->set_guest_notifiers(qbus->parent, hdev->nvqs, true); Nit: these s->dev -> hdev replacements generally don't belong here as they aren't part of the changing logic. Split it into another patch (maybe covering stop()/reset() as well?) and add 'Fixes: ...' tag, so I can squash it during the next rebase keeping only the meaningful history. Andrey