From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from AM0PR02CU008.outbound.protection.outlook.com (mail-westeuropeazon11023126.outbound.protection.outlook.com [52.101.72.126]) by lore.virtuozzo.com (Postfix) with ESMTPS id A78458045D for ; Tue, 27 Oct 2026 14:22:37 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ZnuiDzc9Zj3qQzEGifbYhk5qnH2Yz5ZmKV/kdjRK6+vboL/EGtfl4pure8OIcji1ch0qwkbrXj/xVDxrZ+MQXqJXe1iMbvtk2TftLj+MdrNOFUVTtN+BG7lHOIwRZlnyeeSGEuhQmI5skJ5RPVx3bdVtgbAlwcnAe5PVss742IoCx7YIusqJ8G2SH78yeDy6bGklZY2yS873LFzBcNZnJy+4zM/mv9jFGuYVUR8AN5BpUs9AXiGOawQSr2EvBsLfrCfHVR/L2Nx/7FyR2zib9Xu/uFX0pBpxMLdlZTV9fwaZE1WI41vJUaB9hloHv4U0yNaFcqrEWfw/DErnwi21OQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ZtEhCC96V2rUsAhBQ9tZ7AU7Fla/Bl0bOFurGZ6DIik=; b=zANrtmGZdlBAbf+4JswSnYnEtWDTtlb193I3NLvkBIZs+6MX/TSc+eU9K10oYwTyezYo8UU18Mk8sV0XYPpx5n2k2W06q8x3XGgbp2kHtVESmBvFHXdWBar2kBoOQ9GJIjAlqIM4PeZaIq660TLjO81BIWmg0xOeh7qF98+ZThLNutj0UczccgwHoN/3tO9DBDzXDiaS5Q1kDcOuIMoGPD6+o0xWgPfazE2RBjbMKhaVMZeWTn0K3CIXU/uqCikfY1CHSYzHMomz95Y1MP+MAz1EgqudKBrBE1lf+fx7uHY5yiC3zQfhCRZpSBOAejl975f3CtM7ZMOkZXOyQ77TLg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZtEhCC96V2rUsAhBQ9tZ7AU7Fla/Bl0bOFurGZ6DIik=; b=R0eBoDVbPmzbxYuQX7GDNkHPpCIUdhYyZ6LApgBSXH78QyIXfjsQcX20I+5cnCGBWMxDXIxBEiAn1wkwbnH/5e4pUUMQ0SQuy2T2Dv2eLHIqXEojXDbkoDEJ1hH6Nzj3TPRuE4dLad325OQ4viAjiihktbg6f5tiQLdkEb/ow18nleDNK9QGdXWhlXhVvsMdfv2im4luJj4DSXpca2sAGh/kc9XGxvAO0CQjIZhTycMJ1dIV8din918uv6ZzEv5VQsaGBv6iudUr3U1f23kR4WJVGn+ARFhPwLRGf1fcxT2KWx6WDHEhd70zFt71l7JSLages0JMvuaS4OHIsq1nWA== Authentication-Results: mx.microsoft.com 1; dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=virtuozzo.com; Received: from VI0PR08MB10656.eurprd08.prod.outlook.com (2603:10a6:800:20a::12) by GV1PR08MB8498.eurprd08.prod.outlook.com (2603:10a6:150:82::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.546.22; Tue, 27 Oct 2026 14:22:33 +0000 Received: from VI0PR08MB10656.eurprd08.prod.outlook.com ([fe80::4e37:b189:ddcd:3dd8]) by VI0PR08MB10656.eurprd08.prod.outlook.com ([fe80::4e37:b189:ddcd:3dd8%7]) with mapi id 15.21.0546.021; Tue, 27 Oct 2026 14:22:33 +0000 Message-ID: <1b58bb6c-b384-4925-bdfa-a2021c59a2bd@virtuozzo.com> Date: Tue, 27 Oct 2026 16:22:32 +0200 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH hci-8.1 v2 1/2] vhost-blk: create kernel threads with the mainstream worker ioctls To: Alexander Ivanov , andrey.zhadchenko@virtuozzo.com Cc: svt-core@virtuozzo.com References: <20261027124759.2979043-1-alexander.ivanov@virtuozzo.com> <20261027124759.2979043-2-alexander.ivanov@virtuozzo.com> Content-Language: en-US From: Andrey Drobyshev In-Reply-To: <20261027124759.2979043-2-alexander.ivanov@virtuozzo.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-ClientProxiedBy: ZR1PEPF000077DD.CHEP278.PROD.OUTLOOK.COM (2603:10a6:918::41c) To VI0PR08MB10656.eurprd08.prod.outlook.com (2603:10a6:800:20a::12) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: VI0PR08MB10656:EE_|GV1PR08MB8498:EE_ X-MS-Office365-Filtering-Correlation-Id: 7e290800-1ec4-474a-12c9-08df3435bde8 X-LD-Processed: 0bc7f26d-0264-416e-a6fc-8352af79c58f,ExtAddr List-Id: svt-core@virtuozzo.com X-MS-Exchange-Transport-Rules-Loop: 1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|1800799024|376014|366016|261026140227099003|4143699003|5023799004|56012099006|261009221227099003|261021121427099003|261022105027099003|261022075727099003|261009223027099003|10067099003|18092099006|22082099003|18002099003|261026121527099003|5113699003; X-Microsoft-Antispam-Message-Info: LH+uNy41FVVuWw/pBPFcBNVCVN2FbZzOSixg94ELtwAPXesW4bfHk3HkZVmSrG6eVrsAcPoYwAQV8B3YEoHG4MSfQsGx7HvubHARKKLbEcd2mHUPJDh1p/CYgaNVWQI8h0VjCTaAynx5JRsc0Glw2NEJRa86gMo4dnSprH8ezTZAu6H7izkuuW+gW8mtrHqBe1hvbFr3cWJgmPXiWFPXsOtE3RgZNZEMqRj9qvVNjLdlYnzB+w25U8th3txtX+gxWqnUxHwUKRe3r+8nEw5jPjVd3mfg/LyqBxFn97hjrxy78rbQ11O/pvHp+6+VHuNhe2cL90zGjBghE3E/3xSi4PbLuMaUoTsUhL9exauua/LLIGLtqPdv3C6XiN1avYEgeB59ha5ntuJbLpHgTcRyf3tmZmhAZUvVpefDrSh3gqkeytwu71qbBsCd1dNVN/e95spEVwy6XfU+tIICi2LztD+0j1msM683QrTNnOOA2duAW25CgAMjIpz4U5g5SFRwIJ1zRUymr7KQuRjnchtjr9HofDmk08SZYv1k9axw7mEHf++BfVMAn2lR+w1UiSNDw7RzJS19AqXa56J6h2Y4Cy8EX0VAUCQPqnKiOy6r6gxCzsnbgF259MKj8LmJ/mdI X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI0PR08MB10656.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(1800799024)(376014)(366016)(261026140227099003)(4143699003)(5023799004)(56012099006)(261009221227099003)(261021121427099003)(261022105027099003)(261022075727099003)(261009223027099003)(10067099003)(18092099006)(22082099003)(18002099003)(261026121527099003)(5113699003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?UUpWdWpHZ2tyOE1zaDdaTjlzd0dCYVM3dmdWbVdjRjRkQ0dYYmZSMHlON0Fz?= =?utf-8?B?OEN1Wk9xK0FmWGIwbnEwNnp5SjY5UlgyNlY5NVpFTHFoY3JGOCttQW9Ed0lm?= =?utf-8?B?TG5EeDZBdjlyU2FOTmwwTFpqNURsK2kzVS9kTE4vUVhNTmJadjIraUIvVUh4?= =?utf-8?B?YXhKOHpPV2xwTU5Vc2F1OGVqZU12V0ZFL0NGK2Y4cnpNc0pMcVRjM0hIS0Rz?= =?utf-8?B?U2xZaCsxa01TdEhwZ1ZrVW5MTTYyMS8ybERpRW5VUnBRU2RNeFhBcW44R0Q2?= =?utf-8?B?ZTFibE05SGpkSm1yMGhwWEphdWdMcU9xTTFjeXhjRkN6MThZLzdUUUJPMTU2?= =?utf-8?B?NHBtUDRFQ2U2a0lXcHkxRFVMUHVxeWMxNDJWTDZ2NWJYb3kwMCtCVlRSdHBP?= =?utf-8?B?SWs1VlJoMzNCR2F1dWN1Rmp4TTVpMHFSTlRKSVh4bWFObHBkTVJ2U0ZDUzJK?= =?utf-8?B?MGpzK2Y2bEJjYnB4VFFod2kvMXI5cmxtZXEzM25RSWtEcnVNVEVsZ3p1ZHRN?= =?utf-8?B?UU92UWVWd0lKNWtZaUVPeGZSemFVSWJsbVBYWEZxTnZXdXV0N3R2L3JWaGZ6?= =?utf-8?B?Zm53TFlibTdoaHlCcFZkK0Q5OWc5djAwN0hFbjdDRksyRXVONjJSSWgwa2pW?= =?utf-8?B?WVBGNlI0d2pERmFUdlJlWXlyREJSdm1Eb2dIUFNzbXRsZUhXbDJtOUM2czZN?= =?utf-8?B?TlhJK2FDa3h4UTdXS0ZCbUYwNWdLUGVSRnZ1S0hncWZYY0xtT0h5SW1PbzNV?= =?utf-8?B?aHh4NE9xL3NKOTN6d0dRVzJEZTFIUXhFWFZDYUsxTFZ5UXY5WnhVVlVQRjlJ?= =?utf-8?B?SVNqSW10ZlBhZHNjRWtXb2E2UjdtdnltOGZkNzFUNEhiOTJxdTNZRWtpd2dF?= =?utf-8?B?VWxkUHZUSDh2Mk1hWkNoazhoSWhiVVNRdnpzYVVNMi96Zzhtb3dRcmorSTAy?= =?utf-8?B?L2xrNWNjNEFRenBxT3dyYzQxNzZLbFlSRWxMOTRFWHY4Rkxwc3EzM0QzSkw5?= =?utf-8?B?dExPamllTjY5eUhlanFNQmlJYTVhNUVoaitFVzMwWnhmdHB6VXZGRzlUS0x5?= =?utf-8?B?czNJZ0ZCaE54WlJrZG1UNU1iVEYwNk9QbG1XdXIwcW5wZkdGTmVIUjRHWVlx?= =?utf-8?B?ZStCQjRMbW53dTlXcG8wS0RvOVN0bTBIQWRZRWlCOEY3anpJaEdnWWhWUXQ0?= =?utf-8?B?MzNjWm90b1YxeTV5YzhtaFBVbTJocGVjNVIxMW5pc0xzamU0TVU1M1VuUE01?= =?utf-8?B?WmxkN09obmtGWnhHMUwxWi9kQ05NQytzaStWaGQrM2dnb0I4S3N4SEVNL2pK?= =?utf-8?B?SzRXMGZ3STFaVUxXTDB2cllyNjZCVmlKZUtBZE4waENFc0JhaHVQTGFmQzR5?= =?utf-8?B?eXVxUE9mdFhwNkk5K2Z0SXltQTExeWR3c3NlejhvR3lNK2hIU05Xdkp1UUhq?= =?utf-8?B?NlJsbU5ObGJsV1RCMGthcnZBR04wcW8rb1REUHVRd3BrVkFaV2NKZUUxM2NV?= =?utf-8?B?aldBQmVlenNuNDYzb01TdGFDZjF3c0s4aGpjZVRYVE1pMkxUVVRTUWFRYk4r?= =?utf-8?B?c01WOVpVRTVNSkgrSm1MbVE3NmswTnE1eWY4bUkvL3JhVEoxQ1U3OTU1TmNl?= =?utf-8?B?enZOVWljd0xpMWRhNlUvNVR6b3VCY05aNlhoSXRSQ1BMWjVLdW00YmZCMlh6?= =?utf-8?B?M1B0aUlzQ0ljYXFqTWxDakFESTFWWFRtNWpoaUY1eXhmVDN0bFRHcVQreW9D?= =?utf-8?B?YXh4eTlUdHpIdnh2cThxeTBKSk9FeU85MHR3NkMrYkl3RFlHR0hGU2JDeVh0?= =?utf-8?B?b0ZzN3pLcVN3QU54SU9DcjVxSG95R3BpWGlqdjJ5bHNGQlZmWE5uczE1SjhY?= =?utf-8?B?Y21HTkxhYjlrQnZCbWRHVG9qa2hGMmJuSHFVUlhQQWlNeG81UFNSKytJZTdr?= =?utf-8?B?SnNCUVhKVFczODcrM1lZa1psbXA3anRmbmk2bGsvRHVOUnFwWVNnc3dvR01F?= =?utf-8?B?WFhRaWloaFFabzk2ai8wdG1vbFFRMUdTMXFtSmpHNjZMcFBwWFRzMlJaUHh0?= =?utf-8?B?K1R3eHV4STdpNFBwc1Exc29iRzlVTjY5dFdOWG04KyszT2R0K2kzbUIzWmto?= =?utf-8?B?R0dtUnkzcE9ncTZFMGMxL2JUTlo1eFpHNnUvR2MvVWJKRWErbGExZlBZSno3?= =?utf-8?B?bXRwQWdrSUVyOFpybTJvbFV0QkhYeUZldTlyRjBaUkVwWkFhWWp4Y3dZVjRL?= =?utf-8?B?QmpsZjlTMzh4TXBQTEgyTDhtK0NRc2lOQXE0V3VZUEpRbTVpcndCQ05sdTNn?= =?utf-8?B?Vlpzam5TSFR6TUc1bGxjc1ZwTWRRSkw2dnIraVd2QkwwYktlcFFzbUlsbGJ0?= =?utf-8?Q?KMKogfvEvmv0Lu6s=3D?= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 7e290800-1ec4-474a-12c9-08df3435bde8 X-MS-Exchange-CrossTenant-AuthSource: VI0PR08MB10656.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Oct 2026 14:22:33.5276 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 9WatrEVJwyQzdcK++hPKE1ltEK1G9PjuHURLEXAOjW4FKSRN/SXd/2kW4fxtKcL9hM3xP/b41WmkTnJOZFsbzCqM7sKGIoyiqwsUAA92kmc= X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV1PR08MB8498 On 10/27/26 2:47 PM, Alexander Ivanov wrote: > num-threads was passed to our own VHOST_SET_NWORKERS kernel ioctl. > The kernel is dropping it in favour of the mainstream worker ioctls, > which vhost-blk now supports, so use those: create num-threads - 1 > workers with VHOST_NEW_WORKER and attach them to the virtqueues > round-robin with VHOST_ATTACH_VRING_WORKER, the default worker being > the first one. This is the layout VHOST_SET_NWORKERS produced. > > VHOST_NEW_WORKER works only in vhost_task mode, while our kernel, > unlike mainline, defaults to kthreads, so switch the device to > vhost_task mode with VHOST_SET_FORK_FROM_OWNER before VHOST_SET_OWNER. > VHOST_RESET_OWNER restores the default, hence this is done on every > start. With num-threads=1 nothing changes: no worker ioctls, kthread > mode. > > On a kernel without these ioctls warn and run a single thread rather > than fail to start vhost-blk. > > https://virtuozzo.atlassian.net/browse/VSTOR-107537 > > Signed-off-by: Alexander Ivanov > --- > [...] The overall logic is fine, but I'd like patch to be simplified. First, there're dedicated vhost_ops->vhost_new_worker() and vhost_ops->vhost_attach_vring_worker() calls. We should reuse them. For consistency, let's add another patch "vhost-backend: add vhost_set_fork_from_owner op". Smth like: > /* hw/virtio/vhost-backend.c */ > static int vhost_kernel_set_fork_from_owner(struct vhost_dev *dev, > uint8_t fork_owner) > { > return vhost_kernel_call(dev, VHOST_SET_FORK_FROM_OWNER, &fork_owner); > } Other issues (by Claude): 1. Dead fallback in vhost_blk_set_workers(). The ENOTTY && i == 1 branch handles a kernel that has VHOST_SET_FORK_FROM_OWNER (6.17 / vz10) but not VHOST_NEW_WORKER (6.4) — no such kernel exists. Once the fork-mode ioctl succeeded, a worker-ioctl failure is a real error. Drop, do a single fallback point in start(). 2. Fork-mode block in vhost_blk_start() is hard to read. A declaration inside the if, the ENOTTY test nested under 'ret != -ENOTTY', the 'nworkers = 1' fall-through at the bottom. Switch to flat three-way if (ret == -ENOTTY) … else if (ret < 0) … at the call site, ioctl plumbing in the backend. 3. Redundant clamp. MIN(nworkers, VHOST_BLK_MAX_QUEUES) is already guaranteed by nworkers ≤ nvqs ≤ VHOST_BLK_MAX_QUEUES (realize, :666); a clamp hides an invariant. Switch to assert(nworkers <= ARRAY_SIZE(ids)). 4. memset() + field stores for two structs per iteration. Switch to designated initializers, and the two loops merged into one: thread w is created by its first user vq (i < nworkers), then attached — same ioctl count, one error path per ioctl, loop body reads as "which thread, create if first, attach". 5. s->dev. repeated throughout. Use struct vhost_dev *hdev = &s->dev; once, as vhost-scsi does; vhost_blk_set_workers() takes hdev and no longer needs VHostBlk. 6. Commit message nits (1/2). "The kernel is dropping it" needs the kernel change/ticket; say that VHOST_NEW_WORKER returns -EFAULT (not ENOTTY) in kthread mode, which is why the mode switch — not the worker ioctl — is the capability probe; "default worker being the first one" belongs in the code comment, where it already is. Here's what the changed version might look like: > /* > * Spread the virtqueues round-robin over nworkers kernel threads. Thread 0 > * is the default worker VHOST_SET_OWNER created; vq 0 and every nworkers-th > * vq after it stay on it. > */ > static int vhost_blk_set_workers(struct vhost_dev *hdev, int nworkers) > { > uint32_t ids[VHOST_BLK_MAX_QUEUES]; /* ids[0]: default worker, unused */ > int i, ret; > > assert(nworkers <= ARRAY_SIZE(ids)); > > for (i = 1; i < hdev->nvqs; i++) { > int w = i % nworkers; > struct vhost_vring_worker vq_worker = { .index = i }; > > if (w == 0) { > continue; > } > > /* vq i < nworkers is the first user of thread i: create it here */ > if (i < nworkers) { > struct vhost_worker_state worker = { }; > > ret = hdev->vhost_ops->vhost_new_worker(hdev, &worker); > if (ret < 0) { > error_report("vhost-blk: unable to create kernel thread: %s", > strerror(-ret)); > return ret; > } > ids[w] = worker.worker_id; > } > > vq_worker.worker_id = ids[w]; > ret = hdev->vhost_ops->vhost_attach_vring_worker(hdev, &vq_worker); > if (ret < 0) { > error_report("vhost-blk: unable to attach kernel thread to " > "queue %d: %s", i, strerror(-ret)); > return ret; > } > } > > return 0; > } > > static int vhost_blk_start(VirtIODevice *vdev) > { > VHostBlk *s = VHOST_BLK(vdev); > struct vhost_dev *hdev = &s->dev; > ... > int ret, i, nworkers; > ... > if (s->vhost_started) { > return 0; > } > > nworkers = MIN(s->conf.num_threads, hdev->nvqs); > if (nworkers > 1) { > /* > * VHOST_NEW_WORKER needs vhost_task mode; our kernel defaults to > * kthreads and VHOST_RESET_OWNER brings that default back, so ask > * on every start. Must precede VHOST_SET_OWNER. > */ > ret = hdev->vhost_ops->vhost_set_fork_from_owner(hdev, > VHOST_FORK_OWNER_TASK); > if (ret == -ENOTTY) { > warn_report_once("vhost-blk: kernel does not support vhost tasks, " > "using a single kernel thread"); > nworkers = 1; > } else if (ret < 0) { > error_report("vhost-blk: unable to set vhost task mode: %s", > strerror(-ret)); > return ret; > } > } > > if (ioctl(s->vhostfd, VHOST_SET_OWNER, NULL)) { > error_report("vhost-blk: unable to set owner"); > return -ENOSYS; > } > > if (nworkers > 1) { > ret = vhost_blk_set_workers(hdev, nworkers); > if (ret < 0) { > return ret; > } > } > > if (s->conf.serial) { > ... Andrey