From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from MRWPR03CU001.outbound.protection.outlook.com (mail-francesouthazon11021135.outbound.protection.outlook.com [40.107.130.135]) by lore.virtuozzo.com (Postfix) with ESMTPS id DAC58808A3 for ; Tue, 13 Oct 2026 16:00:56 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=x26SIOQVAJA17114VcKOMc26otVJQCRuDUUJ8ha31uw1a+RRsoY4aqT5ZBKPrAktZqbbvwbY/0bWn//nTju7vMmlfGBkpJH4AOEsXbK+ivZlGOEM4lJV1WdVWBfX9QWx6GUxeQmaO3LVzBGNZd5regEZpJHc/0Vt/s4dppznFZGg8Cj+2A5AYI91E1DR99H1N7fT1U4H8OTGh6xdvlN9RLv9OBXW7Aix9laQgO8T2Ic3VuJ+3W+v7FIJwA6rkFfqQCyW5pNAKOBfcBFzssOIA37gP3xJnEFQoAUR4ihArU4EbTywmt3OqSAM9sKEtQ18GgZIzvQIpT5GHxFB5xltgg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zEYBDh4k3sVm5c7ElyynlYgnN73LnCuCk7HMJnCO9Qk=; b=hjDyDmnuCDryFgq3DaDfY/f1rN8rPmjhrcqn769uKthkpvnRoyBVUsGwTFZI5HSRajMDqtm0S6zRAXzYHe5a+6J9F4QdMsbL5lcDOaW+QROvIJ/tkmpSnHO+X/T45jxUwh2IDSt0RZ9qO6iGkg70jfgSJAIlXk4H9lEGXeO/xJnvfnpRSsZqhUx6+p1E0BF9HITwrYt5fMgTSUS8Qx1hgaF6ZtCDZpIbuSu2W+rJ4oCHragK14IswbYxtPg+cg6eNKAUEtvif/GPz8VhuNb+T9zfvROODBHp2lLJp36PowSHGn7RChXa+aTAT38IIiBANPgi3BV3QK1sEgMfKYkpIw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 69.169.224.1) smtp.rcpttodomain=virtuozzo.com smtp.mailfrom=eu-central-1.amazonses.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=sashiko.virtuozzo.com; dkim=pass (signature was verified) header.d=sashiko.virtuozzo.com; dkim=pass (signature was verified) header.d=amazonses.com; arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.onmicrosoft.com; s=selector2-virtuozzo-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zEYBDh4k3sVm5c7ElyynlYgnN73LnCuCk7HMJnCO9Qk=; b=F3+G9aTomvrULBjLsEX1FAtaQHAmkWXBBEQDT39wMoXjS+AW75bciOfE1OgRf9ijajkm0e0U9xOUokl+c2SIBM4RAwsSU3HSoSesAvzB/eYvGvRu87URUkPSWJZ839/kGDUXLT6xUvoWf8W2UrLqFUuhnr7OuAXSKh8O3CAmtoY= Received: from ZR0P278CA0061.CHEP278.PROD.OUTLOOK.COM (2603:10a6:910:21::12) by AS8PR08MB9648.eurprd08.prod.outlook.com (2603:10a6:20b:617::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.472.14; Tue, 13 Oct 2026 16:00:54 +0000 Received: from MI3PEPF00008487.eurprd02.prod.outlook.com (2603:10a6:910:21:cafe::67) by ZR0P278CA0061.outlook.office365.com (2603:10a6:910:21::12) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.496.5 via Frontend Transport; Tue, 13 Oct 2026 16:00:54 +0000 Authentication-Results: mx.microsoft.com 1; spf=pass (sender IP is 69.169.224.1) smtp.mailfrom=eu-central-1.amazonses.com; dkim=pass (signature was verified) header.d=sashiko.virtuozzo.com;dkim=pass (signature was verified) header.d=amazonses.com;dmarc=pass action=none header.from=sashiko.virtuozzo.com; Received-SPF: Pass (protection.outlook.com: domain of eu-central-1.amazonses.com designates 69.169.224.1 as permitted sender) receiver=protection.outlook.com; client-ip=69.169.224.1; helo=b224-1.smtp-out.eu-central-1.amazonses.com; pr=C Received: from b224-1.smtp-out.eu-central-1.amazonses.com (69.169.224.1) by MI3PEPF00008487.mail.protection.outlook.com (10.167.240.134) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.522.6 via Frontend Transport; Tue, 13 Oct 2026 16:00:53 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=23yjbh4k34hwzfubhuiim7m4inbd3ick; d=sashiko.virtuozzo.com; t=1791907253; h=Message-ID:From:Subject:To:Cc:In-Reply-To:References:Content-Type:Content-Transfer-Encoding:Date; bh=kyoLaE8rs1LCgozfyQ7AeF20QOeywB1A5OTzwSge/QE=; b=igpTrihJEUhLmq135Ff1BXiM/5pCYKH0ezxpRxj/VsBH+9pl8IJiurD4Lv7+uWfh pt8lj7bFyw4ApvyAw9s68rxXQK+29jt18IecAFrEAN7HW46sqBJbPQbzvImIZhqVXoM VKukGRAvG2buqj6wKcIBoIVYHXKoagrhWE8fk19bCdc/irnU1VXqGy1i0gFnGQMBPYP /T5TpSjiCMF2w8MQz8lASzGNQI3+v6baxks9dRecpN/t2t7vAFUrnKtdveA2yXCWrYG VyDLWLOXYDpl/o3VVLuHVkCzo+luQZCjvCMk6z3tyRqp7qiQlcfSvOHu3btFPOeYTc0 dRQUa1RRnQ== DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=qftdzk2dqsatjnlrq4r5brjbihpfcrsh; d=amazonses.com; t=1791907253; h=Message-ID:From:Subject:To:Cc:In-Reply-To:References:Content-Type:Content-Transfer-Encoding:Date:Feedback-ID; bh=kyoLaE8rs1LCgozfyQ7AeF20QOeywB1A5OTzwSge/QE=; b=KTwPhRts04KmK+0o6AyhVt4yPUHhoOq97+d+C5VeoreiMar1ofs2OAn7dGrCo6Ex d27oos8gdrNy6SbL3WUYd5+ldZPZQ0bpofxZz+fvWPyEDpmrqigdcUdR5UbiEn4eCzO YmSFt2mv7Pcqw8olvPRsCX/yAmIb1CgXZ6IgUzHg= Message-ID: <010701a135fecd05-db919099-27ad-4baa-9ef0-265e0a85916d-000000@eu-central-1.amazonses.com> From: no-reply@sashiko.virtuozzo.com Subject: Re: [PATCH QEMU HCI-8.1 v2 23/24] VZ: vfio/region: preserve BAR mappings across cpr-exec #VSTOR-103860 To: Andrey Drobyshev , svt-core@virtuozzo.com Cc: andrey.drobyshev@virtuozzo.com In-Reply-To: <20261013-vfio_v1-v2-23-433455c0a294@virtuozzo.com> References: <20261013-vfio_v1-v2-0-433455c0a294@virtuozzo.com> <20261013-vfio_v1-v2-23-433455c0a294@virtuozzo.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Date: Tue, 13 Oct 2026 16:00:53 +0000 Feedback-ID: ::1.eu-central-1.2wRQJBpg9g1fGRYexx3dUuYGS25Bux+2TEOx72/m6U4=:AmazonSES X-SES-Outgoing: 2026.10.13-69.169.224.1 X-EOPAttributedMessage: 0 X-EOPTenantAttributedMessage: 0bc7f26d-0264-416e-a6fc-8352af79c58f:0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MI3PEPF00008487:EE_|AS8PR08MB9648:EE_ MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: d6170e8f-8a13-4ef9-7e72-08df2943292e X-LD-Processed: 0bc7f26d-0264-416e-a6fc-8352af79c58f,ExtAddr X-MS-Exchange-AtpMessageProperties: SA List-Id: svt-core@virtuozzo.com X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|9140799003|48200799018|32142699015|61400799027|376014|5023799004|56012099006|4143699003|5063699009|55112099003|16102099003|4113699003|5113699003|10067099003|22082099003|18002099003|261009223027099003; X-Microsoft-Antispam-Message-Info: BqcYr94w8e4IDIzVLv3RSHJ3vMny6lLQHIQYp4tRU3+t5P6E+hqKJp05IzH4Lrui5lwidmCI+pgwPxVdhbrh9dt/xQpQvvluZBBLOJev5ZR51oL6EJ34vLvX1ZsSkVt23K5w+0CbSr9zUvp5ZEXoXHu3Au0o0/0c70b27s6Ws6a5ERAN0lbxt6SHBhDotFK0hxLUt7TOF3XyoZQVizRFcTZVmnDOpkSpuBeW96UVsjfznIS0QJucdLisw/1wt2D5EUOVayBfnrANrWVp8qfnr/fPGMJrsi2gnW8r4Ilcfoc7SumK1DYMbI/cmTg1sXTs04HyxU1MH+9Hmgu+8y8rrQNDH1CB/xEn9byRYvucQw6hToyuKGD4Jd362B8ycEHIxegPmudxqLqnJZF6i+2M+X8kCVsWDPhE2po0kla+g6w9+/ryfKWw4rMW8nhl948Y+GcheqUrIevEF+fQew7VH9q8iLLDZj+gui0epyzuTu9xOJsiophiROWp6t72f1ZJFHgCRlNO3Odj9F5Lv/rjFLxU/fEHVbRQlP/OqzxChyI3307WHDZWjKeM4a78o9tTAzFHY6sXkuVjrJD4eeAkFLxIf0UulFbEzQKW09EiPxOaSplfepkn+PbLcR6r/f9G0nV9TeC6xOWcMw/5eolo1Rs/yF2Is5hTIiqyz4dwciPyvJhIlGbfTtWiMy6eyzGlz2HBPUQB7YAaaqFk0HaFwHyLPxtXde4RU5v5DD/PKSof6JWDEO0IxfFF7Foc/97jDCrP+/kio4zDwiDsKsivDz76qy20+M1tiUWT1Avzr5YyGnGTEYP2LvpZfsoEOh9P7pWyahWDB/vkm/OHLKSUDzpgD5Rq+MKcoehHR/FbndttNle9lhmPcZ0Hq7CD9X/w X-Forefront-Antispam-Report: CIP:69.169.224.1;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:b224-1.smtp-out.eu-central-1.amazonses.com;PTR:b224-1.smtp-out.eu-central-1.amazonses.com;CAT:NONE;SFS:(13230040)(9140799003)(48200799018)(32142699015)(61400799027)(376014)(5023799004)(56012099006)(4143699003)(5063699009)(55112099003)(16102099003)(4113699003)(5113699003)(10067099003)(22082099003)(18002099003)(261009223027099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: +w+2M2FuOtx3/OBqNXp6UherSWFVKv2eoU5lLa0oFjSbsnkyhlg5vMEAr1q67scdblRF+CjoyNT0H2OvUYDCtabLDBmPCz0ZkMltMekv9VDEL2CLzK6H2i4BfLS6iJD51sXf8JQL1V4+g+Cb0/WlzPn8m4s/d8wFn42rQTRwpiOeMcpkwk3m1Bb7mcPLelAJpPAjZyagYCw+RLGFcgxzDn87QIUhODBYf9U0IDwynoO2GSOJTrwN2yRh6czPwroHRVEOz+cBLjpJSVldCybXcy2YIUXry29ak0tZ8OkZ3lXFamcNhTFcite+vStNYO5N7z74n0CScHqPFol3Wta9IcZ6KvBHAscbxCXZvfqxzYBopw0x/spe61ZiJeaPwdFSBe/sOPkZr1o+Cv9yTrM8t0VS5OykvmQvMek/f+dYeTo001DR/evbAg7Sw5IXhyV/ X-ExternalRecipientOutboundConnectors: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-Auto-Response-Suppress: DR, OOF, AutoReply X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Oct 2026 16:00:53.9665 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d6170e8f-8a13-4ef9-7e72-08df2943292e X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-AuthSource: MI3PEPF00008487.eurprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: Internet X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB9648 Thank you for your contribution! Sashiko AI review found 6 potential issue(= s) to consider: - [High] vfio: vfio_region_cpr_find() exits QEMU for lazily mmapped display= region after incoming mode reset - [Medium] vfio: vfio_region_mmap() adds cpr-transfer blocker with error_ab= ort - [Medium] vfio: cpr-exec new QEMU exit(1)s when a BAR was not mmapped by o= ld QEMU - [Medium] vfio: display region CPR entry leaked, vfio_region_exit() runs a= fter unparent - [Low] vfio: cpr-transfer blocker added before mmap and kept after failure - [Low] vfio: cpr-transfer is blocked for every VFIO device with mmapped re= gions -- commit 715bce8fddbc1bc88d530d34be1f40b7da065453 Author: Andrey Drobyshev VZ: vfio/region: preserve BAR mappings across cpr-exec #VSTOR-103860 This patch saves each VFIO BAR mmap in the CPR ramblock list, so that the new QEMU can reuse the mapping after cpr-exec instead of calling mmap() again. It also adds a cpr-transfer blocker for devices with mmapped regions. > diff --git a/hw/vfio/device.c b/hw/vfio/device.c > index 5a14a7d5c0a2e..88b9e8752cd93 100644 > --- a/hw/vfio/device.c > +++ b/hw/vfio/device.c > @@ -514,6 +514,7 @@ void vfio_device_unprepare(VFIODevice *vbasedev) > =20 > g_clear_pointer(&vbasedev->reginfo, g_free); > g_clear_pointer(&vbasedev->region_fds, g_free); > + migrate_del_blocker(&vbasedev->cpr.mmap_blocker); > =20 > QLIST_REMOVE(vbasedev, container_next); > QLIST_REMOVE(vbasedev, global_next); [ ... ] > diff --git a/hw/vfio/region.c b/hw/vfio/region.c > index 1deaef12fd0b8..fce09a0d17b1a 100644 > --- a/hw/vfio/region.c > +++ b/hw/vfio/region.c [ ... ] > +static char *vfio_region_cpr_name(VFIORegion *region, int index) > +{ > + g_autofree char *path =3D qdev_get_dev_path(region->vbasedev->dev); > + return g_strdup_printf("%s%s%s mmaps[%d]", path ?: "", path ? "/" : = "", > + memory_region_name(region->mem), index); > +} [Severity: Medium] The name built here depends on qdev_get_dev_path(), which returns NULL once device_unparent() has cleared dev->parent_bus. Can the save-time and delete-time names diverge for the display region? The display region is mmapped lazily: vfio_display_region_update() vfio_region_setup() vfio_region_mmap() vfio_region_cpr_save() /* saved as "/display mmaps[0]" */ On hot-unplug, vfio_exitfn() runs vfio_pci_bars_exit(), which covers only the BAR regions. The display region is torn down later, from vfio_pci_finalize(): vfio_pci_finalize() vfio_pci_put_device() vfio_display_finalize() vfio_display_region_exit() vfio_region_exit() By then device_unparent() has run, so the name becomes "display mmaps[0]" without the path prefix. Then cpr_delete_ramblock() in vfio_region_exit() finds nothing, and the saved entry stays while vfio_region_finalize() munmap()s the region. Could this leave a stale entry that makes a later hot-replug of a device with the same path hit exit(1) in vfio_region_cpr_find(), or makes a later cpr-exec try MADV_DOEXEC on an unmapped range? > +/* Same checks as for RAM blocks in ram_block_add() */ > +static void *vfio_region_cpr_find(VFIORegion *region, int index) > +{ > + g_autofree char *name =3D vfio_region_cpr_name(region, index); > + cpr_ramblock_t ram =3D cpr_find_ramblock(name); > + > + if (cpr_exec_is_incoming()) { > + if (!ram.addr || !ram.length) { > + error_report("CPR mode=3Dexec is requested, but mapping %s c= an't " > + "be reused: addr=3D%p length=3D%zu. Aborting", > + name, (void *)ram.addr, ram.length); > + exit(1); > + } [Severity: Medium] Can this exit(1) be reached for a region that old QEMU legitimately could not mmap? In old QEMU, vfio_bar_register() treats a vfio_region_mmap() failure as non-fatal: if (vfio_region_mmap(&bar->region)) { error_report("Failed to mmap %s BAR %d. Performance may be slow", vdev->vbasedev.name, nr); } The no_mmap path in vfio_region_mmap() calls vfio_subregion_unmap(), which calls cpr_delete_ramblock(). A failed region therefore has no CPR entry. The only blocker added by this patch is mmap_blocker, which applies to cpr-transfer only. Nothing blocks cpr-exec in this state, so the new QEMU runs vfio_region_mmap() again. Then vfio_region_cpr_find() sees cpr_exec_is_incoming() true and no entry, and exits. The comment says this mirrors ram_block_add(), but RAM blocks are always allocated while BAR mmaps are optional. Should this fall back to mmap() or the slow path when no entry exists, or should cpr-exec be blocked when a region could not be mapped? > + assert(ram.length =3D=3D region->mmaps[index].size); > + } else if (ram.addr || ram.length) { > + error_report("CPR mode=3Dexec wasn't requested, but mapping %s w= as " > + "saved: addr=3D%p length=3D%zu. Aborting", > + name, (void *)ram.addr, ram.length); > + exit(1); > + } [Severity: High] Can this second branch kill QEMU after a cpr-exec that had otherwise succeeded, for the lazily mapped display region? The sequence would be: 1. Old QEMU maps the display buffer from vfio_display_region_update(), and vfio_region_cpr_save() adds "/display mmaps[0]" to the ramblock list. cpr_exec_cb() preserves it. 2. New QEMU loads the entry in cpr_state_load(), which also sets incoming_mode to MIG_MODE_CPR_EXEC. Nothing maps the display region at realize time, since vfio_display_region_init() only registers the gfx_update callback. 3. migration_incoming_state_destroy() calls cpr_set_incoming_mode(MIG_MODE_NONE). 4. The first gfx_update with an active plane after that calls vfio_region_setup() and vfio_region_mmap(), and then vfio_region_cpr_find(). The saved entry is still present, because only vfio_subregion_unmap() and vfio_region_exit() delete it. cpr_exec_is_incoming() is now false, so the "wasn't requested, but mapping %s was saved" branch calls exit(1). The check was copied from ram_block_add(), which runs at realize time while the incoming mode is still set. Is it valid for a mapping that can be created after the mode is reset? Even without the exit, the preserved mapping would be neither reused nor released. [ ... ] > @@ -307,6 +352,19 @@ int vfio_region_mmap(VFIORegion *region) > return 0; > } > =20 > + /* > + * Only cpr-exec keeps the address space, a cpr-transfer target would > + * have to map the regions anew, which some drivers (NVIDIA vGPU) re= fuse > + * while the old mappings exist. Not supported. > + */ > + if (!region->vbasedev->cpr.mmap_blocker) { > + error_setg(®ion->vbasedev->cpr.mmap_blocker, > + "cpr-transfer is not supported with mmapped VFIO regi= ons " > + "(%s); use the cpr-exec mode", region->vbasedev->name= ); > + migrate_add_blocker_modes(®ion->vbasedev->cpr.mmap_blocker, > + &error_abort, MIG_MODE_CPR_TRANSFER, -= 1); > + } [Severity: Medium] Can passing &error_abort here abort QEMU at runtime? migrate_add_blocker_modes() calls is_busy(), which fails with -EBUSY and fills errp when the following holds: if (runstate_check(RUN_STATE_SAVE_VM) || migration_is_running()) { error_propagate_prepend(errp, *reasonp, ... vfio_region_mmap() is not only a realize-time call. The gfx_update callback reaches it lazily: vfio_display_region_update() vfio_region_mmap(&dpy->region.buffer) For a device whose BARs have no mmap, or whose BAR mmaps failed, the first blocker registration happens at runtime. If a migration or snapshot is running at that moment, error_abort turns the -EBUSY into abort(). Since is_busy() does not depend on the mode, would this also hit ordinary non-CPR migrations? Could the -EBUSY be handled instead of aborting? [Severity: Low] Is the blocker installed too early here? It is added before the mmap loop and is not removed on the no_mmap error path, nor by vfio_subregion_unmap() or vfio_region_exit(). The only removal is migrate_del_blocker() in vfio_device_unprepare(). The commit message says the block applies "for devices with mmapped regions", but a device whose first mmap() fails, or whose mappings were all undone, still blocks cpr-transfer until vfio_device_unprepare(). [Severity: Low] Is the blocker intended to apply to every device with mmapped regions? It is registered unconditionally for any region with nr_mmaps > 0, so nearly every vfio-pci device blocks cpr-transfer. Before this patch, VFIO devices reached cpr-transfer through no such check, since vfio_devices_present() was only consulted through migration_is_blocked()->migrate_cpr_is_blocked() when migrate_mode() =3D=3D MIG_MODE_CPR_EXEC. The commit message gives the NVIDIA vGPU case as the reason, but the code does not narrow the block to drivers that refuse to remap. Could the blocker be limited to the affected drivers or devices? [ ... ] > @@ -314,6 +372,11 @@ int vfio_region_mmap(VFIORegion *region) > size_t align =3D MIN(1ULL << ctz64(region->mmaps[i].size), 1 * G= iB); > void *map_base, *map_align; > =20 > + region->mmaps[i].mmap =3D vfio_region_cpr_find(region, i); > + if (region->mmaps[i].mmap) { > + goto mapped; > + } > + [ ... ] > @@ -410,6 +475,13 @@ void vfio_region_exit(VFIORegion *region) > =20 > for (i =3D 0; i < region->nr_mmaps; i++) { > if (region->mmaps[i].mmap) { > + /* > + * The mapping is unmapped by vfio_region_finalize(), but by > + * then the device is unparented and its path, part of the C= PR > + * name, is gone. So drop the CPR entry here. > + */ > + g_autofree char *cpr_name =3D vfio_region_cpr_name(region, i= ); > + cpr_delete_ramblock(cpr_name); > memory_region_del_subregion(region->mem, ®ion->mmaps[i].m= em); > } > } This comment is correct for the BAR path through vfio_exitfn(), but is the same ordering guaranteed when vfio_region_exit() is reached from vfio_display_finalize()? As described above, the device is already unparented there. --=20 Sashiko AI review =C2=B7 https://sashiko.vzint.dev/#/patchset/20261013-vfio= _v1-v2-0-433455c0a294@virtuozzo.com?part=3D23