From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from DU2PR03CU002.outbound.protection.outlook.com (mail-northeuropeazon11021101.outbound.protection.outlook.com [52.101.65.101]) by lore.virtuozzo.com (Postfix) with ESMTPS id 1991080E7E for ; Mon, 2 Nov 2026 11:16:49 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=aWxahf2IVmiFtPMVV20pCgbUTxrrbFFDhW8sswtEAOZ4I69fg/y+c9ng0FPW1jzO1Xrr4yGfSG2aQNFp+T7KOq6OT4lKx+oWp7dmazvqsdngdFu/THhcXASAViZMuOau0NchIC+lExVfLnCvQrAfxxE7WB5PYnvmYgQM6QoCnSge7RumuPIlGuqU+1Sq/w47VKNiChKTeCrSKKmmCGOZKzTP9/ChirdG711ec7KuVDgmUsyBcYu9UalTdNhv++oKL848pdwFCJWblZeAVHoE7WznvGuFA4cVwS5EdOiQ9J6nz4ixffmUw5jVLInWw48iXbrUnkHksJM7Ozu82ROIcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=npqMkiy8lUcaClHuwwcT6nDzd1U3sv8fNK+sOc0IRVg=; b=oCxqZScqizpI+Y51bu1wJn4vPCpW8lHa0IHLHcAMu79gq8vImYQaFsiodWBCkX/ZbyVIvM32mxGaC/Z17GOpb/83gOMfUtDb1YJ/R6p6BRW8goB9RdQF0c8L6Jxx/cnz0DFfC5krpFAc/XMYg08w1pS8qSltf3ubKQTVGR1m+Rx3qV5yc6DPOOjCajWFUTEzMbOzZOIBSpgT6ONcjmBRHGY28NLeK48OQoK97OmZ196E1J1RdE53Xw+gv8a9uz3Lbug0enNmjS7u+3/0p8iD7uesCWp7AtUz2McEus0Yec5sY2tp41qhfliILYRRbq4hUePpNcECj43Y4pAmD/FXSQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 69.169.224.2) smtp.rcpttodomain=virtuozzo.com smtp.mailfrom=eu-central-1.amazonses.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=sashiko.virtuozzo.com; dkim=pass (signature was verified) header.d=sashiko.virtuozzo.com; dkim=pass (signature was verified) header.d=amazonses.com; arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.onmicrosoft.com; s=selector2-virtuozzo-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=npqMkiy8lUcaClHuwwcT6nDzd1U3sv8fNK+sOc0IRVg=; b=Gsffl7b28yCiVP9+pAvTB2OZ2VEwzDxgpX1Fj9AMDLkM4OC8J0gQcdveBb5kyEI4p30D1CiaXnqF44TeGskeHKRHcU2lUlHtCu5FwYa8wfBbBvf3g57sZYkM4vvn9amkrpmzdDEwWKv73SOIp6c3990lOSFZ1ck8mrOHCVj0BFY= Received: from AM8P191CA0015.EURP191.PROD.OUTLOOK.COM (2603:10a6:20b:21a::20) by GV2PR08MB8366.eurprd08.prod.outlook.com (2603:10a6:150:b2::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.569.12; Mon, 2 Nov 2026 11:16:43 +0000 Received: from AM2PEPF00070CF5.eurprd02.prod.outlook.com (2603:10a6:20b:21a:cafe::5c) by AM8P191CA0015.outlook.office365.com (2603:10a6:20b:21a::20) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.569.12 via Frontend Transport; Mon, 2 Nov 2026 11:16:43 +0000 Authentication-Results: mx.microsoft.com 1; spf=pass (sender IP is 69.169.224.2) smtp.mailfrom=eu-central-1.amazonses.com; dkim=pass (signature was verified) header.d=sashiko.virtuozzo.com;dkim=pass (signature was verified) header.d=amazonses.com;dmarc=pass action=none header.from=sashiko.virtuozzo.com; Received-SPF: Pass (protection.outlook.com: domain of eu-central-1.amazonses.com designates 69.169.224.2 as permitted sender) receiver=protection.outlook.com; client-ip=69.169.224.2; helo=b224-2.smtp-out.eu-central-1.amazonses.com; pr=C Received: from b224-2.smtp-out.eu-central-1.amazonses.com (69.169.224.2) by AM2PEPF00070CF5.mail.protection.outlook.com (10.167.242.7) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.592.5 via Frontend Transport; Mon, 2 Nov 2026 11:16:43 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=23yjbh4k34hwzfubhuiim7m4inbd3ick; d=sashiko.virtuozzo.com; t=1793618203; h=Message-ID:From:Subject:To:In-Reply-To:References:Content-Type:Content-Transfer-Encoding:Date; bh=3hK/F76ru7YGx3nlwo3GCgaAT31bVssrLPHOUh56M+I=; b=gebZkdzaJiUoDtIKsjdqUv1m/qgJdIHv68gzY6AVQmX0+0qx66dlq/lYkD7+Ha3p 6Gu37Sqm/O8EhHTjCrMXNRKzU7usJrOv4ojEl7CMg0aPkxNCR9oXsBPcsQF2pCl9SNt o50KGt3tFOrC6GDia4PgeYuKmrNCGHVXzVxZvUpOByMIp7zu1HaU/vdaYGBtwQB2S5L NNjOS9FtoVvz4c/zRf9Xae97ipwSEq7A6HqDxw+qYdbmXhKKF0ks4uy2Uz+Fo1RlAp5 WB46dhFQZDOGtGUKzA+iQB1G4j9fFjCXEpkcg6NOSWlrhpKvmXKa8PA4iTV9A6PSlGw F2/2AJTf6A== DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=qftdzk2dqsatjnlrq4r5brjbihpfcrsh; d=amazonses.com; t=1793618203; h=Message-ID:From:Subject:To:In-Reply-To:References:Content-Type:Content-Transfer-Encoding:Date:Feedback-ID; bh=3hK/F76ru7YGx3nlwo3GCgaAT31bVssrLPHOUh56M+I=; b=eltDZkKycjhC+i4bMVgTfg1RyCsWrYx9ix0wi+D46csFZnObkLZQWpsRClJcgahA Eo4hjZnolhtq+GAWpsOfXhbbmTTITHhqkHEnuD3bhoIesBlfRCp0TCjT5y8aQGJqE9a yrFkiK9JZj+muCtTFC4nlIZ0iiTxrWzUkByXSKn4= Message-ID: <010701a19bf9d28e-a958f0e8-613e-4e5d-b829-ef9a7739c1af-000000@eu-central-1.amazonses.com> From: no-reply@sashiko.virtuozzo.com Subject: Re: [PATCH QEMU HCI-8.1 v6 32/34] VZ: vfio/region: preserve BAR mappings across cpr-exec #VSTOR-103860 To: Andrey Drobyshev , svt-core@virtuozzo.com In-Reply-To: <20261102-vfio_v1-v6-32-9bc7cabc850d@virtuozzo.com> References: <20261102-vfio_v1-v6-0-9bc7cabc850d@virtuozzo.com> <20261102-vfio_v1-v6-32-9bc7cabc850d@virtuozzo.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Date: Mon, 2 Nov 2026 11:16:43 +0000 Feedback-ID: ::1.eu-central-1.2wRQJBpg9g1fGRYexx3dUuYGS25Bux+2TEOx72/m6U4=:AmazonSES X-SES-Outgoing: 2026.11.02-69.169.224.2 X-EOPAttributedMessage: 0 X-EOPTenantAttributedMessage: 0bc7f26d-0264-416e-a6fc-8352af79c58f:0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM2PEPF00070CF5:EE_|GV2PR08MB8366:EE_ MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 6acfd4af-ee05-4533-869c-08df38d2c6aa X-LD-Processed: 0bc7f26d-0264-416e-a6fc-8352af79c58f,ExtAddr X-MS-Exchange-AtpMessageProperties: SA List-Id: svt-core@virtuozzo.com X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|61400799027|48200799018|9140799003|32142699015|4143699003|5063699009|56012099006|261009223027099003|261028144727099003|261021123027099003|261021121427099003|261026121527099003|261022075727099003|261022105027099003|261026140227099003|10067099003|18002099003|55112099003|22082099003|16102099003|5113699003|4113699003|13003099007; X-Microsoft-Antispam-Message-Info: WEoY5JecQcvk6jn1db3GlcJOaJoHkbHLyfmNRPKt+dHdow/mubFY7IGZ3n7K5CoYN0eYfz4nYDnF/emB81kL8sAhBzJX8jysFSq8UDngiRM3R0RLzAGv057wUa74ES4v4FMYrNQt9WVxrpJAFocR76CbnT8pbyXCHabPRBp8WIwYcYxdjDmNFZLP2/UsDiOeyXDqinXx8WxQnmByElAv/w4EpSaFFNKqDJZ1uWbxSoQmmP7YyCxqYxIDCNj5m/eNcTd431SAqgS93UBEA5QHrMh+hjBquCW0WS0hiJECM5ixjNyPDOQOY/+2YXFF29LDOMCPGs7cOtX/WBRznOYe6UqyTFlo9FJcZGhU/15wQ8/8P1AjWYGeR1w465e6kweEbfoR4rRIGnAB0/0Qj3frpTAu2D0FfjAgiFWbo+fHZysJUTMSinp9BUjA4zefW3a3KpKH9Kn8oktBOYzFAOUP4MIqiV+N/zu0HzRKPBxqXSpQoJnWpPbsUXCyfq6VEEkvWAsoPg9k/TNFcCQfuFMrH3lB1NLeqsyynTCt9N+vXorsJ+Id68Rcjfd4CxTgtCp8/1OrjXp4Bsn7BpDC2cY1KY9PRIctXLILveS9IJYlQseZ/JZfXMLcODVn48DSC4gwDZBsxqINY71b1Wci6PFtKJXQKgXUB8r9BuEbDuHsbnEZSJFvikpIV8sdYgmEMHvNBNOB16Bv+Pkb3ZzDfQYcFDKc6f3Khna8fsdDwLWZVCeWv6XljVR7z8Hg1hpj6hzfxGw62eoq5/KamKNryiQVAzkdrVNGZURZMiXy69DEERvtQhgC0pgjucGrt9l0GpBC2qYc9NVLgjs4BPRVZYmocVkNLcoeCveUqbKv4Xx6BpfWyc8WjZvQOG75Bnq8sJZEcB7FQZPiG3C6SgpW6hlHvEahLjKlNKGRjFzXQEmgexs= X-Forefront-Antispam-Report: CIP:69.169.224.2;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:b224-2.smtp-out.eu-central-1.amazonses.com;PTR:b224-2.smtp-out.eu-central-1.amazonses.com;CAT:NONE;SFS:(13230040)(376014)(61400799027)(48200799018)(9140799003)(32142699015)(4143699003)(5063699009)(56012099006)(261009223027099003)(261028144727099003)(261021123027099003)(261021121427099003)(261026121527099003)(261022075727099003)(261022105027099003)(261026140227099003)(10067099003)(18002099003)(55112099003)(22082099003)(16102099003)(5113699003)(4113699003)(13003099007);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: anEf/ez20gwub45cAsAsTwHD9H40VpPupX+KNdpA/F6/WIqcfT/IEKjpDt7Q6lX7qIdqWNesEHGPlFPg6AItMQzYJ3ZeLy8QBxk7NjKDjbsWvv5EyEJJwQ63gB9/0vg3nempf1KCkxKNbvOMbwLFCbGJNJiQUzytdKh02FmpNVnf29yN0Apu6cNa+1I2uIJq3br4GpcCOQNDbpcaX2U4YD4+arMc5zn8qKXwxKklB2swCUgZnciffe7aNIU2YeUDsqti67A5UIAHHTRI9dsKTTcAE6CRNEGsdJJ42IDD4hd2/i8mfV9uBEQd0IFR2bPNmg6uBsn3ZtnUc7yS+udASBQxYyWSwvBWLPrG5yhHdk2tr4ft/BKe+/KX5PQ1NyY/8gCIEkgznN5qBHPP3UuVkTcZANhuY7ndmb2Cwryza5gu8clr3QFMYZfInZXeEtmq X-ExternalRecipientOutboundConnectors: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-Auto-Response-Suppress: DR, OOF, AutoReply X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Nov 2026 11:16:43.6511 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6acfd4af-ee05-4533-869c-08df38d2c6aa X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-AuthSource: AM2PEPF00070CF5.eurprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: Internet X-MS-Exchange-Transport-CrossTenantHeadersStamped: GV2PR08MB8366 Thank you for your contribution! Sashiko AI review found 3 potential issue(= s) to consider: - [Medium] vfio: unconditional cpr-transfer blocker added in vfio_device_at= tach() regresses working mode - [Medium] vfio: cpr-transfer blocker is skipped for vfio-user devices - [Low] vfio: BAR mappings in CPR ramblock list lose PROT_READ-only protect= ion under -cpr-ram-rdonly -- commit 6269c4e3bb1402af4f2335050cc5373497aec6be Author: Andrey Drobyshev VZ: vfio/region: preserve BAR mappings across cpr-exec #VSTOR-103860 This patch saves each VFIO BAR mapping in the CPR ramblock list, so cpr-exec carries it over with MADV_DOEXEC and the new QEMU reuses it instead of calling mmap() again. It also adds a cpr-transfer migration blocker to VFIO devices in vfio_device_attach(). > diff --git a/hw/vfio/device.c b/hw/vfio/device.c > --- a/hw/vfio/device.c > +++ b/hw/vfio/device.c [ ... ] > @@ -464,12 +464,32 @@ bool vfio_device_attach(char *name, VFIODevice *vba= sedev, > TYPE_VFIO_IOMMU_IOMMUFD : > TYPE_VFIO_IOMMU_LEGACY; > =20 > - return vfio_device_attach_by_iommu_type(iommu_type, name, vbasedev, > - as, errp); > + /* > + * Only cpr-exec keeps the address space and so the region mappings, > + * see vfio_region_mmap(). A cpr-transfer target would have to map = the > + * regions anew, which some drivers (NVIDIA vGPU) refuse. > + */ > + error_setg(&vbasedev->cpr.mmap_blocker, > + "cpr-transfer is not supported with VFIO devices; " > + "use the cpr-exec mode"); > + if (migrate_add_blocker_modes(&vbasedev->cpr.mmap_blocker, errp, > + MIG_MODE_CPR_TRANSFER, -1)) { > + return false; > + } [Severity: Medium] Is this blocker too broad? It is added in vfio_device_attach() for every VFIO device and every iommu backend, whether or not a BAR is mmapped. At the base commit, cpr-transfer with VFIO was not blocked. The vfio_devices_present() check was only reached under: if (migrate_mode() =3D=3D MIG_MODE_CPR_EXEC && migrate_cpr_is_blocked(e= rrp)) The existing cpr-transfer notifier registration in vfio_cpr_pci_register_device() and the cpr-legacy and cpr-iommufd transfer blockers and notifiers also become unreachable for VFIO. Can this regress cpr-transfer for vfio-pci devices that do not need the BAR reuse path? Would it be better to add the blocker only when a BAR is actually mapped? migrate_add_blocker_modes() can also return -EBUSY here, for example during savevm or an active cpr-transfer migration. Is it intended that a device attach or hotplug now fails in those cases? Also, docs/devel/migration/CPR.rst is not updated to mention the new restriction. > + > + if (!vfio_device_attach_by_iommu_type(iommu_type, name, vbasedev, > + as, errp)) { > + migrate_del_blocker(&vbasedev->cpr.mmap_blocker); > + return false; > + } > + > + return true; > } [ ... ] > diff --git a/hw/vfio/region.c b/hw/vfio/region.c > --- a/hw/vfio/region.c > +++ b/hw/vfio/region.c [ ... ] > +static void vfio_region_cpr_save(VFIORegion *region, int index) > +{ > + g_autofree char *name =3D vfio_region_cpr_name(region, index); > + void *addr =3D region->mmaps[index].mmap; > + size_t size =3D region->mmaps[index].size; > + > + cpr_save_ramblock(name, (cpr_ramblock_t) { (uintptr_t)addr, size }); > +} [Severity: Low] Can the BAR mapping lose its original protection here? vfio_region_mmap() derives prot from VFIO_REGION_INFO_FLAG_READ and VFIO_REGION_INFO_FLAG_WRITE, so a region with READ but not WRITE is mapped PROT_READ. This function records only the address and length in the generic ramblock list. With -cpr-ram-rdonly, cpr_exec_cb() runs mark_ram_rdonly() over every entry including the BARs. mark_ram_rw() then calls qemu_mprotect_rw() on every entry, both in cpr_ram_restore_rw() and in the new QEMU path. Could a read-only BAR mapping become writable after cpr-exec, or after a failed cpr-exec rollback, when this debug option is used? [ ... ] > @@ -347,7 +392,9 @@ int vfio_region_mmap(VFIORegion *region) > ret =3D -errno; > goto no_mmap; > } > + vfio_region_cpr_save(region, i); [Severity: Medium] Is the cpr-transfer blocker skipped for vfio-user devices? The blocker is added only in the vfio_device_attach() wrapper. vfio_user_pci_realize() calls the lower-level function directly: if (!vfio_device_attach_by_iommu_type(TYPE_VFIO_IOMMU_USER, vbasedev->name, vbasedev, as, errp)) { It can still reach vfio_bars_register() and then vfio_region_mmap(), and vfio_user_get_region_info() shows that vfio-user regions can be mmappable. So vfio_region_cpr_save() records the BAR address in cpr_state.ramblocks for those devices too. With cpr-transfer, can vfio_region_cpr_find() in the target process then return an address that belongs to the source process? That address would be passed to memory_region_init_ram_device_ptr(). I did not find any cpr or blocker code under hw/vfio-user/ that would prevent this. > =20 > +mapped: > name =3D g_strdup_printf("%s mmaps[%d]", [ ... ] --=20 Sashiko AI review =C2=B7 https://sashiko.vzint.dev/#/patchset/20261102-vfio= _v1-v6-0-9bc7cabc850d@virtuozzo.com?part=3D32