From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <010701a169d6442c-24c92fe0-8b4d-4478-9fe1-1edaffabdecf-000000@eu-central-1.amazonses.com> Received: from DU2PR03CU002.outbound.protection.outlook.com (mail-northeuropeazon11021133.outbound.protection.outlook.com [52.101.65.133]) by lore.virtuozzo.com (Postfix) with ESMTPS id 2839B8045D for ; Fri, 23 Oct 2026 17:36:56 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=lJ/8N7VOrZ6NYMzyxKOfMh3zO1g2k4cXtsk8RnWWB14Righutn2tY04an3dsEaODgqJE2xKHQRdMFQa1ApOqH3tC55c5vIzr8sPSRxR3zXvmP19CtFUFk7tneNkr2y0apdoTLQ60H9tuVXGWqJbzqkoofaMVXWwN5EbzqFMOT+I3B7LmeUjUve384VDPsPSkwzi2cd4KqN6eYvBCBQn8eveXlsqYx6deQJzy2J5R0AoVC/z84zKWdZP/NDwV50XElaYswBiusG0w919dYdQY7nTtlN2DBg5NvdHt2U8XaEQrHT7ab8ahWGgWJLd1+9kqP/MAAyjvGXFmZZOibaTC3g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HdXNJKkh/CoVse14k4etkLMNQCro6cRbmU5sPgy1wOc=; b=HSokExI3HDdEoq8panWU77lscaxXmRPM12U0bEtvAwN50nRm0E6esQIgzy4Uga71/mtxEXfoNdEqQHmAizMy6UB2qJvhjbs1u2AEUhGWoqQ3fOFGAsg488JKJTCAKzGvUAC6i6d+eX96NYNd8zK7EYAIiT+pDoDRbmrSxZJIMCsu4N0T1lxfu4niCSwsXKZPu10HuIxHsMnHoU6vU0nB2/D9eQeE7F0NywYjAtEo+pz/Cv7u//hvAjPMRUkRHkxmDl2Yy5Mx7w4xYzJTZ+jcNWtgzja1LP29ZDDO5b7dG2dugZMGTdwd4tkcXCfPEr9UQSbyFKOku8YTmZCThYz+UQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 69.169.224.1) smtp.rcpttodomain=virtuozzo.com smtp.mailfrom=eu-central-1.amazonses.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=sashiko.virtuozzo.com; dkim=pass (signature was verified) header.d=sashiko.virtuozzo.com; dkim=pass (signature was verified) header.d=amazonses.com; arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.onmicrosoft.com; s=selector2-virtuozzo-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HdXNJKkh/CoVse14k4etkLMNQCro6cRbmU5sPgy1wOc=; b=Qwc3/Z/Nemb6G/S1bMoZwJuLMAf4dbjZWL6DuFlc4kqaY2SIHSC3Rr8PVpLlFRj+XME5sRRsdRPd2Z8hz4yP/NM891y7Ai2R6oPYXdv9U5H7GX6pdUlJ88r8GPbaDbpb2OfNGgbfC3aCeYilSCiR0D26xsUf0wDLL14kYjnexEY= Received: from AS8PR04CA0150.eurprd04.prod.outlook.com (2603:10a6:20b:127::35) by PA3PPFED601E0B5.eurprd08.prod.outlook.com (2603:10a6:108:1::136) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.546.18; Fri, 23 Oct 2026 17:36:52 +0000 Received: from AM1PEPF000C1AD9.eurprd02.prod.outlook.com (2603:10a6:20b:127:cafe::48) by AS8PR04CA0150.outlook.office365.com (2603:10a6:20b:127::35) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.546.20 via Frontend Transport; Fri, 23 Oct 2026 17:36:52 +0000 Authentication-Results: mx.microsoft.com 1; spf=pass (sender IP is 69.169.224.1) smtp.mailfrom=eu-central-1.amazonses.com; dkim=pass (signature was verified) header.d=sashiko.virtuozzo.com;dkim=pass (signature was verified) header.d=amazonses.com;dmarc=pass action=none header.from=sashiko.virtuozzo.com; Received-SPF: Pass (protection.outlook.com: domain of eu-central-1.amazonses.com designates 69.169.224.1 as permitted sender) receiver=protection.outlook.com; client-ip=69.169.224.1; helo=b224-1.smtp-out.eu-central-1.amazonses.com; pr=C Received: from b224-1.smtp-out.eu-central-1.amazonses.com (69.169.224.1) by AM1PEPF000C1AD9.mail.protection.outlook.com (10.167.242.104) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.546.15 via Frontend Transport; Fri, 23 Oct 2026 17:36:52 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=23yjbh4k34hwzfubhuiim7m4inbd3ick; d=sashiko.virtuozzo.com; t=1792777012; h=Message-ID:From:Subject:To:In-Reply-To:References:Content-Type:Content-Transfer-Encoding:Date; bh=gn64mCiAQVjelDY9e82DMK8Y47fHt6AvXWZLkfaplZg=; b=fOFzbu/EuoXldVwaScgti/Tw5rqSAiokmX4+SMoZf8cC2+jBAbCOA9n7EDfa0pux kYiyozdcIPkHFIQBD+je0Q31yD+sE2QS3PXiSIR5HKSOjelO49/oFFuVsq8gJ0azAGR o8PTM7Eu60r/6IGEqzLvU5oFetj+bdHmQMfWvbjAlPOflyleupji3T+UwrwifTOVS0o s+FAAQaSpWflbmqXhcvlSj8rAbKuLgqxeF5HheAWt03Lmua2dwPL+cusIyeliYgeZH6 BuivOeDhqWmk7bQ33TVAjatsGM58craChvCDPvlO3yp1YvCecgL67NLnnxKLkFxLRFK uR/o3PwO3w== DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=qftdzk2dqsatjnlrq4r5brjbihpfcrsh; d=amazonses.com; t=1792777012; h=Message-ID:From:Subject:To:In-Reply-To:References:Content-Type:Content-Transfer-Encoding:Date:Feedback-ID; bh=gn64mCiAQVjelDY9e82DMK8Y47fHt6AvXWZLkfaplZg=; b=nO1p4ouocdyiSgpcmKj7XVbQo+oQH86WEDis2w85Qvc97psii4lQB7EPvK9b66Bk m29/Jv865KzSyNa6VEo3aWmr8XKrq2/ns7jgWCTGl07SQb8b4g39ElQuDuBfle/5PA4 LFkDzOrYLAu7kaKzvDg12KT+CqU3JAU023p3xx/E= Message-ID: <010701a169d6442c-24c92fe0-8b4d-4478-9fe1-1edaffabdecf-000000@eu-central-1.amazonses.com> From: no-reply@sashiko.virtuozzo.com Subject: Re: [PATCH QEMU HCI-8.1 v5 32/36] VZ: vfio/region: preserve BAR mappings across cpr-exec #VSTOR-103860 To: Andrey Drobyshev , svt-core@virtuozzo.com In-Reply-To: <20261023-vfio_v1-v5-32-ba0bf3c6d122@virtuozzo.com> References: <20261023-vfio_v1-v5-0-ba0bf3c6d122@virtuozzo.com> <20261023-vfio_v1-v5-32-ba0bf3c6d122@virtuozzo.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Date: Fri, 23 Oct 2026 17:36:52 +0000 Feedback-ID: ::1.eu-central-1.2wRQJBpg9g1fGRYexx3dUuYGS25Bux+2TEOx72/m6U4=:AmazonSES X-SES-Outgoing: 2026.10.23-69.169.224.1 X-EOPAttributedMessage: 0 X-EOPTenantAttributedMessage: 0bc7f26d-0264-416e-a6fc-8352af79c58f:0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AM1PEPF000C1AD9:EE_|PA3PPFED601E0B5:EE_ MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 189af795-9082-4806-485e-08df312c39d7 X-LD-Processed: 0bc7f26d-0264-416e-a6fc-8352af79c58f,ExtAddr X-MS-Exchange-AtpMessageProperties: SA List-Id: svt-core@virtuozzo.com X-MS-Exchange-Transport-Rules-Loop: 1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|9140799003|61400799027|48200799018|32142699015|13003099007|10067099003|261021123027099003|261021121427099003|261022075727099003|261022105027099003|261009223027099003|4143699003|6133799003|16102099003|18002099003|22082099003|5113699003|4113699003|5063699009|56012099006|55112099003; X-Microsoft-Antispam-Message-Info: /fo38fZRVX/hle9FMbWq08LWl4A3rwVEIcqR6nJbJ/6MNJLyk5yYWyzcVhRIc3kRlNw2toTAHn5xNd41+w3gqhtiu54h7CWYTXqz8qaeKIrLaDZFGDiLDAYT+HiI30bwVdcy9C3PWUWor/o4Sx4xshu8Id9/p5xVWwfhNVgS+div86zk9C6AtGv+usiUON7hGX+zO7K4mD2d2RshgoKbJ0tY+Ap/p4dE8VQGYgUnmxtUXzaVUYsu1K+xVEHsPQPcvcjXpjp8gkbmzynbPaxpIUvWnENZYkErDH7cj8YccXL7imLZfV9qLmM1KFv1AXsvyd+KpVVmno3O445hU4h7Ov2dMRgfuELi2NcKisyZKIF+VZ1sCJkH7Oza07gokllHPBVc+D1CSaAoyeqi4hLpMazl1uOcxLbiqM5VVLI5zIZ66qchXCQVtfCLqNFGiMV7MH91i6FIKtq0Vu8ugc4+JtUeMdHdyPdqej/E7uEGximn0MRwe6TElHpKAjsJ6K42F3nYXFM2UW6nU7E14BLlVebm/a+ZfH3lANhoEfkygaIJgW/0pF4mZ3tvZlwG+q+MUXeze9oS3AlRjgIVzhHsDW1ECDRG9Wy6jXs+8/Zr+9gSyu4TDG1VgJloPWnMs0VXBOZlBKXo9iSe7VCJJ72Opmn1O/Mtj8nPFXxP+2FIbNzB3PoCi2GN5ztG2VlPav8r06u6a3vm0rrNM4lRSVOfp5nojaRZiQql1we6iBzvPmOUoBAQBUfzJNa9ye/vym+1XZGcb3l5Hfj9zZQ/CvOpj5iy1vNjHOiofNbaPNQ9cSzoP7sKmD7Bqyp2Gxqm/zEBbdFTdox/FHwKNSMgZU0SnydZOzkav9kmsBykjVj5jTW6HdRlzLaEf+qS7U+dNha7 X-Forefront-Antispam-Report: CIP:69.169.224.1;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:b224-1.smtp-out.eu-central-1.amazonses.com;PTR:b224-1.smtp-out.eu-central-1.amazonses.com;CAT:NONE;SFS:(13230040)(376014)(9140799003)(61400799027)(48200799018)(32142699015)(13003099007)(10067099003)(261021123027099003)(261021121427099003)(261022075727099003)(261022105027099003)(261009223027099003)(4143699003)(6133799003)(16102099003)(18002099003)(22082099003)(5113699003)(4113699003)(5063699009)(56012099006)(55112099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: nn4lKd4kmKrrFTKeCQPuw4JzRsRIkwA8wm+9Rc4NWWT3h2Ohzx8VlgGO1WU9EGvCHJKqMcUDMOMH6SD85qWMilSASIB8pByt4WYzijb4OYRv4GWwYq0JgQXZGCqsfmG2CBerKm8V2smizGssHRkcCbrNs7rJ2oQR64meAC4O4eQhJC5pa35tEbO8HJluyMRvWu0KZ2edRUDb0EpuO/Bz28ZVxHuEDT8upLlbw4slkACQwBw+f2Kk2c65k48soSv1lWJtpBwC6Bxl8Hf7gIiRVJvw8QWpqMGU3CxiDY4ajXrYpK+E8/nT+kfl8/+ZD8GxXC9FfkgM0jJBchOy5AUUT3rXsXWh7hCbmUdTsicZU3gj7u4jcj/8WSwt4FfNUe5Zv8N5oRM0hqgGfJGL/vq1KXg0Dta7eHsTuR1l/TslMIVEdUldezHUU20rbXsZwdHd X-ExternalRecipientOutboundConnectors: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Oct 2026 17:36:52.7980 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 189af795-9082-4806-485e-08df312c39d7 X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-AuthSource: AM1PEPF000C1AD9.eurprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: Internet X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA3PPFED601E0B5 Thank you for your contribution! Sashiko AI review found 2 potential issue(= s) to consider: - [Low] vfio: unclaimed CPR BAR mmap entries are never reconciled after cpr= -exec - [Low] vfio: BAR mmaps in the CPR ramblock list get RAM-only mprotect in c= pr-ram-rdonly -- commit a087f4249b17de71aa8260f7340069a28fcc6353 Author: Andrey Drobyshev VZ: vfio/region: preserve BAR mappings across cpr-exec #VSTOR-103860 This patch saves each VFIO BAR mmap in the CPR ramblock list so that cpr_exec_cb() preserves it with MADV_DOEXEC. After cpr-exec, new QEMU finds the saved mapping and reuses it instead of calling mmap() again. It also blocks cpr-transfer for VFIO devices. > diff --git a/hw/vfio/region.c b/hw/vfio/region.c > --- a/hw/vfio/region.c > +++ b/hw/vfio/region.c [ ... ] > +/* > + * A saved mapping is one preserved by cpr-exec, reuse it. None means o= ld > + * QEMU didn't map the region, e.g. its mmap failed or it's mapped lazil= y, > + * so map it afresh. > + */ > +static void *vfio_region_cpr_find(VFIORegion *region, int index) > +{ > + g_autofree char *name =3D vfio_region_cpr_name(region, index); > + cpr_ramblock_t ram =3D cpr_find_ramblock(name); > + > + if (ram.addr && ram.length !=3D region->mmaps[index].size) { > + error_report("CPR mode=3Dexec is requested, but the mapping of %= s " > + "can't be reused: addr=3D%p length=3D%zu, expected " > + "length=3D%zu. Aborting", name, (void *)ram.addr, > + ram.length, region->mmaps[index].size); > + exit(1); > + } > + > + return (void *)(uintptr_t)ram.addr; > +} [Severity: Low] Is there a path that removes an unclaimed entry after cpr-exec? vfio_region_cpr_find() only reads the entry with cpr_find_ramblock(). It never consumes it or marks it as claimed. The entry is removed only by vfio_subregion_unmap() and vfio_region_finalize(). Suppose new QEMU never calls vfio_region_mmap() for a region that old QEMU had saved. Examples are a different device configuration, a no-mmap configuration, or a region that new QEMU maps lazily. The entry then stays in cpr_state.ramblocks. cpr_exec_cb() calls cpr_walk_ramblock(preserve_ram), which walks every entry in the list. A stale BAR entry would be marked MADV_DOEXEC again at the next cpr-exec. If a stale entry is later found by name with a different length, this code calls exit(1) instead of falling back to a fresh mmap. I could not find any code in the series that reconciles unclaimed entries. Should unclaimed entries be dropped after the new QEMU finishes device setup, and should a size mismatch fall back to a fresh mmap? > + > +static void vfio_region_cpr_save(VFIORegion *region, int index) > +{ > + g_autofree char *name =3D vfio_region_cpr_name(region, index); > + void *addr =3D region->mmaps[index].mmap; > + size_t size =3D region->mmaps[index].size; > + > + cpr_save_ramblock(name, (cpr_ramblock_t) { (uintptr_t)addr, size }); > +} [Severity: Low] Can putting device BAR mappings into the generic CPR ramblock list interfere with the -cpr-ram-rdonly debug option? vfio_region_cpr_save() adds each BAR mmap to the same list that migration/cpr-exec.c treats as guest RAM. When cpr_ram_rdonly is set, cpr_exec_cb() runs cpr_walk_ramblock(mark_ram_rdonly) over this list, and that now includes the BAR mappings. mark_ram_rw() runs on the exec-failure path and in the new QEMU. It calls qemu_mprotect_rw() with PROT_READ|PROT_WRITE on every entry, and it calls abort() on failure: migration/cpr-exec.c:mark_ram_rw() { if (qemu_mprotect_rw((void *) ram.addr, ram.length)) { ... abort(); } } vfio_region_mmap() derives prot from region->flags (READ and WRITE). A BAR mapped read-only or with no access would be widened to read-write. If the kernel or a driver rejects the mprotect(), QEMU aborts. Typical vfio-pci BARs are read-write, and the option is debug-only. Should BAR entries be excluded from the mark_ram_rdonly and mark_ram_rw walks? [ ... ] > @@ -427,6 +474,8 @@ void vfio_region_finalize(VFIORegion *region) > =20 > for (i =3D 0; i < region->nr_mmaps; i++) { > if (region->mmaps[i].mmap) { > + g_autofree char *cpr_name =3D vfio_region_cpr_name(region, i= ); > + cpr_delete_ramblock(cpr_name); > munmap(region->mmaps[i].mmap, region->mmaps[i].size); > } > } This is the second of only two places that delete a BAR entry, as noted in the vfio_region_cpr_find() comment above. --=20 Sashiko AI review =C2=B7 https://sashiko.vzint.dev/#/patchset/20261023-vfio= _v1-v5-0-ba0bf3c6d122@virtuozzo.com?part=3D32