From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from AM0PR83CU005.outbound.protection.outlook.com (mail-westeuropeazon11020112.outbound.protection.outlook.com [52.101.69.112]) by lore.virtuozzo.com (Postfix) with ESMTPS id 60422808A3 for ; Fri, 9 Oct 2026 14:56:32 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ear8144Slhh4wDWDDnsHxR0MJEMz47JCCUNUEDvbamvLQyh4bsqpNBZHzAq6MV9cZZXBo3Xln1+wL3jAkEtbg2cqonY2xu2xIWsvy812qtC7prLtYYNmtiYpydUdtycsYqdYub+KUx5Z4S/kVi0av2ccrX3jNp3b3FpIaiONtmSFjmory4v6n4Vx/rAJZoyt/Xpth2eSlCVJrGN7jtXk0UDIc9P9RanJJcN2PpM+tmWzOAMFYyk9Neh3iMZZ7Q1qz+S5TX5bX9e5cfBYpIi9wrKNGUEdKUCFODjb6R7w5+X9YaZ+xyWNtgnzCy7eNf//V9OE14ed18Waw+bSKbVnlg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=uCgwlcB4vUCg4Ouy3N9lY4kUr6yHCtbPF5BidFpZ6fw=; b=eOjb0xAsay3+7TjHs+2pvz60zsKN/Rw+yRqsy9tqOW6G+P+LyKgMw/qfmcc1Vq0h61Qc4pLoC6z4IjFBJoO5LC1aWFjWr2IMJn8re+guPctNTQUyEDAGkFPH5wjY2u/3IusfCJPd6OvI5gsSTo3SKCc+LHKfTYloKXdnju/oP82Vfj2bOtG95uALEnzz3ke6mQ7KSlyj2bS2Y3jwMqUKSRpd2o6EXznG0UdiXyEfNIKf9V1+uiDwBgSCd4cvT3BIIsROlnI8YY+6VOtmQUk+MM612rJ6FRpEbaQkxT0so/QgCwbX+5Jr830CwlwsVcjbTYAAV9Xg5As7vJ7YAlHZZQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uCgwlcB4vUCg4Ouy3N9lY4kUr6yHCtbPF5BidFpZ6fw=; b=LgwcC3gBpFEnZ9dgFoCcY6b/ZdwEhMZVBZkWqvCMO5qtytD6dlQYTMqtGfh6WYICPKtnKMSqwMHF0hhIkUnRkf61QsI1L1z7EiOku5hLV+/U3PnkHY8yF7LEbgIlMwu7zFz3iZAkl2YWMCBjQoi3ZTmNnV8GPOVpEriHg+Ctgo9Uql/Uc/u/lZd+pTN4scP/SqLfVyLoj/4GniEeZPqbpWxoRe1rgKnbCQhSDDjjXOxqg3L0STznSI+fQJmTQa6egfwM1sayayUHDC6c3SehnFm5CHy/Ay/gqqPxMKXCwRMF4Bl1MG0p2f6RG5NqN3MWHbX93gtgpzaY7WqVFSlUxg== Authentication-Results: mx.microsoft.com 1; dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=virtuozzo.com; Received: from VI0PR08MB10656.eurprd08.prod.outlook.com (2603:10a6:800:20a::12) by DB3PR08MB9110.eurprd08.prod.outlook.com (2603:10a6:10:428::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.496.17; Fri, 9 Oct 2026 14:56:28 +0000 Received: from VI0PR08MB10656.eurprd08.prod.outlook.com ([fe80::4e37:b189:ddcd:3dd8]) by VI0PR08MB10656.eurprd08.prod.outlook.com ([fe80::4e37:b189:ddcd:3dd8%7]) with mapi id 15.21.0496.010; Fri, 9 Oct 2026 14:56:28 +0000 From: Andrey Drobyshev Date: Fri, 09 Oct 2026 17:56:19 +0300 Subject: [PATCH QEMU HCI-8.1 20/21] VZ: vfio/region: preserve BAR mappings across cpr-exec #VSTOR-103860 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20261009-vfio_v1-v1-20-2ae91d5b8048@virtuozzo.com> References: <20261009-vfio_v1-v1-0-2ae91d5b8048@virtuozzo.com> In-Reply-To: <20261009-vfio_v1-v1-0-2ae91d5b8048@virtuozzo.com> To: svt-core@virtuozzo.com Cc: Andrey Drobyshev X-Mailer: b4 0.16.0 X-ClientProxiedBy: VIUP296CA0132.AUTP296.PROD.OUTLOOK.COM (2603:10a6:800:34a::14) To VI0PR08MB10656.eurprd08.prod.outlook.com (2603:10a6:800:20a::12) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: VI0PR08MB10656:EE_|DB3PR08MB9110:EE_ X-MS-Office365-Filtering-Correlation-Id: f8587dea-d6a7-4d75-1895-08df2615764b X-LD-Processed: 0bc7f26d-0264-416e-a6fc-8352af79c58f,ExtAddr List-Id: svt-core@virtuozzo.com X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|366016|1800799024|52116014|23010399003|56012099006|38350700014|10067099003|22082099003|5113699003|18002099003; X-Microsoft-Antispam-Message-Info: x28Hi5JE+y+3OMyCV6oifWY5ZByI8qJpG/xt6/iKNUj/Jk8Pj0NslsdcWz6Th4OwqZsTe+7OpyLd9YKulooQEPG0bQA8+IvXmGn144sNR2napkO7k01JKY7PfsZI+vdMzyCI7chF4184Yx8piNs65uWbnNZocI+dxDr8TUwG+XFj3raguvVzqBHXgnDwoluJqrH/VF7qMPxTi2arH3zU1ZAtXp/wglE5sSYiz9TUnPhlxc4IFz+vUcjrx0pbAXXMBU5dZ/qH5tgqJDpwuA10lbr8HoB8WUNM5VPvxbSdS8afJp7IINE7motjvnrazZvao36cjVTFfcUbohoZrvd4VbcQX6Mx8pu3rZQQhmQnmHF4UQvR2pN/wwawpckbVWrNDN2Ho/Z4n4dxaNkO3apfbUxJJGaKIsA0EQ9MLUkT0ZI6WXFLendW9/aancqp3mzeqgqVq/pU1Y1kOTgZY2MkVRMFgmZXDiswZaQ4lTdTzziL0BHprIfhyLyt9AGGnSUHpfptB9PV04/IGL2FSToq9/NuE/pZ3lVy45txyo4Fm6/2DMziuytxp2x/fs0DR1hxrW6k4uKlgdYL8qdnMizaG2T0hwWvot0iJaVxZIu5HNouOcaCTVegQNAP2vJBijuWIkv+Lj9H0M7yXT2OI0rweRPPk5FqccYD8vAScBo60oy0nNC+Xnj4zBs3xdFIg3+yZ26lVjHPi7QlK1Wj2JdPPgXhSO4fAjpdg4OmdWRH9rU= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI0PR08MB10656.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(366016)(1800799024)(52116014)(23010399003)(56012099006)(38350700014)(10067099003)(22082099003)(5113699003)(18002099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?M3JFVWF5YjdpMER0MU9zTDI4RG1ZVFVEbTRJYlFDMjEvc3hobnhlQ3VaZnB5?= =?utf-8?B?TjZta240WDBJQ1RDUHBOcFM3UTdtc0hYMTBMOUNlWU5vdmZwZjNsbFlGMUFB?= =?utf-8?B?WEpkMTh0RW9hNUVvZnQ3czdkU242djM4Qm50Ti9FWm5XY0ZQaWV4dTN6R3BG?= =?utf-8?B?d25GSStaeWhFenl3dFF0WWdpRTQzNnNHWlE5TGc4czBSMkJtQ0dlbUN2eHRE?= =?utf-8?B?Q0xhbTM0aGFwdlNjK1AwRGhNZVYvTzhEZHRMTFZCV1c1cUJ2QTl0cWtSR2hq?= =?utf-8?B?d0tBNTdOT3duRXRDRllpMVdKT0k0ZWxudVMzT0ljbGZyRFl6U1BjaGlWdEdJ?= =?utf-8?B?UUlCZ2loZE4rSVZianhXOTREQmphRC9jUmsrVFlweHQrK2NiTEhQZXlZZjB2?= =?utf-8?B?V1ZvWCs5YVgzTldzNFJqTFBqbXFQVzgwTjNITFNtWUZzZC9jNXNXYjJjbnNW?= =?utf-8?B?eFpjTUQ0WDRwbnhUcmFwTDZXTHVwczlZaEtCVjRqVGxBa3pUWDhuUithV05t?= =?utf-8?B?ZTN4TXJXUVd0MG15K2t4Rm5nKzFtTlhSaGhsVDFGUnVJMzVWdjdONlhmQUo0?= =?utf-8?B?YVBZTWZLa1NGaDZnVHBNdHZOeTlmdlI0dDlIMWJ3TVFsMEhEUENUbzQyMUtx?= =?utf-8?B?S1BzVGh4QkVXQ2NScWkrZFMxblhiOGxOeHJIdzRiRnlOQW5TT2JGSTJ3OVFn?= =?utf-8?B?LzlWbkZSYmlxSlNsRHduUlVuSGV0TWFQb0dVUy9mVGVxcW5UcjBBMFJhcDBK?= =?utf-8?B?amUydFNXMG4wNFgwcnZYNGVsUjVsSDc4cWdmcmwxV3VteFc3M1dxbi9hY213?= =?utf-8?B?TTF3YTF3ZzA3UTYyVzMzV2hTVU5td3hIcHJia1RNSitIaEdGcWM2Rm5VZE1X?= =?utf-8?B?M1ZITThuUFFpSzdpd0RERThZTkdIZ1FvdmFFSW9oRUxCcDFOcXJTRDJaKzZ4?= =?utf-8?B?WE0xaWVoclRCOE9udFBIQXpjcjZCUEI0cDBadEJEc2RWNnUzQzAxU0RUQUdD?= =?utf-8?B?WGpQNlhpQ2Jzcks1Rll2MFprWFowWVZ4U0JSVjNpYzV2Z0FWVk9GR2xkTnJP?= =?utf-8?B?RVBMNFFFNW1XK29kdTJpNUhKaXo5UDFWeDkwTDJCekhYSkwvMlVlZXJIUjdS?= =?utf-8?B?eVlZSXNDRGlmTUdGVWQ1Q0I3Ykk5c3BiMTB1aFl4SDV5SWd1akg0MGVjZ1pO?= =?utf-8?B?NUltVEdyWkhZY1M1Y1lGbE16K21Tbk9XSy94QnR4K3RYV2Vxd2hGWXpsSDg0?= =?utf-8?B?V0wrbXMrc0tHRFhYcGpldGZrN0NFMUtlM3RjczVNQmtIMUNhWENUanVsRGJn?= =?utf-8?B?UFBiMmR1VTZIMGlWUWJWeEE4d0pKdTZvekQ5czlYZDZzMjkwOG5nM0p0OHBL?= =?utf-8?B?dEl6RjUrSmFyNit0RUdXMThENTdqYTE5Yk1KUktZcWxNVWRMWlc4L0hvNVdu?= =?utf-8?B?RjF2OTFMSjZZTHRkdks2SVptSjJuZTM5T0N0Zk9pcEZKS25XdHcwZXNZS2Ji?= =?utf-8?B?dE5PT2R6bklJdmFSUk5MaEFkWEdmMiswUGRWS2pPUWdOdERBNWRicXE3dFo0?= =?utf-8?B?TTcrWEwyT1FaR1FjMUFDa2R0SWF3eXVULzI0cWY2Zzc0ZUNFaHNvdWhQcHow?= =?utf-8?B?RFFtUFl2U055c3R3MEJyaVQybjJjcWtFWHlrMmpCbmJsZ3dtemtYUDJqN1B5?= =?utf-8?B?NjJHTVF1Wll5cWQ0V1F0MUIzdHp0U3pCcUJ3Q1hTSEcrL29uQzEvV09jNEtF?= =?utf-8?B?SlhtWHp6RjFmakw1RGZJdmVWS1BWL2ZZcmVJQW1WYmEwbFIrYkNmRVh2TGdL?= =?utf-8?B?QmxuZTdqVFlEc1VMUHdNZzFzMk53WDQvdS8yRTFFU01hajd6bGlqTDl4Slh3?= =?utf-8?B?ZVRTK2VWaTlTdmJPY0tWWnJwTWhBTFRhczEvOXBKejFJWVR3V3ZRVldkb3ZX?= =?utf-8?B?akI2M29sQU1uSm1FaUZCakhROVpiakw0YmxGWklPZlhvWkhKQ0J0aHJyOGsz?= =?utf-8?B?R0lrK1k3Tm5MeDJsTnBaaHdQS3lONHFSajFSR2NZdXFxMGhtNHRiUnAvbXZK?= =?utf-8?B?QTBoU2RDUGZNRS9CeURwNDZMQWI0Wk96RFU5YTNBSWdENGczREt3dXJvODlC?= =?utf-8?B?dDFhbElDZjhvZUFZamFzbU4xcGppMnozSXBXc3Z1aXpjNXZZNG5UdlJmWnJs?= =?utf-8?B?M3dXS1Z5VTBudjRlNGVRUnZnWHRBcXdSZjZvMkdUQVFyNm44SnB4OFV0QUVk?= =?utf-8?B?aUMrdkFQS05BYnI0bU91VU1LYXlMd2lFRG5sRXZBcWdrc3N5T05PZEI1UGVm?= =?utf-8?B?MWFicG4waHNCdnRjL1dQT0VHZnlKdUpUOUFMaVFBUWY5dytPaXlXNmRWNVpV?= =?utf-8?Q?qWc2KTxkxk+uIYAY=3D?= X-Auto-Response-Suppress: DR, OOF, AutoReply X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: f8587dea-d6a7-4d75-1895-08df2615764b X-MS-Exchange-CrossTenant-AuthSource: VI0PR08MB10656.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Oct 2026 14:56:13.1646 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: GBQT/0JbFU27KeEKdCQvvnIfj6HvjPhcdNOGXbfKyY/zDfjIReBnVbYfFB8uNQCTHQAdGnlV31TJY2S+MkioF9MLRUjD8/bsHkrriCvd4nE= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB3PR08MB9110 After cpr-exec, new QEMU mmaps the device BARs again on the preserved device fd. Some drivers don't allow that: the NVIDIA vGPU VFIO driver refuses to map a VF BAR that is still recorded as mapped by old QEMU, and that record is dropped only when the device fd is closed. So BAR1 and BAR3 of the vGPU can't be mapped, and the guest loses the device. Keep the mappings instead. Save each BAR mapping in the CPR ramblock list, so cpr_exec_cb() marks it with MADV_DOEXEC together with guest RAM, and new QEMU finds it there and reuses it instead of calling mmap(). Same find-or-create pattern as for other preserved objects, e.g. vector eventfds in vfio_notifier_init(). The ID is what cpr_name() would give to the mmap subregion: "/ mmaps[]". This needs a kernel which supports MADV_DOEXEC on shared PFN mappings of device memory. Without it cpr-exec of a VM with VFIO devices fails, and the VM keeps running in old QEMU. Signed-off-by: Andrey Drobyshev --- hw/vfio/region.c | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/hw/vfio/region.c b/hw/vfio/region.c index 1deaef12fd0..07027ee79f3 100644 --- a/hw/vfio/region.c +++ b/hw/vfio/region.c @@ -30,6 +30,7 @@ #include "qemu/units.h" #include "monitor/monitor.h" #include "system/ramblock.h" +#include "migration/cpr.h" #include "vfio-helpers.h" /* @@ -227,8 +228,39 @@ int vfio_region_setup(Object *obj, VFIODevice *vbasedev, VFIORegion *region, return 0; } +static char *vfio_region_cpr_name(VFIORegion *region, int index) +{ + g_autofree char *path = qdev_get_dev_path(region->vbasedev->dev); + return g_strdup_printf("%s%s%s mmaps[%d]", path ?: "", path ? "/" : "", + memory_region_name(region->mem), index); +} + +static void *vfio_region_cpr_find(VFIORegion *region, int index) +{ + g_autofree char *name = vfio_region_cpr_name(region, index); + cpr_ramblock_t ram = cpr_find_ramblock(name); + + if (ram.addr) { + assert(ram.length == region->mmaps[index].size); + } + + return (void *)(uintptr_t)ram.addr; +} + +static void vfio_region_cpr_save(VFIORegion *region, int index) +{ + g_autofree char *name = vfio_region_cpr_name(region, index); + void *addr = region->mmaps[index].mmap; + size_t size = region->mmaps[index].size; + + cpr_save_ramblock(name, (cpr_ramblock_t) { (uintptr_t)addr, size }); +} + static void vfio_subregion_unmap(VFIORegion *region, int index) { + g_autofree char *cpr_name = vfio_region_cpr_name(region, index); + + cpr_delete_ramblock(cpr_name); trace_vfio_region_unmap(memory_region_name(®ion->mmaps[index].mem), region->mmaps[index].offset, region->mmaps[index].offset + @@ -314,6 +346,11 @@ int vfio_region_mmap(VFIORegion *region) size_t align = MIN(1ULL << ctz64(region->mmaps[i].size), 1 * GiB); void *map_base, *map_align; + region->mmaps[i].mmap = vfio_region_cpr_find(region, i); + if (region->mmaps[i].mmap) { + goto mapped; + } + /* * Align the mmap for more efficient mapping in the kernel. Ideally * we'd know the PMD and PUD mapping sizes to use as discrete alignment @@ -347,7 +384,9 @@ int vfio_region_mmap(VFIORegion *region) ret = -errno; goto no_mmap; } + vfio_region_cpr_save(region, i); +mapped: name = g_strdup_printf("%s mmaps[%d]", memory_region_name(region->mem), i); memory_region_init_ram_device_ptr(®ion->mmaps[i].mem, -- 2.47.1