From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from MRWPR03CU001.outbound.protection.outlook.com (mail-francesouthazon11021134.outbound.protection.outlook.com [40.107.130.134]) by lore.virtuozzo.com (Postfix) with ESMTPS id 5DDB780B14 for ; Fri, 16 Oct 2026 15:26:21 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=CtKlZmNqBXB7LAy3+dl8mj2FpBKy0P2wkIUI85jg+vmiy/gTX137VKzJ+VHPtbO3w3UFlaK5ltvD46HAC3t7/mzGuJ4XgwK+GDVTZoJZIivYqCX3ISB0P2nFvhBDwS7fXGEJu3Bblbt2mVrv3VOpycglpBHGQiFW3TsMHGIUgBRAL/iUtwoyP4qw55u3UCM30IGP9Gu1hmKju+RLmsUSEB/ny3ttrjgm7b0pLGbT4cFQd/HHDwomFqz1Li6k+zBUpJd6KN73lUPtDV3raz5gPb8H/2BaSSZyCjOLrTpuUCyYzLUFxL0RaFVSSB7cxKyICg5/NdJ/Gkpjj+UY74V4sQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nIIReshAOQ6r4B9i/dysktnjYagoEJ6i/Tncu2wr1qE=; b=GZW8xFAhWvol84PiIzXRTXjszhlviN3TqRVrru518Fan/zLx8CD3EoXCokbiyOHAVnYre7+P1BSzL0FPW2HW0WGn38nAH8nOengnPGCd+Y+Djb6spImTZqqZHW9eE3yqZ+XCYG/NcJ8MS8MmbUwSFfa/zlTLff8ISgCPUuo5Bery3Tfx4OtvvCOKR6gf89JmvYbcqUTjs0n2N6QhYRNMGs5I8U/Zw5voevKseqgebNYFSdyGCquWGVPoNaOa9oYFUWt1fK4umVyrgeHq+xLZjbWNrbgukANnU5XscjyM167ZSWL4feP8cg1VWG8zATxXevSzwClOuzYJsNsW2JSAxA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nIIReshAOQ6r4B9i/dysktnjYagoEJ6i/Tncu2wr1qE=; b=hdgrNAn64gAhy/RJaxx3gWNdsfU47LAVyKXYdGklRS7Iicr0SGNpgzK/fxnDeda88LGKsnZU2yNFy1dvo3K/oVaw1NW5ksJAd2ArC9If56MZ/C5tiscmOAihMKCIPZbkjqNcg07NYuGdy+1ultuk3va6RC8NHnF7lrvq8FjTDVVIDwIB9hnAG0j7GENF08wgoNjLRCcxdI9o7Tdxs4qBJKEO2E96J21V3W535d4IEDh2Mz8jEyKtnJu08b73z58JkTSYfLji8snUuhcnmqKThe5PEknF+bdPhq32oJUmMmTwhe6rmoApHee9zOBfukv+9CLMHXLtgu60CIjbZV1Dtg== Authentication-Results: mx.microsoft.com 1; dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=virtuozzo.com; Received: from VI0PR08MB10656.eurprd08.prod.outlook.com (2603:10a6:800:20a::12) by VI0PR08MB10941.eurprd08.prod.outlook.com (2603:10a6:800:250::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.546.7; Fri, 16 Oct 2026 15:26:19 +0000 Received: from VI0PR08MB10656.eurprd08.prod.outlook.com ([fe80::4e37:b189:ddcd:3dd8]) by VI0PR08MB10656.eurprd08.prod.outlook.com ([fe80::4e37:b189:ddcd:3dd8%7]) with mapi id 15.21.0546.006; Fri, 16 Oct 2026 15:26:19 +0000 From: Andrey Drobyshev Date: Fri, 16 Oct 2026 18:26:21 +0300 Subject: [PATCH QEMU HCI-8.1 v3 23/28] VZ: vfio/cpr: don't close KVM on cpr-exec Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <20261016-vfio_v1-v3-23-47487cb5a8e5@virtuozzo.com> References: <20261016-vfio_v1-v3-0-47487cb5a8e5@virtuozzo.com> In-Reply-To: <20261016-vfio_v1-v3-0-47487cb5a8e5@virtuozzo.com> To: svt-core@virtuozzo.com Cc: andrey.drobyshev@virtuozzo.com X-Mailer: b4 0.16.0 X-ClientProxiedBy: VI1PR10CA0088.EURPRD10.PROD.OUTLOOK.COM (2603:10a6:803:28::17) To VI0PR08MB10656.eurprd08.prod.outlook.com (2603:10a6:800:20a::12) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: VI0PR08MB10656:EE_|VI0PR08MB10941:EE_ X-MS-Office365-Filtering-Correlation-Id: e5ac86c5-337d-4953-ac45-08df2b99d184 X-LD-Processed: 0bc7f26d-0264-416e-a6fc-8352af79c58f,ExtAddr List-Id: svt-core@virtuozzo.com X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|1800799024|52116014|366016|376014|5113699003|10067099003|22082099003|18002099003|261009223027099003|38350700014|56012099006; X-Microsoft-Antispam-Message-Info: hQ8K6b9ZOMIVPU/Ed/OOlgDePuaiushGAUm4v8QX0h7ONOhH6YSL0MlH6r7PFN734EFjG87M7GBQaZkqaydfLV5GalE0n7eYRSc1N5qOs1LlYD0ZlgNgPyI4cLYEYpcBKpMYnx/yMozHVNjHv4fwJl9xue+5STVG3dVrAARl4Ui0T1u1j+hxgrqKxXgJJ/P0iGCY6EOMD3DAbPgT7kcteZFH0LVU5XaZQRR7nMqfWAVnp3qtmKYXy6H7DrGK/wpfUAaszx4sjuSPXJDCTrgVYqSn4sqfK6iGl5r8AGP6Qn8KayPGXm/Dlcg3HZ2Sjyu7iq3EdGPuRJahPtP4GZkGBH3br0ZTvzuGLNxFse3HjB9A6wKQZpfG8Q1gUsll6555TcR1+iYOnA1d+uZAXQQNy6/2aEb0q4Url1EtOt1SH7UF7qqTJspVDWCvBrRMfcCu0G08acJ9sgqW2FO3y5x+L3xikYoyZ2V4hXCEMGlN9garGxC3RvKF1s1mOav23Emhhd3uh5+2C0qL717L9yQpkvaKOuASC1Y7xYG9KrqZQHNgCwSaXSDlytXdiVImgLqR/46BAASw/ZDIfDneASlBVN4+1ZfKX/ByGVqIv6MKygCwkb1+xgtqqCY9MwDnFFpi0O2kVH1sPb6qtrD6D9bA58RTTOTsb2OW7HJkhWoKHwBXdHrSQuRmtuA0tbUaK9BAr59vD9w1bw4wJRzSy/UMe94zqETit2yPF8dBrS8KwMo= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI0PR08MB10656.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(1800799024)(52116014)(366016)(376014)(5113699003)(10067099003)(22082099003)(18002099003)(261009223027099003)(38350700014)(56012099006);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?S1VDTEcvQ3gyNnUzaU1OakVjazhwbnI1WUYxcFhRT1lncVNFL0NqbmkrZlBS?= =?utf-8?B?L2F1NUcxN3JnaUlyOFY1dUZRUnZieU5Pc0JJSUtRWlpxZ3NHTGFGSXQyekQx?= =?utf-8?B?ZDZRZEJxREw3TDhmTXFURjJuckd0amc0bXlhTmFWYmZNVVc1elpmZ0FVTG1P?= =?utf-8?B?S1VpQUlaV3NVMlhMWm5HbXBPRTVQTVpuOEI2ZExReVNOUnJNNVFVSW1JMUgv?= =?utf-8?B?cnc3eERmSTJQVnh0bmZOWUpCc01lRnNoTnlHMktORkRBOEluRGFhS0NQK2kx?= =?utf-8?B?YktrOFRiVjdMd3doME42b2lhN3NORStWWU15R3NteDB3T0ExQk1TaFJOc0Y1?= =?utf-8?B?cmJBVDFEOVRvbmpVUGtFam91M3NPQnJDVU43RGVqaE5SaGVSZzFyMUxYTTJo?= =?utf-8?B?L0JwRWl4VUd4R2g4UnlsSWpYSXJ4RXFaYVhIYlVvcUxNNENkUG93UnZPaC9i?= =?utf-8?B?cFpZMkp1U2lNU0dTQm41aVV5cDBQQjFSMURWU09neDJVdE5WenB0bm55QUl5?= =?utf-8?B?QXFZVXVBWkVvNjBDRytLdDNCcytYQnQzUVNFN0tQNWlsQ1VRdnhhT1ZsMWRy?= =?utf-8?B?QWFoYU1LTGhpNTBNclBLVWszS2xucklIdHdINDdENHNiR3dYS21OUW1jVTNh?= =?utf-8?B?OWRtYWpMZXh4WlNpYTBEYm93YWtRWVpsNEp4dENEdnZZbm9EZmxNREhnT3JT?= =?utf-8?B?b2ZUM3EvK0RRcWNORWRjYkp4N3ROTTZMY1JMb1RCT0dCSjRMbks4SHkveVJw?= =?utf-8?B?Mkt6M2ZuL3pwQml2ZFZ1OHcvbjhaQVBycEk2OE9qVXY5eHJEaWl2a3NpUnlr?= =?utf-8?B?U1VwbG0rMmtYSkNLUENmeE0rZ3d4QUtMdnExTWpYSTIwRFFlVlo3a21QNnRG?= =?utf-8?B?d3RXN2xKMXQ2KzloMW1ycUdKV281MlpJRDQ1S3Arb3g2YjFKSnBzSDNVZXNy?= =?utf-8?B?OU5GazFJWE1Oc1llcnh3N2lRZjUzVVhaTGN5VlozbW1vQ0pSQ0wzTy9vYU14?= =?utf-8?B?d0E3cmFRcUpDbTM2b2VwZ2dTMVJiR0twcmI3RjBDTHdTc0FJS21XZEprbW5k?= =?utf-8?B?T01ZcEZYMmZqTFZXODY1SXVCZG96VWVUbi9TYy9NY3FoZWZJd3BUbUZBSUdL?= =?utf-8?B?UmxVSkFSeFdvbFB0bVRBSnFIRGJyZVZDL3VqcUQ0L0ZjVXg4SkFWRzdqSkdj?= =?utf-8?B?MDJRSFJuQ1IvWFpHOGdJeFZJanhSQXpzUUZLNmZaRDZPWVR6SzRxZmVHUTF5?= =?utf-8?B?S3NSQXRXQmxLOW1sUm14MzFMNVk0NTRFNjBvUlZrTkJUUlF1TjlPUlgyU1lo?= =?utf-8?B?cFJnaUgxT2hlV1Aza3RLSnQvZG5IbVIzcEVqT3NITVdaY2Vqc04yWVBZbUlN?= =?utf-8?B?NXB0SXMxd2lyeXRuQVRYK0R2N1NhSDQyTE5FZUNXK2I2OWUxU21kZzlnTEtz?= =?utf-8?B?UytXaDdGaFpPWVR6N2pyaHROeVA5VXpSUWRWN1RkRlB4T2NqRnc3Rk9kb3lB?= =?utf-8?B?bHJNaitWWkxQUVZ6QmRVZkFqMnpzTVl5K0xHRTV5TU9qdjlDczA1dldta0lh?= =?utf-8?B?TjNDaWtZblJIK0V4UDV3MDZ2VEhnRnNPVVZRejhodUxqNTNGcjdmMmloN3k3?= =?utf-8?B?UDRueSs5V1FvNE0yYnhySjBnMnduMGZrSVBrM1cxSGt3RmdNSWdUbTVQNFFL?= =?utf-8?B?Y2pTbEtxb2NUZzI2STk3VzkrSHBHSXlpaVVDSkdYWDFhOE41KzR2WDNhNlkw?= =?utf-8?B?emdjNkpvdHJNWm5IMC9vRXE2ZDkrNFhUdlhjTm1mYjBsczRhME9jMnU1N3Vo?= =?utf-8?B?b1ZzeTh5ZGRyc0w3WlhneHB2NXZpQWlRMzd1RTcxbDBpb3l6SXlua0czc1ZO?= =?utf-8?B?RHdKR1Q5aTRvN3NPbjZhQXV4d3VlcXhlT2lsTjBNNmlocVlLUkJzMzNjUkRI?= =?utf-8?B?Tng3S0xBVjM0VnlrMTdpTDFHVTZ6c2psWDRMWllsNTdZaGFTZDBHVUU3Uk51?= =?utf-8?B?TGptTlI1elFXN3BDNGsrZmhhQzBabmlya0h3T2tJUkNuRitrcEJDSkMyK1hR?= =?utf-8?B?WVNHSGN6Wk5rR3N4ZDl1akJ1V0hpUll1V1Y0VVZaMnRXNFlMZ2VNRGdLYy9w?= =?utf-8?B?bEpDWnVBRWRhVzhXR3BNUEgzZlBVRFdSUjFXS05sRkIyRW5DOVN0M29aSW5O?= =?utf-8?B?L21tWXZHSGpxbXBFeXZ1ZXdLeHQzV3JPRVBtZU1hRXpUYjNIV1JFRkdrdCtL?= =?utf-8?B?U1V6dW91QXM3WTUxTmVVWSt1cTVGTmtwV05RMHp1N0tBM1ZuSGtWZDBvT2RW?= =?utf-8?B?VWxpaXNZazJPZE1hVXVVSGthQmh0K1VoTXo1ODE4ZkpiQ2JVYVFiWTc2MGJU?= =?utf-8?Q?f1Ltmf8uZzA2PMVc=3D?= X-Auto-Response-Suppress: DR, OOF, AutoReply X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: e5ac86c5-337d-4953-ac45-08df2b99d184 X-MS-Exchange-CrossTenant-AuthSource: VI0PR08MB10656.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Oct 2026 15:26:15.6544 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 1iDWhhrFam2N7iJhcUeUIcddf7c5gHZ2uMgG9IzVYA/gqYr9sHSrUsiM9N5lJTkWhHM307ONwBY00bUPdXQUrn2iyde01DHpRTp6ZouiHlw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI0PR08MB10941 With VFIO devices, KVM is closed on the source when the migration completes, so that the new QEMU can create its own KVM VM with the same devices. That is done from a migration notifier, which for cpr-exec runs before cpr_exec_cb(). But cpr_exec_cb() may still give up, when the memory can't be preserved or execve() fails, and resume the VM in the old QEMU. A VM without KVM can't resume: kvm_close() removes the vCPU threads, and the first KVM ioctl of vm_start() aborts QEMU (KVM_SET_CLOCK in kvmclock_vm_state_change()). cpr-exec doesn't need it anyway: the KVM fds are close-on-exec, so the kernel closes them during execve(), before the new QEMU runs, and kvm_vm_release() tears down the irqfds then. Keep the notifier for cpr-transfer only, where the source process lives on. Tested on the vfio-net rig: cpr-exec 5/5; a cpr-exec aborted in preserve_ram() and one with a bogus cpr-exec-command both left the VM running with no lost pings, and a later cpr-exec completed. Signed-off-by: Andrey Drobyshev --- hw/vfio/cpr.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/hw/vfio/cpr.c b/hw/vfio/cpr.c index db462aabcbf..7ca761d5bf5 100644 --- a/hw/vfio/cpr.c +++ b/hw/vfio/cpr.c @@ -192,13 +192,16 @@ static int vfio_cpr_kvm_close_notifier(NotifierWithReturn *notifier, return 0; } +/* + * Not for cpr-exec: the exec closes the KVM fds anyway, and until then + * cpr_exec_cb() may still give up and resume the VM, which needs KVM. + */ void vfio_cpr_add_kvm_notifier(void) { if (!kvm_close_notifier.notify) { - migration_add_notifier_modes(&kvm_close_notifier, - vfio_cpr_kvm_close_notifier, - MIG_MODE_CPR_TRANSFER, MIG_MODE_CPR_EXEC, - -1); + migration_add_notifier_mode(&kvm_close_notifier, + vfio_cpr_kvm_close_notifier, + MIG_MODE_CPR_TRANSFER); } } -- 2.47.1