From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from DU2PR03CU002.outbound.protection.outlook.com (mail-northeuropeazon11021143.outbound.protection.outlook.com [52.101.65.143]) by lore.virtuozzo.com (Postfix) with ESMTPS id 72C6180E7E for ; Sat, 7 Nov 2026 20:56:37 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=VLyrklVB8WTC7d7XxYHlKEoaRvxdL0YsjIxsk9s26+ll8UFW5v0E5dgf6gdh27jOAT3yf5fJBVVA/yixqNnZy4drMwgoHcT4wQ1KplteWRYXW9v1suCUJf8UuFiObvLJ3+dUmWiF/2tq+Kb2F0Xiu4ieTGud4VQ5IZ12zobou8atUEfRklfZvxDq6gDk3xNaMG4mPjiyN7VtCxVcpatcvLWGBzT1hCKZjeg2V5ZtzSO42pzboaVH670dpaiU6Fmm1+izVs673xtYeTSuHQ1uj2qFLc+UxJbWmWxWJEDY6iIYwYuuFOWrW987JSNN/HZSEIftyi3CodqkZwJGg1E1Ow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=UJfsaJmvpTQQ6pkVGfANyDyturxF63QYjeijcOES8qY=; b=Y+Sks4cWaJK0j+fwmWYC6qz9dvqacohnWic44jhq/9zldSRcffY4iYAad0sFHcH6jxarduBm+hlmoxCBWQGx3Tn8u/iR1AIBPPlgIUrbgxw+PPuNJoymruqGW4rtslaC89KUhqwGk/Z7BG8dAwX0eba55ndCxR7tflIsX8/svSEYr0pzCPL7gn76grSIoeDVahkXf+jpsPcw+9B9gpK71NmFRsPnMBeQXizmp8NRCzYodaykd8seFhfExpgH8r+FtIZoHfgMCOfpC5iAqAMUItw04WFCcg4qqudj/iMkY5S4Va9NvlT+rkX/XnPY7eOMsAxkM3y2zk/rnJrg1R0zrg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UJfsaJmvpTQQ6pkVGfANyDyturxF63QYjeijcOES8qY=; b=i1sWf61JxkTKuQMSSvelP3QoWMPnhj1AuWxeV0t7kLk3jKqp0IKkwY0FKl/o4SYcl838MTM7K3HYgtZr8XlxKdnqIPDpJKtYrgjFEut19LXup++kIEEtg4IXn9nWpRq4LqCdd4eI1El0CXY3tymxWUE4UxQnt9/pWLLycCs/1yMjxRznXxXhkOtg2zqiSvsAec8TwW9ZVfFRmTrLs/Nzt4lUvPdlEkZZqwyh5nRiOoUG0CzId0two+1eDq4q9AIsQif491hV2o8cRwPRYLDDApzs0NsvCpwSw93pvEBSxIfbbk0s6Ro1pNamEdPoQWB5r1xjC/85dz2qTb0SkTfGkQ== Authentication-Results: mx.microsoft.com 1; dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=virtuozzo.com; Received: from VI0PR08MB10656.eurprd08.prod.outlook.com (2603:10a6:800:20a::12) by VI0PR08MB11081.eurprd08.prod.outlook.com (2603:10a6:800:257::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.592.11; Sat, 7 Nov 2026 20:56:34 +0000 Received: from VI0PR08MB10656.eurprd08.prod.outlook.com ([fe80::4e37:b189:ddcd:3dd8]) by VI0PR08MB10656.eurprd08.prod.outlook.com ([fe80::4e37:b189:ddcd:3dd8%4]) with mapi id 15.21.0592.010; Sat, 7 Nov 2026 20:56:34 +0000 From: Andrey Drobyshev To: svt-core@virtuozzo.com Cc: andrey.drobyshev@virtuozzo.com, den@openvz.org Subject: [PATCH hci-dev] vfio/migration: leave the device alone when the guest shuts down #VSTOR-153976 Date: Sat, 7 Nov 2026 22:56:26 +0200 Message-ID: <20261107205626.983714-1-andrey.drobyshev@virtuozzo.com> X-Mailer: git-send-email 2.47.1 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: VIUP296CA0073.AUTP296.PROD.OUTLOOK.COM (2603:10a6:800:349::14) To VI0PR08MB10656.eurprd08.prod.outlook.com (2603:10a6:800:20a::12) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: VI0PR08MB10656:EE_|VI0PR08MB11081:EE_ X-MS-Office365-Filtering-Correlation-Id: 295c48cc-fb87-49a7-7c59-08df3d119b63 X-LD-Processed: 0bc7f26d-0264-416e-a6fc-8352af79c58f,ExtAddr List-Id: svt-core@virtuozzo.com X-MS-Exchange-Transport-Rules-Loop: 1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|52116014|376014|1800799024|366016|38350700014|5023799004|56012099006|261026140227099003|261021121427099003|261009223027099003|261022105027099003|261028144727099003|261026121527099003|261102123227099003|18002099003|5113699003|10067099003; X-Microsoft-Antispam-Message-Info: +FYAJuPsptKmdnscczvk3WHSXymsD1CTC9lIH+maVNNu/O0qpMnb34d7b/eEdC7SwCUWX38QCoC/EoNvjiPeLhlSBbVoGCvLD5jKhkYR/6WFCsdz3ixlQqHtOZeslZWk54TQ6fQO5N6uo2+SBed1LoAzCPrT1I7z2IIRp4UFcysulWOAdfYnMCeyboVP9t+OCTJR2MbFwN2WyVXUnxx9PuSi26wJ4gOX51bcm1VkA+tc6r4t++LsrYpmOAkVd2dskUevs+9D6HWWjE281mmcunkEvbGKyCYP9D42ZR1MpTOa/Ih5HUsutnEwQTDVwkqyN/W5b9ieBTDQjP9QJ/HH7HGIMPS3kN9JpF0XNmnChlvKtYYNgmmlh+lZEsSFi7aJi4lkg0trLFDS7pYL5hsbwqJT86IkXY5RIiDRDPtwjNXjlJSXsDq8/TjLghErmU2d74aIjk0zY+OJrtizPa3i9wnEqHmEvQAyhDNft8Q5FPDAf/bQfDf44AiYlgsJvl2pr4jXhW1eXVkhg2hYJWHvtH7WKQ72zOJi+e0EWKW0nB36HZbllS8epFlg5eLg8bNlc2DirZqSEtOp1Y/Nfb6dgKmnYb6RGPZbWgBtHaJfJEfbtXNEsiRqeaG2WGLv565jE0QlOR3lNB55346VJbKZcT71aXKi9kUqjYZUfm2a3j6ydsKGuYUuxznJeIpcCvivbsYnG6wQKBWBUr+/y4CKubSKDtblmlnk3tj7vENGMnM= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI0PR08MB10656.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(52116014)(376014)(1800799024)(366016)(38350700014)(5023799004)(56012099006)(261026140227099003)(261021121427099003)(261009223027099003)(261022105027099003)(261028144727099003)(261026121527099003)(261102123227099003)(18002099003)(5113699003)(10067099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?VIxAu4QBgi2uYnL7JX9ZmAdfqXb9JWDGJWgWXgCkJKaY+I3hsQykRj35Nvt5?= =?us-ascii?Q?Ofckv9g5qRPEeN3qWdzH5TuurB6ca7jAhCxdJj3lnNdb8bBnwHa7AgDUPbB8?= =?us-ascii?Q?6fpbB9qLOcwGS1QkM+vMX3A3LYIsHJXuuj+qoZsx00v6zoHC8sgcsONZeaJ3?= =?us-ascii?Q?vm7qWjEFtX+fdXpy5cs/5DlfZ2YHiY4lQlWkwi/3r4VNBurDmTFfCYetGJqA?= =?us-ascii?Q?mF7aGd4z367iBkdZlVjs91jN8T8kYgqTfkGcvI44JzV4frJmyNdamx9Cl0i5?= =?us-ascii?Q?MeQj67KWDVNfTM+5+1uhr2LPwFDfeixmFkrDpmuwvu6lvaxgnFvpbkvfJpil?= =?us-ascii?Q?TJglFrt5DPbzpPJ5bcwmV8TbV/7TnrvL50v7G9Q1mQrWv/DJ4fnuUiij0z4M?= =?us-ascii?Q?kLRwN5W2Ixe0TumN01FHHUP5SRjrdpZ0ciCKcG9dfFlUyv3HUxEZ4tMlD4U0?= =?us-ascii?Q?tSxvjlzSm/6MPTE7dr1zWk3P5EGJH4axAtmL9TKrMA+AUJaqTVCE9hQDpUHT?= =?us-ascii?Q?YtN2FcaNDMNCun9L6v9XkBSdlGQCFvKrGSfmlpBtCh+D51FIV7Sy9odfJxgg?= =?us-ascii?Q?UJTlweBvGJN6lh2lpUAQ7URFU0dUB48iFof4jlCNU+h0CPuxDB8ZT9NQE7xQ?= =?us-ascii?Q?N25L9sN80WSFDidP3jKTP4hlixAv3LXQ28tin8WOvBrUPAN0EdOuM9NSB6N4?= =?us-ascii?Q?avuTcz7xTakmatjwpJrwj6etcWPJ3J6JRnQfvbp+NmlpvVRy5IAdRcp3eFsE?= =?us-ascii?Q?jOB6apms2xq3URLH1P9xy7/3kuV7Gvyfs9Myo0kEat4fdqIZc3CXXJP4pipp?= =?us-ascii?Q?Mwx42PH2OcQTbkmYhO0O1bXtRXP51uup6121g0/yU9zlmjLS4FGXjgeBHjMu?= =?us-ascii?Q?KOyEULwy4jRG1ce/vU/uCrY/HX4Xe5sAhwrS8JKksoJ5hUPacGspWuFRmaBg?= =?us-ascii?Q?53axACKl2eX8oCzuo3azv9QIQHiuv0/gYE9d0SUt5bLex1A3kga9qnz4FnUD?= =?us-ascii?Q?Re+BQZmuXa+5noceRchgjHNTxVcHbC8TG90N/W9yHTTKopTNnTWQ3STECry3?= =?us-ascii?Q?DJgBpStG2LgVKD9auLmZVNdi7zLlO5SyT6E87gZXZhXOq2aiDmWDLY74cwDn?= =?us-ascii?Q?xl5EAndEb5Jz5JOsBNzRH0YezNocXzE6aB/59qpPt5haONx127359kF01yNG?= =?us-ascii?Q?xrY/faAl9Q7KTn6MQhMAYkn12PInaj5zXddOinIjYeFe2EuDdxH9B7VzL/aq?= =?us-ascii?Q?8/mfARlVsV1ipvOv95B3i3OXcaVoH/Rgce1WCVkBuvbb6e7kXtQofYHYnQPQ?= =?us-ascii?Q?J0yPcap0xDfSsypKMNIcGZIe4UaNL4ZLuucokQomKng2mbmgWQmgMgvaxqX3?= =?us-ascii?Q?zBYOzRGT7OLoyF3Hox8EOSgSQSE+SaZa2H0XMjLNmTt4ToCraT+wdlwoizbW?= =?us-ascii?Q?utoe+uONaIcqZQt1yv8/2S1Tw6w/2BTQc/I4/hBdHbVRNWXNtqC5FlYFCmhQ?= =?us-ascii?Q?YDqHeXUZDKs4NrirEIsXQemufV7aU48eB/fBUUuNuRh5nwhLRcJSwd46qeeS?= =?us-ascii?Q?B9U9mmKD/52Oa/s6yzu3pUfifZCpR3yJ43vYcSVZESs6AssJup4ZW4gZeJPA?= =?us-ascii?Q?JdRlIZtHCKrKzovQFOG6vUVey4l+A2B3mSO6x++4syzOkHilIU7dmpNp9H4G?= =?us-ascii?Q?Tqt2QVWNJgtjszag0T0Sfmo6S8WqIQsh55RszH1AMFPIPArszTZ+s+YpArC+?= =?us-ascii?Q?2tHlLQbMB/QcFdAoKH3qSCr+NnWunq0=3D?= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 295c48cc-fb87-49a7-7c59-08df3d119b63 X-MS-Exchange-CrossTenant-AuthSource: VI0PR08MB10656.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Nov 2026 20:56:34.2170 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: V2IP6ZE3CUGT/ASojZZ0l1InFwG9wyc0npXamxVhQoORcxgLOUAKTA7qjcOlCdTmmcinK/iiPCkuH4i/nRMGVV61dpT2bcLu0pAnSfq5cd0= X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI0PR08MB11081 When the guest shuts down and QEMU runs with -no-shutdown, the VM stays stopped and can only be reset or quit. A libvirt soft reboot is exactly that: guest shutdown, system_reset, cont. Today vfio_vmstate_change() treats this stop like any other and moves the device to STOP, then cont brings it back to RUNNING. NVIDIA's vGPU manager (595.71.03) takes that STOP for the start of a migration, finds no guest driver, breaks its dirty page tracking and never repairs it when the driver loads again. Every live migration of the VM after a soft reboot then fails: "Error saving page in pipelined mode". Do not touch the device state on a guest shutdown. There is nothing to quiesce, and the reset that must follow resets the device anyway. Pause, migration and savevm still stop the device as before. With this patch applied, soft reboot + subsequent live migration pass successfully. Signed-off-by: Andrey Drobyshev --- hw/vfio/migration.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 7e7fb496bbe..c02b6a199b8 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -899,6 +899,11 @@ static void vfio_vmstate_change_prepare(void *opaque, bool running, return; } + /* The guest has shut down, the device state stays as it is */ + if (!running && state == RUN_STATE_SHUTDOWN) { + return; + } + new_state = migration->device_state == VFIO_DEVICE_STATE_PRE_COPY ? VFIO_DEVICE_STATE_PRE_COPY_P2P : VFIO_DEVICE_STATE_RUNNING_P2P; @@ -936,6 +941,15 @@ static void vfio_vmstate_change(void *opaque, bool running, RunState state) return; } + /* + * The guest has shut down and QEMU keeps the VM stopped. It can only + * be reset or quit now, so leave the device state as it is: the reset + * resets the device too. + */ + if (!running && state == RUN_STATE_SHUTDOWN) { + return; + } + if (running) { new_state = VFIO_DEVICE_STATE_RUNNING; } else { -- 2.47.1