From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from DUZPR83CU001.outbound.protection.outlook.com (mail-northeuropeazon11022092.outbound.protection.outlook.com [52.101.66.92]) by lore.virtuozzo.com (Postfix) with ESMTPS id E14D481404 for ; Sun, 8 Nov 2026 20:51:10 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=pcEMF26iodUvCBz9W43Eki0s7ssC//XjyoqhtzYaVs5jpzb+5RFj0/XOIkULJjGDerLthl4QUKEhq0hDHA7jjcnqblY9iMjzIq0rmAdjwnnvqzKUlnZWIkq7Z/PQwXrySWyc8q/B4SL/4cE6ZDkYT1zMHKRLeNbSvqWQ3MBmbkVQJf7Gfo3rhzswcIS0coFgCf2GvZebe/jkj1qh7B17++iC400ncx1Ad2hBg8PGdLUr0TvrmY0ejhdG+aM8KxnATo0fRqyHBykTfV/kUYfF5Vodf9cqjYYOjPrcRYCk/cP7q5JkuUi8mQjqlvse5CBlwqa2aIdTHKSu2uUPeWHT6w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3D8vcGH9Wqk4F0QWcg8VmXTmKzk6H/qGyGZ06PpzCrg=; b=OVVfq3DfPQqn9rohuK6s9Dfb8uissCHyu3eQwhwNnUDcgg9B2XLYRCDE0H0/F9V9gaV82FLedeBOva8xT4UPsp20cF4is8nnMNvBZe+Jo4K3gJfWfAj5s8vLD1ekb+qaYdG2jzzaLMA3Te28peTqIzhGXU4meCYp9zLECAor2kUpHswQq5fZaEKsY6rBxUD+hvAD2GhXSwTFPf5I2ERudaov6e5EcAqhhRznOnn3g9mnuEcmbmD3wjWSDdJA5K+7jTSfyHoDCe9BHljTzRTCI3Yus+38W4JgcPPGS8CK2xdlO04McMnfmmKsM+Ril/pFZcOfHDNb57gi6oUNQavbHg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3D8vcGH9Wqk4F0QWcg8VmXTmKzk6H/qGyGZ06PpzCrg=; b=PhG/mX2eoqCZ97v1c03o84ph9liwRPv4vdoBsLUnYIcK3LAn2N+igymNJlLz0GWV0hyHCkIaI74/K+1wAZXjRtvZ9ElJT7/BU6/+wWoWtemJ9/WIjV/jaXO24my9slSEiVnQZ7HkwCr8VTK610b1TAvipH58oB3bdjv6zeW1tjbUYnQGCdnBrxtw9GlFvShjHeSJIDfC3uDZ6A3bZ5j0pwH7Wd3FpDXu2Nx11taxlIG6WiTmg3xBcGqh7/SW5eH+JUb0gpXrqtRDOzPSqfby9fL47D5oREfScBKd7o1Vqru1O46K3qZWhzfJ04i8VB0yT+AalFYr4HocLgH6+OnfLg== Authentication-Results: mx.microsoft.com 1; dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=virtuozzo.com; Received: from VI0PR08MB10656.eurprd08.prod.outlook.com (2603:10a6:800:20a::12) by AS8PR08MB8298.eurprd08.prod.outlook.com (2603:10a6:20b:536::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.592.12; Sun, 8 Nov 2026 20:51:07 +0000 Received: from VI0PR08MB10656.eurprd08.prod.outlook.com ([fe80::4e37:b189:ddcd:3dd8]) by VI0PR08MB10656.eurprd08.prod.outlook.com ([fe80::4e37:b189:ddcd:3dd8%4]) with mapi id 15.21.0592.010; Sun, 8 Nov 2026 20:51:07 +0000 From: Andrey Drobyshev To: svt-core@virtuozzo.com Cc: andrey.drobyshev@virtuozzo.com, den@openvz.org Subject: [PATCH hci-vev v2 1/1] vfio/migration: don't touch the device on guest shutdown #VSTOR-153976 Date: Sun, 8 Nov 2026 22:50:50 +0200 Message-ID: <20261108205050.37277-2-andrey.drobyshev@virtuozzo.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20261108205050.37277-1-andrey.drobyshev@virtuozzo.com> References: <20261108205050.37277-1-andrey.drobyshev@virtuozzo.com> Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: VIVP296CA0101.AUTP296.PROD.OUTLOOK.COM (2603:10a6:800:355::18) To VI0PR08MB10656.eurprd08.prod.outlook.com (2603:10a6:800:20a::12) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: VI0PR08MB10656:EE_|AS8PR08MB8298:EE_ X-MS-Office365-Filtering-Correlation-Id: 8a541cbb-fdef-40d6-9242-08df3dda02a6 X-LD-Processed: 0bc7f26d-0264-416e-a6fc-8352af79c58f,ExtAddr List-Id: svt-core@virtuozzo.com X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|52116014|23010399003|376014|1800799024|366016|18002099003|22082099003|5113699003|10067099003|5023799004|261028144727099003|261026121527099003|261102123227099003|261022105027099003|56012099006|261021121427099003|261026140227099003|261009223027099003|38350700014; X-Microsoft-Antispam-Message-Info: h1zw+Pg5/c4j8attO99BgtFUX1pOa/0RxrK2hVWM898Kp3+xx54eG54L14e4hZaihqiiGtYXZthHRzUxT4WObRTKIieC+nRxNNYfoqbHlQmIiY2xko0hiDh/Og7QZ/oVQld1Z6XSg9T3oxgeVfQODgG9mcqlTJvsrQ1SWrGnViIzfZ2R2P7o5U+wtS+uIz/lIe/Mv1kPhxNNUf2zxzqGrtHKLd4k6HS0neuK2BKAwGdRhkfdrgxWmXeXXXPEjoJyqzqELTyMh+MIvN3wTLU6qyne7yddRzQQqnwKCyhfB2FegnR1Xgs3zP4rqIKYMqhrDO2O1m5DxTDlLOmTomi6rOUBVDCz2gWRRwepCBQfDjrL9t8gtIteRvqo9vnfRkn5jY0Y3P2YhVCGmlXtYk7wMHk9yH/92YzGy9FhQKJF/ZqGqfYkyAhGgYmxcAJGgBCKDDVQK7IZHkgnYyoGanZsao/+pypNNff9nqa9wMztQa2eOQKT80X83BP+z5WoxWVyjtum521ihYU2pC+p5MoVy+ddx2YwRooyzKJEYuWfYgpCiQXYC/pUcG/LwP0NLHq/Y0ERbJVwZ83lAPicxIiTKA9X3n34AEltwWdbBM8oZRLAaFRO6dLihDk/0EfWeJJYIaQT7cPxroAg36H2zl9oBepjKNi9DGQ/rcDljnyhhKo3Cbjt7ka0vQDEI0Oo2v5aBY46HN5ZWl9b2b4k/gkBzjH117c30vtHLfbVHK3VKPA= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI0PR08MB10656.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(52116014)(23010399003)(376014)(1800799024)(366016)(18002099003)(22082099003)(5113699003)(10067099003)(5023799004)(261028144727099003)(261026121527099003)(261102123227099003)(261022105027099003)(56012099006)(261021121427099003)(261026140227099003)(261009223027099003)(38350700014);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?sA15Is2C4N09NWGgIcF0VoNQu+80Kw0hYGqqIZ5LHm+8rv9yoRp2wvhO6sKa?= =?us-ascii?Q?uTIp7YDwP+xWYM6j0rJTfDZBdP5NFCjXPGvWAYpagGKk9th/wsMIQxQya3Vt?= =?us-ascii?Q?Q8SR73eTpkcZR62JAfv37wTP8XAWtR249R8oz/dCOGHZxsBa8eh3eXZ7jKtO?= =?us-ascii?Q?wfl9PJZwW2qXxyMuKHIpJlyGZcLJ4+Ce7bd9p67/5js0ysavGMM/NcCxalbC?= =?us-ascii?Q?s0N2HKayM326CPj1855uXnVq/3knUqna11CxLBzt7NwCa1BPU6M7zZCU+3xu?= =?us-ascii?Q?EWtNFnRtukLy/i5se0FLHSEgD++zGYqjdaIRenEQZyqgPt6/Mg+iAa9ASjRU?= =?us-ascii?Q?/u1qiuPYB4L+ch27dYJEafSVkTQIvrp3txb9j1ejvPmsF7H+HawcE0kVVvXc?= =?us-ascii?Q?cr423NuJWKH749hHfQnM3RKQ+9IGeUHOqV9Q19nJUAi2RX+I1R/9vm0BFHLH?= =?us-ascii?Q?krG4Rf3ahSp77gQgHHRaTUq/x1tK7hT4xNZIrbJinn4pgGId+FHM3teiemTL?= =?us-ascii?Q?Lb/XLmBFRmHsr1QbQu90VnzutpUTn4HaJ+dtCQbMY6o+MTZfNCfyMaKRNqXj?= =?us-ascii?Q?V+u5Tu3BJrYr4s3vQ7wnTwrAew0rNiIpOo+sE/jTH3EgBJb1C4OffbW991II?= =?us-ascii?Q?VjrAx3dnBPsrVFv34YZ8Fn/eCf7xwOfG/3FQj93CVAquFPdo0yyFg5P59bF3?= =?us-ascii?Q?qoGsJRYt/fmttu8MIYGWYvXmmmEG+TPaJuTtOKb0fSEqy2kM9sX8f5RC3CwK?= =?us-ascii?Q?HdvPV+PzWRDE3O5R3J4WFrrdjOmZUGINjwyioO/weqOMgQDNBFELqrbtESGF?= =?us-ascii?Q?T5c50ArKdt4yTMd7KiKII1UmeRV5alZlrCu4rJNxJw9NIeuAq211yVfKmSyL?= =?us-ascii?Q?upM7XC6VieSSALGHF4cmqif6YPtIHjtAAG9nIMKnZhbxB7CX9lwXSRJK7ab7?= =?us-ascii?Q?XZsMgGcFEyotIMXtSiUej1GaMlgTLagehoW//LSdBtngVN/O9T4aQs9nOmU3?= =?us-ascii?Q?XSNLc9pCHFONB2aD/KDU7W27mbcSO9LeScxUPi5t0hi0BDTLa35G3BwTNctO?= =?us-ascii?Q?n9YBLmWeMY1z9Gy6N7R607IwGzrSX9uhmTyEeETXlTR5wl8XV36cxdz0ZizO?= =?us-ascii?Q?MGmy8WsAFrrwjGxQpdzmv/M1xvBn98cJFnfCzmSu0JPGMi1F8Sqa+r58UF8Q?= =?us-ascii?Q?oC4QH01L93RtBKXTU8812NlKXzpPzTMYdpNT3y6FYS3T9Rmg+1pk4p3RveFL?= =?us-ascii?Q?av2WNd/0oQhqt7aJZMXxmyU9SwczDkrS6CCWxJxtnmK7EoHDMYFOxRxrhJSf?= =?us-ascii?Q?66gdrKEAUvbOEJ2zEd3vjuzzJEkERYWGBQeWfFEvYFdWLJk00BMAECdb6Czx?= =?us-ascii?Q?C0Y8XnfmaT9b4sPBuxyim2RyLcU9mD+tZtEmJKha5WtlLhorMRrvc9Kg26Ap?= =?us-ascii?Q?ppmNCNlJmH819y9PNSRF6+5jORwVyCZ32rQxlifd+eEqImvOx7PUSY5Nw2Ui?= =?us-ascii?Q?/Xe/kpiQiwltS2D4LAnM+3s+KeGfeCXPDNfxPN7L/lfyg8gUBnQryGOFNjFe?= =?us-ascii?Q?2w9+kDwRqWlc1HO5UGILboGykPNuH5isrBnvZEQUksFTwmijpt8NyQtpDrBn?= =?us-ascii?Q?86xHGfoCf36bqs27exqBOtfp+GFJP3jS+9hforkDAa00VyfT0uVGP1rBPYZq?= =?us-ascii?Q?l0b25wS30uoWV8r/zIyIUMpspkOpmDAoFkUMYtoiFOXMjLSsgOkMWa4wralV?= =?us-ascii?Q?pB+shh5e7yHcG+66Yd5wWOT/4ucye18=3D?= X-Auto-Response-Suppress: DR, OOF, AutoReply X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8a541cbb-fdef-40d6-9242-08df3dda02a6 X-MS-Exchange-CrossTenant-AuthSource: VI0PR08MB10656.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Nov 2026 20:51:06.7449 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: hZp/eoPE7dfL0NK6UfGVJLfgik6IDxdf1QVxGYSiWS7YDZRi9VNaDVhUoZy5gGAh1xo/+coxgOJlERi1jcEqaYY4ZC0S61818Q3+/pX9Q4o= X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR08MB8298 When the guest shuts down and QEMU runs with -no-shutdown, the VM stays stopped and can only be reset or quit. A libvirt soft reboot is exactly that: guest shutdown, system_reset, cont. Today vfio_vmstate_change() treats this stop like any other and moves the device to STOP, then cont brings it back to RUNNING. NVIDIA's vGPU manager (595.71.03) takes that STOP for the start of a migration, finds no guest driver, breaks its dirty page tracking and never repairs it when the driver loads again. Every live migration of the VM after a soft reboot then fails: "Error saving page in pipelined mode". Do not touch the device state in this case. There is nothing to quiesce, and the reset that must follow resets the device anyway. The case is told apart by the shutdown action: only "pause" (-no-shutdown) keeps the VM alive. quit, a signal and a plain guest shutdown use "poweroff" and still stop the device on the way out, as before. If a migration or savevm is started on the stopped VM, no VM state change comes at switchover, so vfio_save_setup() stops the device itself. With this patch applied, soft reboot + subsequent live migration pass successfully. Signed-off-by: Andrey Drobyshev --- hw/vfio/migration.c | 30 ++++++++++++++++++++++++++++++ include/system/runstate.h | 1 + system/runstate.c | 5 +++++ 3 files changed, 36 insertions(+) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 7e7fb496bbe..2ff3e49d5de 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -473,6 +473,19 @@ static int vfio_save_setup(QEMUFile *f, void *opaque, Error **errp) migration->event_save_iterate_started = false; migration->event_precopy_empty_hit = false; + /* + * A VM that is already stopped gets no VM state change at switchover. + * If a guest shutdown left the device running, stop it now. + */ + if (!runstate_is_running() && + migration->device_state == VFIO_DEVICE_STATE_RUNNING) { + ret = vfio_migration_set_state(vbasedev, VFIO_DEVICE_STATE_STOP, + VFIO_DEVICE_STATE_RUNNING, errp); + if (ret) { + return ret; + } + } + if (vfio_precopy_supported(vbasedev)) { switch (migration->device_state) { case VFIO_DEVICE_STATE_RUNNING: @@ -899,6 +912,12 @@ static void vfio_vmstate_change_prepare(void *opaque, bool running, return; } + /* The guest has shut down and QEMU keeps the VM stopped: leave it */ + if (!running && state == RUN_STATE_SHUTDOWN && + qemu_shutdown_action_is_pause()) { + return; + } + new_state = migration->device_state == VFIO_DEVICE_STATE_PRE_COPY ? VFIO_DEVICE_STATE_PRE_COPY_P2P : VFIO_DEVICE_STATE_RUNNING_P2P; @@ -936,6 +955,17 @@ static void vfio_vmstate_change(void *opaque, bool running, RunState state) return; } + /* + * The guest has shut down and QEMU keeps the VM stopped. It can only + * be reset or quit now, so leave the device state as it is: the reset + * resets the device too. The exit path uses the same run state with + * the poweroff action and stops the device as before. + */ + if (!running && state == RUN_STATE_SHUTDOWN && + qemu_shutdown_action_is_pause()) { + return; + } + if (running) { new_state = VFIO_DEVICE_STATE_RUNNING; } else { diff --git a/include/system/runstate.h b/include/system/runstate.h index 929379adae4..29a919348da 100644 --- a/include/system/runstate.h +++ b/include/system/runstate.h @@ -148,6 +148,7 @@ void qemu_system_vmstop_request_prepare(void); bool qemu_vmstop_requested(RunState *r); ShutdownCause qemu_shutdown_requested_get(void); bool qemu_force_shutdown_requested(void); +bool qemu_shutdown_action_is_pause(void); ShutdownCause qemu_reset_requested_get(void); void qemu_system_killed(int signal, pid_t pid); void qemu_system_reset(ShutdownCause reason); diff --git a/system/runstate.c b/system/runstate.c index 6178b0091a3..1cdecde0fcf 100644 --- a/system/runstate.c +++ b/system/runstate.c @@ -433,6 +433,11 @@ bool qemu_force_shutdown_requested(void) return force_shutdown; } +bool qemu_shutdown_action_is_pause(void) +{ + return shutdown_action == SHUTDOWN_ACTION_PAUSE; +} + ShutdownCause qemu_reset_requested_get(void) { return reset_requested; -- 2.47.1