From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from AS8PR04CU009.outbound.protection.outlook.com (mail-westeuropeazon11021126.outbound.protection.outlook.com [52.101.70.126]) by lore.virtuozzo.com (Postfix) with ESMTPS id C5E5D8017D for ; Mon, 9 Nov 2026 15:53:52 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=bwwgVAgIPNwOoJ7+XnfsyX/Ac25YpmVpE5czQOU2ODpIcvTfyUZdl90BL9eS081uZPbUXUB/PPip2/7I7i6ELTE4rtcsoUwsWt/dhIni9J63rMbjB3KmvyunfldWCkJZZP+/5soE/NwVzB+nrSmAPY23GQwzByztvJvqH7N+E+jL2Rb5e+X95QIBBJUyKfv6Fgs2DaFf0bmOpjEsMj+gZM5i1VkhoD/Ptsj1S+q5ovcb8b1XWkNhSgKaQtRxqh9pNN26SvXHQReuQaEe7KyN5Ewg4CaLtP4lLHKDCV6bywmhlXaDH9+pMglp3tJtpeKxjCMqeJ7D5P2PO/eDIMgQww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=gg2ERIM5bDW7AbBsZBYyf0a+KK5OmgFcfKNl6nqQ9Qo=; b=WWvkKKHBxGZ/7wG+zFuUHBYL7y4nVeu1oXSZyKaYGjghd5JOqkEjhfozsnnn8WyD9APECJdmVbnIqEq8PmpIS9BjpbUhj7hemJG5RpUDuk2kRoSS0wDcO6902brnYwFyMWpRyVUqHvzvGP0z3NTTtY8Z9qAxN5HjxZqpSpcPlH9awucdc9Dj59/T518Q1AbgosmfxHKciB7rEs1WZXTc5CD5QerbqhHHpE6C+aIt4mYkXXtDDUlWEMf6miWXy5QpjIaPFPtuYfxeLt+uCGxS6s0VQkMWHYFV+Z7r5py5zbpsUTHLIxTqp43GDlPov4CNSBLlSdoxuDg6nkoErAPrcA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 69.169.224.6) smtp.rcpttodomain=virtuozzo.com smtp.mailfrom=eu-central-1.amazonses.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=sashiko.virtuozzo.com; dkim=pass (signature was verified) header.d=sashiko.virtuozzo.com; dkim=pass (signature was verified) header.d=amazonses.com; arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.onmicrosoft.com; s=selector2-virtuozzo-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=gg2ERIM5bDW7AbBsZBYyf0a+KK5OmgFcfKNl6nqQ9Qo=; b=L3pAlBYq+qTzcq4+JZXgO+drvNfFV4t/wC+acqQ6bcz41jaZ3dQ56DrczMBArb0fyWF61USzOTUkl0Os1qFs3IBjQ4sVddgADeTiGThS1RLCflHeCfIZG6pSFAx02xA7pVNIjGg4kVM8DKAqvu8Ue8wkxBp6VUicMiNRREewwSY= Received: from AS9PR07CA0047.eurprd07.prod.outlook.com (2603:10a6:20b:46b::23) by DB4PR08MB8197.eurprd08.prod.outlook.com (2603:10a6:10:383::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.592.9; Mon, 9 Nov 2026 15:53:48 +0000 Received: from DB1PEPF000509F6.eurprd02.prod.outlook.com (2603:10a6:20b:46b:cafe::9b) by AS9PR07CA0047.outlook.office365.com (2603:10a6:20b:46b::23) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.615.7 via Frontend Transport; Mon, 9 Nov 2026 15:53:48 +0000 Authentication-Results: mx.microsoft.com 1; spf=pass (sender IP is 69.169.224.6) smtp.mailfrom=eu-central-1.amazonses.com; dkim=pass (signature was verified) header.d=sashiko.virtuozzo.com;dkim=pass (signature was verified) header.d=amazonses.com;dmarc=pass action=none header.from=sashiko.virtuozzo.com; Received-SPF: Pass (protection.outlook.com: domain of eu-central-1.amazonses.com designates 69.169.224.6 as permitted sender) receiver=protection.outlook.com; client-ip=69.169.224.6; helo=b224-6.smtp-out.eu-central-1.amazonses.com; pr=C Received: from b224-6.smtp-out.eu-central-1.amazonses.com (69.169.224.6) by DB1PEPF000509F6.mail.protection.outlook.com (10.167.242.152) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.615.4 via Frontend Transport; Mon, 9 Nov 2026 15:53:47 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=23yjbh4k34hwzfubhuiim7m4inbd3ick; d=sashiko.virtuozzo.com; t=1794239627; h=Message-ID:From:Subject:To:Cc:In-Reply-To:References:Content-Type:Content-Transfer-Encoding:Date; bh=QzQmXz0liMPWnpOs6EyT8QgKB/XuO1WCy1O3+y3Wo2Y=; b=kdGG8Cm2Phi08gRpJJz2DTfEV7UdnZWKq9rRq7EjTG1hgdNjmALPEO/X5x3mnCp9 SOby2z8IDu+y2JcBDT1h4OXTv5o9Pv6uwVF8OseydoK5/WkEhUrza1+Jlie7gDwav0E DoZ4YDz9z80q+jPJjl2noCwZVgZgKk1KVHX9F1TbnoWGFPLILhgeaCf/5nA6Pzhc6Pn PkAcofUsmS+S7ekbI/D2FgzpSN49pmXcRo/3QGDk8m3A5ZV8EFf37IaE8JaMmCqoFvj LW6Db6aTd4EqGSoUK5rV1LgYCTsqIdf9Aiq8CEVsy+j2maiqQ6UDVHXe8PUldQRgv6l 5vyeNfC2rw== DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=qftdzk2dqsatjnlrq4r5brjbihpfcrsh; d=amazonses.com; t=1794239627; h=Message-ID:From:Subject:To:Cc:In-Reply-To:References:Content-Type:Content-Transfer-Encoding:Date:Feedback-ID; bh=QzQmXz0liMPWnpOs6EyT8QgKB/XuO1WCy1O3+y3Wo2Y=; b=Bvb9ppISHYxBjd9WifV1cYDGGpy/ymW9AtJRNEvr+JbIUyP1dqQSe7lkvGVOO6ZF 1RDw7Rpf7CTgJTfqtTw+RmIFr5FR58X0M5+So7Y7MxG+RHEGvSSNPECWfhH5a4Pnp3K TzYskBwfP18WkMOWPhvD0AZ2QTHTwCpYXE3aQbK4= Message-ID: <010701a1c10400c7-d72a39c0-ce7a-4582-abb5-91e41913786b-000000@eu-central-1.amazonses.com> From: no-reply@sashiko.virtuozzo.com Subject: Re: [PATCH hci-dev v3 2/2]vfio/migration: don't touch the device on guest shutdown #VSTOR-153976 To: svt-core@virtuozzo.com, Andrey Drobyshev Cc: andrey.drobyshev@virtuozzo.com, den@openvz.org In-Reply-To: <20261109154346.153153-3-andrey.drobyshev@virtuozzo.com> References: <20261109154346.153153-1-andrey.drobyshev@virtuozzo.com> <20261109154346.153153-3-andrey.drobyshev@virtuozzo.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Date: Mon, 9 Nov 2026 15:53:47 +0000 Feedback-ID: ::1.eu-central-1.2wRQJBpg9g1fGRYexx3dUuYGS25Bux+2TEOx72/m6U4=:AmazonSES X-SES-Outgoing: 2026.11.09-69.169.224.6 X-EOPAttributedMessage: 0 X-EOPTenantAttributedMessage: 0bc7f26d-0264-416e-a6fc-8352af79c58f:0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DB1PEPF000509F6:EE_|DB4PR08MB8197:EE_ MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 6420b049-b98e-4396-d357-08df3e79a471 X-LD-Processed: 0bc7f26d-0264-416e-a6fc-8352af79c58f,ExtAddr X-MS-Exchange-AtpMessageProperties: SA List-Id: svt-core@virtuozzo.com X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|32142699015|48200799018|376014|9140799003|61400799027|16102099003|55112099003|4113699003|18002099003|22082099003|5113699003|13003099007|5063699009|261021135927099003|4143699003|5023799004|10067099003|261026121527099003|261026140227099003|261028144727099003|56012099006|261009223027099003|261021121427099003|261102123227099003|261022105027099003; X-Microsoft-Antispam-Message-Info: De6VFSbxMfNwazT5lfV53TNTMsONLHu0rwLHg8xiy2dOHYYp6hiZ+8sfRj7XZ1E6LtZ2ThQaFJVHjkN7f9rJ1qDZFUNZSbHoopoER92mpArHEftIOcnM78HTsfFoLTMAuHEtOZrV6BDIAApxub7ArZXC66OuoPPs0XonaWaUJAtemUo3LAgvmcqNv9WbI6zcbzTQI3K8zq84KfAN9QVH+S3TEa1il8reAcRxkdN5g0qbNyZwfoXy9KUSPRtoGefBJ4YmRSqUXZGjXVVr3p4N633JE/FxqKFZ4S8QeJo4QMhmCrj6C0SzNFGDmsGBftbHiOfp1l47i6/Y1+BAvei3VxmO/ZdEiwqWSy//g+nWHnPiitSssGw210+w2b+ZLXydJJrf6qqMJ11AiMBT/yvJX3Ba/8Sa4royv+Edy1zLA0t/R1M5YwEvjlYzF4M2+F0PCShWou2Q+dzsnJccwoOqKWYBQV9asAslCBA+5aMhMWzp9UmyDg/n5aHfa8S4SfNh+sTR6hCgVF3zkBs3xVGbSNKroKQ0uT/oGzpCKHa0TtYP8f4xFMNj9TkyYiFGOA5uLkN+gPqb/dTfgvyp3j4OgzEWY9aAtl2NMRgrAGFicWrCjQwvigRNQyMHgPt/ckZQUpHKvGB2u/nJ/DAPxqbAziriLQb3SpZIjcXuv5AzOxbbBusG/svbZGUsHOHpxyn2WTrUYYbNVxk9xA2d6Zg7mKMaLz9ujoBz6CwKA/YBrY3oCLbkAJf0+LUWQFSUGk5bFWnc4I5W8GDOWPK4M3S/E2XWK8Q9r11uB1NInfBSyLQwrYIpTIDElMOho/nc4sFBTBL709wH4sxg+3N8mpgOhNQXXvuLa1QgoDMgLa1N0SgK7b/GprNm8tzJYU4rCbEZ4DI+O+atkxUTvmx9pl3z6+HALKZnPC1wjKqOh2oyhz0Vp25/sVRGIejEx6C2AEmp X-Forefront-Antispam-Report: CIP:69.169.224.6;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:b224-6.smtp-out.eu-central-1.amazonses.com;PTR:b224-6.smtp-out.eu-central-1.amazonses.com;CAT:NONE;SFS:(13230040)(32142699015)(48200799018)(376014)(9140799003)(61400799027)(16102099003)(55112099003)(4113699003)(18002099003)(22082099003)(5113699003)(13003099007)(5063699009)(261021135927099003)(4143699003)(5023799004)(10067099003)(261026121527099003)(261026140227099003)(261028144727099003)(56012099006)(261009223027099003)(261021121427099003)(261102123227099003)(261022105027099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: COf/fbDjLt/Bwl0jljeLVLKTIFXG3KlgafM+CmUAgcaebomnZ3PafBdmJdXinRrBbuYHE0isLpx3U5XenM6wBVKZPcTdglPsTPzxRD2d9tlFwMcrM3RuN9MgNubNniK84j2YjfKX25rIB925Bo9/ua8fZu9kXQTe2/1V+JiysKbU5RSoYwT3RofnVFJ6KWP6H5BazSuo0WZzkeJlgeampqq+PpdUsPHlUQQ9hzxbKc37QhKDVf7puJ8gD325B4caGUmui6J/7V+UqG7r65OoiESdKFUWVS3qVreWb3Nm+T1htc7zAb650wRczOI84vP5iubu/1niO/9OUQ/wWAb7gaOD8XqTr7ztRPgI8D7eutvFLzqjJ+X6xAoMCn3nxenp+MqbleyQe25iiNTHro0gMYc4EKbRzWGm/bj+aPTm8jFMv6jW9yW7cJyfWrT1KCXZ X-ExternalRecipientOutboundConnectors: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-Auto-Response-Suppress: DR, OOF, AutoReply X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Nov 2026 15:53:47.9879 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6420b049-b98e-4396-d357-08df3e79a471 X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-AuthSource: DB1PEPF000509F6.eurprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: Internet X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB4PR08MB8197 Thank you for your contribution! Sashiko AI review found 2 potential issue(= s) to consider: - [Medium] vfio/migration: shutdown skip leaves PRE_COPY device unstopped o= n pause - [Medium] vfio/migration: skip-stop predicate matches every pause-action S= HUTDOWN, not only guest shutdowns -- commit 3d78bf00afa301cabb7efe37569c2de747a04bdc Author: Andrey Drobyshev vfio/migration: don't touch the device on guest shutdown This patch makes vfio_vmstate_change_prepare() and vfio_vmstate_change() leave the VFIO device untouched when the VM stops in RUN_STATE_SHUTDOWN and the shutdown action is "pause" (-no-shutdown). It avoids moving the device to STOP on a libvirt soft reboot, which confuses the NVIDIA vGPU manager. A quit or a migration of the stopped VM is expected to stop the device through vfio_migration_stop_after_shutdown(). > diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c > index bfafc14e5b94..454c4e3f553a 100644 > --- a/hw/vfio/migration.c > +++ b/hw/vfio/migration.c [ ... ] > +/* > + * The guest has shut down and QEMU keeps the VM stopped (-no-shutdown).= The > + * guest has released the device, and only a reset, a quit or a migratio= n can > + * follow. The device is left running here: the reset resets it, and the= two > + * other paths stop it with vfio_migration_stop_after_shutdown(). > + * > + * The same run state with the "poweroff" action is a quit of a running = VM, > + * which stops the device as usual. > + */ > +static bool vfio_vm_stopped_by_guest_shutdown(bool running, RunState sta= te) > +{ > + return !running && state =3D=3D RUN_STATE_SHUTDOWN && > + qemu_shutdown_action_is_pause(); > +} [Severity: Medium] Does this predicate cover only guest-initiated shutdowns? It tests only !running, RUN_STATE_SHUTDOWN and qemu_shutdown_action_is_pause(). It does not look at the ShutdownCause. In main_loop_should_exit() the pause action leads to a single call: if (shutdown_action =3D=3D SHUTDOWN_ACTION_PAUSE) { vm_stop(RUN_STATE_SHUTDOWN); This is reached for every shutdown cause. That includes SHUTDOWN_CAUSE_HOST_UI, SHUTDOWN_CAUSE_HOST_ERROR and SHUTDOWN_CAUSE_GUEST_PANIC requested directly by hw/misc/virt_ctrl.c. In those cases the guest driver may not have run its shutdown sequence. The device is then left RUNNING, not P2P-quiesced and not STOP, while the vCPUs are stopped. The claims "The guest has released the device" and "There is nothing to quiesce" are not established for these causes. The predicate also reads the global shutdown_action, which qmp_quit() and qmp_set_action() change at runtime: shutdown_action =3D SHUTDOWN_ACTION_POWEROFF; qemu_system_shutdown_request(SHUTDOWN_CAUSE_HOST_QMP_QUIT); Could the stop-time decision and a later decision then disagree? Would shutdown_caused_by_guest(cause), already used in qemu_system_shutdown(), or a flag recorded at stop time be a more precise discriminator? [ ... ] > @@ -924,6 +939,10 @@ static void vfio_vmstate_change_prepare(void *opaque= , bool running, > return; > } > =20 > + if (vfio_vm_stopped_by_guest_shutdown(running, state)) { > + return; > + } > + [Severity: Medium] Can this early return leave a device that is mid-precopy running? The predicate does not look at migration->device_state. Take a precopy migration that is active, with vfio_save_setup() having put the device in PRE_COPY, and then a guest shutdown with -no-shutdown. Before this patch the device went: PRE_COPY -> PRE_COPY_P2P -> STOP Now both vfio_vmstate_change_prepare() and vfio_vmstate_change() return early, and the device stays in PRE_COPY while the VM is stopped. At switchover, migration_stop_vm() calls vm_stop_force_state(RUN_STATE_FINISH_MIGRATE). The runstate is already stopped, so that only calls runstate_set() and sends no vm_state_notify. Then vfio_save_complete_precopy() requests STOP_COPY with a recover state of STOP: /* We reach here with device state STOP or STOP_COPY only */ ret =3D vfio_migration_set_state(vbasedev, VFIO_DEVICE_STATE_STOP_COPY, VFIO_DEVICE_STATE_STOP, &local_err); Does that comment still hold, given that the device can arrive in PRE_COPY? Is STOP the right recover state in that case? On quit, vfio_migration_shutdown_notifier() calls vfio_migration_stop_after_shutdown(), which has this check: if (!runstate_check(RUN_STATE_SHUTDOWN) || migration->device_state !=3D VFIO_DEVICE_STATE_RUNNING) { return 0; } That returns 0 for PRE_COPY, PRE_COPY_P2P and RUNNING_P2P. The device is then closed in PRE_COPY, or moved to RUNNING by the MIG_EVENT_PRECOPY_FAILED handler. vfio_save_setup() already ran before the shutdown, so its call to vfio_migration_stop_after_shutdown() does not help. The comment on vfio_vm_stopped_by_guest_shutdown() says "the two other paths stop it with vfio_migration_stop_after_shutdown()", and the helper's own comment says "Nothing else leaves a running device in the SHUTDOWN run state". Do those statements hold for a migration that was already in precopy? Would it be better to skip only when device_state is VFIO_DEVICE_STATE_RUNNING, or to extend vfio_migration_stop_after_shutdown() to cover PRE_COPY, PRE_COPY_P2P and RUNNING_P2P? [ ... ] > @@ -961,6 +980,10 @@ static void vfio_vmstate_change(void *opaque, bool r= unning, RunState state) > return; > } > =20 > + if (vfio_vm_stopped_by_guest_shutdown(running, state)) { > + return; > + } > + [Severity: Medium] The same early return in vfio_vmstate_change() skips the STOP transition for a device in PRE_COPY, as described above, and for the non-guest shutdown causes listed in the first comment. --=20 Sashiko AI review =C2=B7 https://sashiko.vzint.dev/#/patchset/2026110915434= 6.153153-1-andrey.drobyshev@virtuozzo.com?part=3D2