From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <010701a1b7cff882-2e4d6e19-0372-4865-8ed1-19d654b0ba16-000000@eu-central-1.amazonses.com> Received: from PA4PR04CU001.outbound.protection.outlook.com (mail-francecentralazon11023111.outbound.protection.outlook.com [40.107.162.111]) by lore.virtuozzo.com (Postfix) with ESMTPS id C80B881299 for ; Sat, 7 Nov 2026 21:00:28 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=cTpfjEuLxWRNvALFujZcC7+fPumRFFmgXvvnVtgdZqqPLzP/d2RqwpR+QQIf7BfeKcVenJWDGp/vAKE+arsZcxJhvk6MLPM6Gyfc0xRK5xsqbabbLdVSsz/uaeYriWHw2fyg87bW4mAnz0y3TFJrF15fCAwWYsDVDmMTPYWaMUYzXAT0bUDOtqkjftMFjKDeh3dVHMIW+nHb2LHHOvAgIlMZ8+sp/FiGjNzAZILMhiK+qoILFIxtUWaQGobHiHvF++0f2adozQbCtKVqEbWfQgPgABjCIJV8ZNrWovH+KfxeewX5zMsCQUTOIfPAlTe38Chu+HXt8Sp+NVavd7eEYQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CrLiuo/uINDGj/Yjz+39IzMjrbdVREbP7NVu+cbqcUc=; b=ShWcqyuffWzC/16GrVojkpyH5/0mEQNi4k8oxUW/is7Z4XcwZMVg54QZYV77PWjC5WSs2rflx08gJe408PomrgdO4IfZHJVqy5BUfZqKymjCFPkcOgRNNPcIirZy54w15HMqFS0m824ZTGYAdPZZ0TkfGHtT5SVGRBj+nde5WTUtRs/4kyB+vTqPsNCAintEAlnB3cGJKOM5SiTe53B7CDAsJNSkaCkEKdx7w+CspEZyhn6BPokQySI7qeWLOQt28n0bjfjebSrG937vHjhZi7aXNwaSd3hE6gR5EDvcvIOIPJYYCxv8aBTm3x8gtilQYDJ1xq/27RslqhC4R+xVcQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 69.169.224.5) smtp.rcpttodomain=virtuozzo.com smtp.mailfrom=eu-central-1.amazonses.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=sashiko.virtuozzo.com; dkim=pass (signature was verified) header.d=sashiko.virtuozzo.com; dkim=pass (signature was verified) header.d=amazonses.com; arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.onmicrosoft.com; s=selector2-virtuozzo-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CrLiuo/uINDGj/Yjz+39IzMjrbdVREbP7NVu+cbqcUc=; b=JlUWZYmC5jnGdm31wvWdUm0QKCOVdffmgP/sX/aZjAsR4QJhsziav1RixRrx5no4l+ocSX9/eDUGvuPDPHd1w+yJjkkoUHhb/NKvRJBxi3x2dkqKfyYwAEb3JQxgupkuLsOMUcFb0+HV+tw15IIF2EnB3AaDxjIp/BTPXN7X5l4= Received: from PR1P264CA0126.FRAP264.PROD.OUTLOOK.COM (2603:10a6:102:2cd::20) by DU5PR08MB10466.eurprd08.prod.outlook.com (2603:10a6:10:516::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.592.11; Sat, 7 Nov 2026 21:00:24 +0000 Received: from MAD0EPF000008AE.eurprd04.prod.outlook.com (2603:10a6:102:2cd:cafe::1e) by PR1P264CA0126.outlook.office365.com (2603:10a6:102:2cd::20) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.592.11 via Frontend Transport; Sat, 7 Nov 2026 21:00:24 +0000 Authentication-Results: mx.microsoft.com 1; spf=pass (sender IP is 69.169.224.5) smtp.mailfrom=eu-central-1.amazonses.com; dkim=pass (signature was verified) header.d=sashiko.virtuozzo.com;dkim=pass (signature was verified) header.d=amazonses.com;dmarc=pass action=none header.from=sashiko.virtuozzo.com; Received-SPF: Pass (protection.outlook.com: domain of eu-central-1.amazonses.com designates 69.169.224.5 as permitted sender) receiver=protection.outlook.com; client-ip=69.169.224.5; helo=b224-5.smtp-out.eu-central-1.amazonses.com; pr=C Received: from b224-5.smtp-out.eu-central-1.amazonses.com (69.169.224.5) by MAD0EPF000008AE.mail.protection.outlook.com (10.167.241.187) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.615.4 via Frontend Transport; Sat, 7 Nov 2026 21:00:23 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=23yjbh4k34hwzfubhuiim7m4inbd3ick; d=sashiko.virtuozzo.com; t=1794085222; h=Message-ID:From:Subject:To:Cc:In-Reply-To:References:Content-Type:Content-Transfer-Encoding:Date; bh=n7hU2PFyTXhMfloTDCpcsqh7IBPrss/LgU+x3hEGslw=; b=sZ7pjUlMCnVAkSnb9rJKAUGUV/w3C5WJNnv3l/ZhWPf0RVh0KjBqpt8HGaxG2x0s +x4Mhj9ZVbmMMymZ4qmqpohhRZS6/gQUYOrzp++ZAOpGo1QV0pJH0bHbY1/7mj8bRgR RBWZs8ycfJCw0Y+Yso6d93rByPmY9fXD+tvW+VJS+At9Nd/hp0zn6rjYNDnHtQCYbWV IPXTwzfJpRvNnNzjVVpGsU9nHBasfGBo9DGqQMht2y2K6nfGM3i7OUXUTPMvsOw+Q41 NOrqgSk4Re+gmGB0/Px4Cjtg2fAgo0VA22Z79kj0h4mDB7q1OyTFad62LCp2hq6oaxb RSJLJAP1pg== DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=qftdzk2dqsatjnlrq4r5brjbihpfcrsh; d=amazonses.com; t=1794085222; h=Message-ID:From:Subject:To:Cc:In-Reply-To:References:Content-Type:Content-Transfer-Encoding:Date:Feedback-ID; bh=n7hU2PFyTXhMfloTDCpcsqh7IBPrss/LgU+x3hEGslw=; b=NrPeaV0jwqtsG4xBfN4+72qdccyjD8C8vJ1XEvj0mAeCzGgegwxwgwMjRjeQfdkk DX7+C7t/4WyBECy0SrYAZaPpsb6uFrh8Y8q3Wbq3jq2zW/VEGfc14HmmL1zBPenVLDt 05qGk8MA4Ylcy8XAnHop6XNLeZ74eSwG8sQBIgVQ= Message-ID: <010701a1b7cff882-2e4d6e19-0372-4865-8ed1-19d654b0ba16-000000@eu-central-1.amazonses.com> From: no-reply@sashiko.virtuozzo.com Subject: Re: [PATCH hci-dev] vfio/migration: leave the device alone when the guest shuts down #VSTOR-153976 To: Andrey Drobyshev , svt-core@virtuozzo.com Cc: den@openvz.org, andrey.drobyshev@virtuozzo.com In-Reply-To: <20261107205626.983714-1-andrey.drobyshev@virtuozzo.com> References: <20261107205626.983714-1-andrey.drobyshev@virtuozzo.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Date: Sat, 7 Nov 2026 21:00:22 +0000 Feedback-ID: ::1.eu-central-1.2wRQJBpg9g1fGRYexx3dUuYGS25Bux+2TEOx72/m6U4=:AmazonSES X-SES-Outgoing: 2026.11.07-69.169.224.5 X-EOPAttributedMessage: 0 X-EOPTenantAttributedMessage: 0bc7f26d-0264-416e-a6fc-8352af79c58f:0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MAD0EPF000008AE:EE_|DU5PR08MB10466:EE_ MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: f86af70b-4f54-488f-743f-08df3d122410 X-LD-Processed: 0bc7f26d-0264-416e-a6fc-8352af79c58f,ExtAddr X-MS-Exchange-AtpMessageProperties: SA List-Id: svt-core@virtuozzo.com X-MS-Exchange-Transport-Rules-Loop: 1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|48200799018|32142699015|9140799003|61400799027|16102099003|22082099003|18002099003|6133799003|4113699003|5113699003|55112099003|10067099003|13003099007|261026121527099003|5063699009|261026140227099003|261028144727099003|261022075727099003|56012099006|261021121427099003|261022105027099003|261102123227099003|261009223027099003; X-Microsoft-Antispam-Message-Info: 9eYg/nQ7TADZVnoncIkwTiLGnB6ZcS/xZjCWjEOKY2XMNdVgfOSYnfP3VygfbVwVk6CPp9FFDKIeBIsjuODnJHur54JUVRjEzt6qJhLtvU5RS6g56ssT7tsA0tost6seSvAIHGTz01L5NppA+X4/sa7rruc51w0sJvSVwbJ0j6INfoJQ5/bJf42rz0IpQwl6BFg3Qe8VcNKEd/9/vD/IqSPqH6nhHXLrQJNjuiBPrfhdAVUobl0DydQ3zZhDMQBxaRxQUGfG4pVa1ptyipIlb05iE1SgsF+HOLt/jFtEtAB9YQQP9nxk+Xs/OuG9JcZAwmTJT5HD19HgJJovYkKawtGP+LHl7wdIx67xgcNcgM+Oc8GzmnQ10yYcapkbuANO68UrF65ABNOPis8+A3uBt/Lz144F45tCevycvk5DVvDl0BvETK5BXGNs5CmVnykWMzYPutV5oiPX9keKh6e3NGoZI9XjSBPL8Cj1p0OCG0iQYA8XSIzvtJnUQ/FSpKiKiVsZlHlDofCQxQmh23GgxExudroxK229bHTKqvXzg2rBh/bD1ZRYT+jLwbVxhI7GYgioqriXW1DGK838uGaySPE+X/f17qaOBPH9AlS6vx808tfbju4XlXWFkeumQGC6g2UZj3Q77UgwfgSHLh4KTajrrdNSyl+Z0VAlhnNitlTUFueKwpzVSyCJpYOBXIe09NvQDtjKkNU5kXRoXutw7eIRWHOgO4sErIp5R6cGKWSE1FKA9dpmThAVTwVyPjptmDSqCjuUW8L2/3JCgjJMCXf4e0dmqG4IdPdvoPS9RoTYrR96MVzi6tv/fQo9933HeC2LTUcfpCBHwLTXMiDgEP6oAB4T5mbnc6iM03QFeqL32Xk4j9+mgZmOAeLp590Q X-Forefront-Antispam-Report: CIP:69.169.224.5;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:b224-5.smtp-out.eu-central-1.amazonses.com;PTR:b224-5.smtp-out.eu-central-1.amazonses.com;CAT:NONE;SFS:(13230040)(376014)(48200799018)(32142699015)(9140799003)(61400799027)(16102099003)(22082099003)(18002099003)(6133799003)(4113699003)(5113699003)(55112099003)(10067099003)(13003099007)(261026121527099003)(5063699009)(261026140227099003)(261028144727099003)(261022075727099003)(56012099006)(261021121427099003)(261022105027099003)(261102123227099003)(261009223027099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 9/Fxc/9IBDAU5xKrmQvPZSLZq7y70AnDOAUUs1ul/RImDo3btp1WCyni0sam8JXUg6iASO6TVlY82ozy9l1xKcET/L6zQeVV3e3cMM4ghG/6ChX0nqabOqnuPQo1GsFp8or5svGpf1LDVK21bldqux9t1g8p4DbngM02rwNZdfDTzl74l9EAc2GTWl+3++L81ttQ04efZH57lYfWaAsCmVknkRwPQ9sz2VBJ5iT7JyOkx3mNvBpPPaI8Ns+8G0ghY/cM72eDvEvaHFORznzA4EYji+0oX+37pGbxxrxjeSPeD5pa6T/LbPj9HTKXVcNrBtEgABh44GF5GaEInDLHhmfm4iF985EoNX6B3iGByGxRFZt4BJAmu5C1W2CJQaqUOstIJl0qa/mSMCkU0eXgaqv7N2kpfJd1E91cJ1CVHAIMFIj/KQdFuRVNEnYrGQgy X-ExternalRecipientOutboundConnectors: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Nov 2026 21:00:23.2728 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f86af70b-4f54-488f-743f-08df3d122410 X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-AuthSource: MAD0EPF000008AE.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: Internet X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU5PR08MB10466 Thank you for your contribution! Sashiko AI review found 3 potential issue(= s) to consider: - [Medium] vfio/migration: SHUTDOWN guard leaves device RUNNING when migrat= ion starts from SHUTDOWN - [Medium] vfio/migration: RUN_STATE_SHUTDOWN guard also skips device STOP = at process exit - [Low] vfio/migration: duplicate RUN_STATE_SHUTDOWN guards in vmstate hand= lers -- commit ebf52957cf374c28d589b66aa4bf2f9c983c8afc Author: Andrey Drobyshev vfio/migration: leave the device alone when the guest shuts down #VSTOR-153= 976 This patch makes vfio_vmstate_change_prepare() and vfio_vmstate_change() return early when the VM stops with RUN_STATE_SHUTDOWN. The device state is then left unchanged on guest shutdown with -no-shutdown. The aim is to avoid the vGPU manager treating the STOP as the start of a migration. > diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c > --- a/hw/vfio/migration.c > +++ b/hw/vfio/migration.c [ ... ] > @@ -904,6 +904,11 @@ static void vfio_vmstate_change_prepare(void *opaque= , bool running, > return; > } > =20 > + /* The guest has shut down, the device state stays as it is */ > + if (!running && state =3D=3D RUN_STATE_SHUTDOWN) { > + return; > + } > + [Severity: Low] This isn't a bug, but the tree this was reviewed against already has an identical guard in vfio_vmstate_change_prepare() just above this one, so the added copy is unreachable: /* The guest has shut down, the device state stays as it is */ if (!running && state =3D=3D RUN_STATE_SHUTDOWN) { return; } Could the patch be respun so that only one guard remains? Because the guard is already present before this patch, the commit message claim "With this patch applied, soft reboot + subsequent live migration pass successfully" cannot be attributed to this change. [Severity: Medium] Does this guard also match the process-exit path? qemu_cleanup() calls vm_shutdown(), which calls do_vm_stop(RUN_STATE_SHUTDOWN, false). If the VM is live, vm_state_notify(0, RUN_STATE_SHUTDOWN) reaches this handler with running=3Dfalse: qemu_cleanup() vm_shutdown() do_vm_stop(RUN_STATE_SHUTDOWN, false) vm_state_notify(0, RUN_STATE_SHUTDOWN) vfio_vmstate_change_prepare() The guard keys only on the run state, so the transition to RUNNING_P2P or PRE_COPY_P2P is skipped when QEMU quits from a live VM. The commit message justifies the skip with "the reset resets the device too", but no reset follows on the quit path. vfio_migration_exit() and vfio_migration_deinit() only remove the notifiers and the vm-change handler, and do not stop or reset the device. Can the guard distinguish a guest-initiated shutdown with -no-shutdown from the vm_shutdown() use of the same run state? > @@ -950,6 +955,15 @@ static void vfio_vmstate_change(void *opaque, bool r= unning, RunState state) > return; > } > =20 > + /* > + * The guest has shut down and QEMU keeps the VM stopped. It can only > + * be reset or quit now, so leave the device state as it is: the res= et > + * resets the device too. > + */ > + if (!running && state =3D=3D RUN_STATE_SHUTDOWN) { > + return; > + } > + [Severity: Low] This isn't a bug, but the same guard and its five-line comment are duplicated in vfio_vmstate_change(), where an identical copy already exists earlier in the function. The second copy can never execute. [Severity: Medium] Can migration or savevm that starts from RUN_STATE_SHUTDOWN now leave the device running? With this guard, the device stays RUNNING or PRE_COPY after guest shutdown. The migration path then looks like: migration_stop_vm(s, RUN_STATE_FINISH_MIGRATE) vm_stop_force_state() /* runstate_is_live(SHUTDOWN) is false */ runstate_set(state); Since SHUTDOWN is not a live state, vm_stop_force_state() only sets the run state and sends no vm_state_notify(). The runstate table allows SHUTDOWN -> FINISH_MIGRATE, so the vfio notifiers never run. vfio_save_complete_precopy() then does: /* We reach here with device state STOP or STOP_COPY only */ ret =3D vfio_migration_set_state(vbasedev, VFIO_DEVICE_STATE_STOP_COPY, VFIO_DEVICE_STATE_STOP, &local_err); The comment is no longer true, and STOP as the recover state is also wrong if the device was RUNNING or PRE_COPY. Could a device that was never stopped dirty pages after the final RAM sync? The savevm path goes through vm_stop() and do_vm_stop(), which also notify only from live states. I did not trace that path to its end. Before this patch the device was already in STOP in this scenario. Could the guard be restricted so that migration and savevm from SHUTDOWN still stop the device? --=20 Sashiko AI review =C2=B7 https://sashiko.vzint.dev/#/patchset/2026110720562= 6.983714-1-andrey.drobyshev@virtuozzo.com?part=3D1