From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from OSPPR02CU001.outbound.protection.outlook.com (mail-norwayeastazon11023078.outbound.protection.outlook.com [40.107.159.78]) by lore.virtuozzo.com (Postfix) with ESMTPS id 7426A80E7E for ; Mon, 9 Nov 2026 17:01:49 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=QCb09U/1KLmk9EUcPc10GcBvmkcAQoLs7sc+biyu6wciaP3YerMEUlr05CnKXQasAxuu5mmObBRVeFMO7h7TDQf6U/X2eDirC+ji2J3K1DBSWmlDHFrBIVfW5dKQqXTmEvCZ59fFo0tfhmoUL5Szu4WEKafSxfHDr/p/8jY1t9XcxTyxyCR53k2ngLqzc4CJNRZQgBvbSfCL49pOBG/q9kzzB9KyDLfHd2XLuyPHQyi45LoAHw1QglAJ3YJii0aNE+N6JUGApYXjeINn7lhanM31zCD6yVqNTL9orwR7CyUdqZ8KBib4rPhsikEeDsvJpdlBdS6I4+mx9ynUH1J2Yw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=o4iUCqPbiw6ffKabxzDs74hMW2x4rlddWsL/B7ndIFU=; b=EKOr0UsXP/RG3XITL1eaLamUtGl1SqkV/Viuj5J4A5K20YNVDCXynDuUxAoxRq2xy3nsoZlIfiTPIiOEqNodnr/X42NVGA7BWRT3AvZKzdjUkBVXrxORjpfkPzMx9/v7+oo9wpJfoeojo+9CE+cr33cOU8ckOPkDSmDbKuE45Dz0Qv84WI8m/ecNTwJx0URtkCGJD1YbTsSY323tat/LnzOOcWEX4eM8vIocUqdCthhKA2pt49g4X0w6g1zGzDWK2MsNzoB0w9ylyPXXxHr34bErH8Imq3XHvsWPeEE7/eAD/3Ih+GfPZbtmhH3XiqVyHrhAZNP3hLBCF65uCKtbdQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 69.169.224.5) smtp.rcpttodomain=virtuozzo.com smtp.mailfrom=eu-central-1.amazonses.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=sashiko.virtuozzo.com; dkim=pass (signature was verified) header.d=sashiko.virtuozzo.com; dkim=pass (signature was verified) header.d=amazonses.com; arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.onmicrosoft.com; s=selector2-virtuozzo-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=o4iUCqPbiw6ffKabxzDs74hMW2x4rlddWsL/B7ndIFU=; b=cFgIAmHCqxMuB1QSsCUzFmqHGA9HrxB/+Ra8JZbkI6GgA+FbRwlyntBR2ePu7FDvIlQdnNulNBb8WVZTNJN5H1VO9DrsSFgxg/e78M8PkfXnVOP53AQdbJKWhHzRnINvLhiiYg0QL+hcP/yCAHDjgZFcJMaLamkJQiPSyeKJAN8= Received: from AS4P189CA0011.EURP189.PROD.OUTLOOK.COM (2603:10a6:20b:5d7::12) by PA4PR08MB7620.eurprd08.prod.outlook.com (2603:10a6:102:261::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.592.10; Mon, 9 Nov 2026 17:01:45 +0000 Received: from AMS1EPF00000090.eurprd05.prod.outlook.com (2603:10a6:20b:5d7:cafe::75) by AS4P189CA0011.outlook.office365.com (2603:10a6:20b:5d7::12) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.592.13 via Frontend Transport; Mon, 9 Nov 2026 17:01:45 +0000 Authentication-Results: mx.microsoft.com 1; spf=pass (sender IP is 69.169.224.5) smtp.mailfrom=eu-central-1.amazonses.com; dkim=pass (signature was verified) header.d=sashiko.virtuozzo.com;dkim=pass (signature was verified) header.d=amazonses.com;dmarc=pass action=none header.from=sashiko.virtuozzo.com; Received-SPF: Pass (protection.outlook.com: domain of eu-central-1.amazonses.com designates 69.169.224.5 as permitted sender) receiver=protection.outlook.com; client-ip=69.169.224.5; helo=b224-5.smtp-out.eu-central-1.amazonses.com; pr=C Received: from b224-5.smtp-out.eu-central-1.amazonses.com (69.169.224.5) by AMS1EPF00000090.mail.protection.outlook.com (10.167.242.87) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.615.4 via Frontend Transport; Mon, 9 Nov 2026 17:01:44 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=23yjbh4k34hwzfubhuiim7m4inbd3ick; d=sashiko.virtuozzo.com; t=1794243704; h=Message-ID:From:Subject:To:Cc:In-Reply-To:References:Content-Type:Content-Transfer-Encoding:Date; bh=0N2G6iQpOGYjZ+rVWE6ymsypEl8v3wR9Swvbfafhp6Q=; b=kRz7i4dUCO9lfR8jL48XBw054smgIWob4o/rT0DP4G9sIg0s2C92/92I622BIj5U ywj7bCvugHQhpgxGjv8fELUzujgnSJKj+WbvPMdPHDgfc2ieXrXE/CiIXJUdqkan+Xq WmLHa/TwkrBOnRhNjiDaOFdv5B+isOfQc/V1p0UFrDLBJrdFDfPr+amomBuwXDxMgBx rKyDaITqse3jq9XC2jOxDLSWYg3ol4yQwtmehfmMVd3aGume3JSANvGErQhU1QYFFwp +jiyE2sOGZvyVPlQmI4ePbqm7dN6WLBw7pXae/SV15T0Y7jDKmMowEgck0CoVqoMF++ FqugZXJvZw== DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=qftdzk2dqsatjnlrq4r5brjbihpfcrsh; d=amazonses.com; t=1794243704; h=Message-ID:From:Subject:To:Cc:In-Reply-To:References:Content-Type:Content-Transfer-Encoding:Date:Feedback-ID; bh=0N2G6iQpOGYjZ+rVWE6ymsypEl8v3wR9Swvbfafhp6Q=; b=nNKwQiuFwjaLgAGdk3LdQMJiwusbeTRRLh1hPkoqZQdrVt8m1oiXCt4o4pVvwiFA rr2XAd4qqPwXMndkGR/0B4pIxMXgMFj4vSzmbHipUaakS75lUkESSwqi4p9FVu9/Gu/ gXdnr/ive1TgyUBsN0eZ+lJrxgVBSEfoHisVKIWA= Message-ID: <010701a1c14236c2-7670ee71-2293-4025-a716-8c77ab71661e-000000@eu-central-1.amazonses.com> From: no-reply@sashiko.virtuozzo.com Subject: Re: [PATCH hci-dev v4 1/2]vfio/migration: stop a device left running after a guest shutdown #VSTOR-153976 To: Andrey Drobyshev , svt-core@virtuozzo.com Cc: andrey.drobyshev@virtuozzo.com, den@openvz.org In-Reply-To: <20261109165435.175715-2-andrey.drobyshev@virtuozzo.com> References: <20261109165435.175715-1-andrey.drobyshev@virtuozzo.com> <20261109165435.175715-2-andrey.drobyshev@virtuozzo.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Date: Mon, 9 Nov 2026 17:01:44 +0000 Feedback-ID: ::1.eu-central-1.2wRQJBpg9g1fGRYexx3dUuYGS25Bux+2TEOx72/m6U4=:AmazonSES X-SES-Outgoing: 2026.11.09-69.169.224.5 X-EOPAttributedMessage: 0 X-EOPTenantAttributedMessage: 0bc7f26d-0264-416e-a6fc-8352af79c58f:0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: AMS1EPF00000090:EE_|PA4PR08MB7620:EE_ MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 559c9024-ff76-4954-218c-08df3e832284 X-LD-Processed: 0bc7f26d-0264-416e-a6fc-8352af79c58f,ExtAddr X-MS-Exchange-AtpMessageProperties: SA List-Id: svt-core@virtuozzo.com X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|704161411799003|376014|48200799018|61400799027|32142699015|9140799003|4143699003|56012099006|5063699009|261026121527099003|261022105027099003|261021121427099003|261021123027099003|261026140227099003|261009223027099003|261028144727099003|261102123227099003|10067099003|55112099003|22082099003|16102099003|18002099003|5113699003|4113699003|13003099007; X-Microsoft-Antispam-Message-Info: 1An/+BYjeFVI4kcWRQurq7wkR7zm67+xwdzYJJ795UMk85Gu+XNyHi6vBVLJu63xVt/tffuqA9l0KsxgpCNv8uoNRhx7cma/Pz4/DXQpAcn0QarquBqiIYZI6eIp7lAYEsZwBIE1DFibqDjxzSUi4Q7iM69Z/cpTCGkjogmx+5UaJn4wUagcWyDEsp1PfkHwVeGa9W98yp6jbQJV236k2/JaDFuM+iFyQQIVlGQxdwqHnY2IOfzshePJdN0jbMTk3bimW8BLXRYXkRtVmbRJm6Z2Kc2bbL1oQA+Nk8CLppdYUhWxj8AJYsQUzZK9VR7EPHA3Pf6vniHYkZkZ8K2WYiy8smn/6eRVJrdB0kkHdfLjJhH7WkTZgb7H0s6sI/nkwPbPq/MA72P8kx1qGQ+GOkIDYFH+bj93jmyYrNsKG7dviQ5G2qwTRcOrZt6fQhvEPJhMT1+7AnpEkFWdz4n0iq1Y/j24RZK3Nvt4G2mqPlt0E7k6lvV/IUiTR2JmoD6C4UohsaqA3lJHMisisWGWELpblxTKNO647Rb3c3UQSCMROIVOCoeYOwq1NzkWd9X4xr+qZVcPQyH4UYXu8cTe1/hCnsX6XQieFtDYY3zKDKIKgVvxboSbAQL6CTrfSUyL3g0Ib0ODNiBa7TKyNWIV7j0mwOJmyvNwS82xVRNblA087Z/3DxQbVBwiqzjEkxxjMB3k8WcZ1ASnarfmZuYPl9cZg07n71D9mukwQEvJcBygya5QE6t4jW8hPlDfxlazaj0XuMPS/3NJsOrkkiaNz3yGnUoB93xOK5dqXMZ/6se85URipBx1bmFqH/qsVrISAif6z85jQOuiJYg+dMy59FLdJjxHdxZ2M90Ft8aDQLw= X-Forefront-Antispam-Report: CIP:69.169.224.5;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:b224-5.smtp-out.eu-central-1.amazonses.com;PTR:b224-5.smtp-out.eu-central-1.amazonses.com;CAT:NONE;SFS:(13230040)(704161411799003)(376014)(48200799018)(61400799027)(32142699015)(9140799003)(4143699003)(56012099006)(5063699009)(261026121527099003)(261022105027099003)(261021121427099003)(261021123027099003)(261026140227099003)(261009223027099003)(261028144727099003)(261102123227099003)(10067099003)(55112099003)(22082099003)(16102099003)(18002099003)(5113699003)(4113699003)(13003099007);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: kBz7gq7bDEDWf0ctzM7d+ouq3fTh7a1j7oCrpd5to1jus0v+cdSyOUN3OAIgF5R8Gq5rMhrMwEUMT+YpIO2x64r1LsE/9us4IwRX6VCiycGLiHF9exzUMlHGKuJlQpDyeNuZC6ZfjBtaP+4cwOwKDhPhR7x/OAdQezruejk2grWl5gAInvyqkp2Vc81vL1APpx1EUD6FC2d8Prj3Gv9w93urTt1KAwcy6vvbpsj/UIE/U62l0sHxs0wcsQR86JC7mqev+5OS9Jyg+Hf4Hpph/oSxJaLfy3U0maAbaOZSLo7D4y0iglOv8TC5vl/5W6PBDwEwu3ycvskb2NDmXygVWnv8wIpAwo8RcJycBeNxloj9t4iEI3z/LJjY/eG3f+2FHioll/B6o9O9fP9Y1UPR3hTsG2Evsum/Qdf9Skhmjd0nZyJp8dpQBPiyaQ9jOnc/ X-ExternalRecipientOutboundConnectors: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-Auto-Response-Suppress: DR, OOF, AutoReply X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Nov 2026 17:01:44.9857 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 559c9024-ff76-4954-218c-08df3e832284 X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-AuthSource: AMS1EPF00000090.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: Internet X-MS-Exchange-Transport-CrossTenantHeadersStamped: PA4PR08MB7620 Thank you for your contribution! Sashiko AI review found 1 potential issue(= s) to consider: - [Low] vfio/migration: hot-unplug in RUN_STATE_SHUTDOWN still closes a run= ning device -- commit 1b9a9aea32b0d6687f5ee4b3260bb1b513b99f8d Author: Andrey Drobyshev vfio/migration: stop a device left running after a guest shutdown This patch makes VFIO stop a device that is still RUNNING when the VM is in RUN_STATE_SHUTDOWN. It does this from a new shutdown notifier when QEMU quits and at the start of vfio_save_setup(). The goal is to avoid closing or saving a device in the running state, which breaks NVIDIA vGPU VFs. > diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c > --- a/hw/vfio/migration.c > +++ b/hw/vfio/migration.c [ ... ] > @@ -1150,6 +1196,7 @@ static void vfio_migration_deinit(VFIODevice *vbase= dev) > { > VFIOMigration *migration =3D vbasedev->migration; > =20 > + notifier_remove(&migration->shutdown); > migration_remove_notifier(&migration->migration_state); > qemu_del_vm_change_state_handler(migration->vm_state); > unregister_savevm(VMSTATE_IF(vbasedev->dev), "vfio", vbasedev); [Severity: Low] Can a device still be closed in the running state through the hot-unplug path? The stop logic is only reached from the shutdown notifier and from vfio_save_setup(). The unplug path does not call it: vfio_exitfn() vfio_migration_exit() vfio_migration_deinit() notifier_remove(&migration->shutdown); migration_remove_notifier(&migration->migration_state); qemu_del_vm_change_state_handler(migration->vm_state); unregister_savevm(...); vfio_migration_stop_after_shutdown() is never called here. With the VM in RUN_STATE_SHUTDOWN after a -no-shutdown guest shutdown, the device is left RUNNING (by design, per the next patch). A QMP device_del can then unplug the device without guest cooperation, for example when a PCIe slot is already powered off: pcie_cap_slot_unplug_request_cb() /* slot is powered off -> unplug without round-trip to the guest */ pcie_cap_slot_do_unplug(hotplug_pdev); In that case the device would be closed while RUNNING, which is the failure the commit message describes for NVIDIA vGPU. I did not trace this for every machine type. The commit message says "From there QEMU can only reset the VM, quit, or migrate or save it." Is that accurate, given that device_del is also possible on a stopped VM? Should vfio_migration_deinit() also stop a device left running in RUN_STATE_SHUTDOWN before the notifiers are removed? --=20 Sashiko AI review =C2=B7 https://sashiko.vzint.dev/#/patchset/2026110916543= 5.175715-1-andrey.drobyshev@virtuozzo.com?part=3D1