From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from OSPPR02CU001.outbound.protection.outlook.com (mail-norwayeastazon11023118.outbound.protection.outlook.com [40.107.159.118]) by lore.virtuozzo.com (Postfix) with ESMTPS id 97E98814F6 for ; Mon, 9 Nov 2026 21:07:30 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=E0c0zL5uY4b5beUvl8qYGi6RK2/oytE9YqWh3W8ntkAq8JhazYah+/hReBrg74dOUQADF6MnAudZ86gWyrpfdQLk5U+sxs0/dwpSkWeJxCf2SX9DXfAvgHva3rMDRQV7SWoxYk/ZEwqcMcOXZ09bD2td6QOAxpXz36eXmJPy5raCH0B3cLYi1kdn83jP/dFO+U/iryH9r63CS+aTy30XidAeIdyIuNXn6GQYXey3Gb3NIuyTWeo9Anunj6vWbhiJ9aqqJWq1/JO8pGxLmVzHhvc8AU4TDj1ZerNPmo+cTkpg3SIqCQ0wOL5045a2VoyY8ofjJZdVrbudgAKwfE2BUQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=mujTuXvf3V0C/8OiyDmWajrQc8qdd02/yjw8oExCsA0=; b=V2KvjMutGrDxj8uoi6fzpf6Y3H7HR5NPVL1flE3bjTUD2ZgcIEZkuu0ojb9Fhri2l0ZMjZ4vm/+ovQBMB5y0LOXOvaPdzbGuvyPOEW6p4MW6sXsTwGCuX5bskxM6gyWYlKeVL+KWE9cfFON7Bcpyycdczb5y3hzA4Er3zJiKLXMF7vIf0d9vNelmzTI9RJVq8V1iMBFlcYMRCQFImBt1eieOTWT6lomehaDIpUAQKDEDK7YdfdMZEiQObz+39dE1oel/L2Lw3mtZwRoSKt7pJdUtXTkWxM40cBSfIgfaBjT+kG7qd5/tsTQWHRX26JMDfZ0ZXSEoo2ozKpTvClL0+w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mujTuXvf3V0C/8OiyDmWajrQc8qdd02/yjw8oExCsA0=; b=fm8WZPyBmJIR4HHAbLm/i3eVdMKz3LIMuRNqtQ81xWfj6H8VFiPmRClNkJOp3aXmt1CJQfaJ/0DBtqtloicTclM/pQE+OHRkOi0URkUf5QhBtmz4XqEvMBFYFM12+xokKFVd6GSFvKnL+IM37hFeaHHo0yp8rY3iZOpdttfgY1/QP0cEnjd3XY591+NpZzZxLGDQPlKZvWuev65l8J9uJZMcN8x9ZiRG9CHinarApbc1drzngS5OWF6AKy+MqhYYc4q7dH6Bu5yIWPlmoQm2yHUYZT9LnhW3wY3gBtFyenwPGYakuYiIYlquFz3e8SYu2Us4u/mwoxLQWCPccPsNeg== Authentication-Results: mx.microsoft.com 1; dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=virtuozzo.com; Received: from VI0PR08MB10656.eurprd08.prod.outlook.com (2603:10a6:800:20a::12) by DU0PR08MB9608.eurprd08.prod.outlook.com (2603:10a6:10:448::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.592.13; Mon, 9 Nov 2026 21:07:28 +0000 Received: from VI0PR08MB10656.eurprd08.prod.outlook.com ([fe80::4e37:b189:ddcd:3dd8]) by VI0PR08MB10656.eurprd08.prod.outlook.com ([fe80::4e37:b189:ddcd:3dd8%4]) with mapi id 15.21.0592.012; Mon, 9 Nov 2026 21:07:28 +0000 From: Andrey Drobyshev To: svt-core@virtuozzo.com Cc: den@openvz.org Subject: Re: [PATCH hci-dev v6 2/2] vfio/migration: don't stop the device on guest shutdown #VSTOR-153976 Date: Mon, 9 Nov 2026 23:07:21 +0200 Message-ID: <20261109210721.222909-2-andrey.drobyshev@virtuozzo.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <010701a1c219e9a1-c180c4e9-e74d-4b09-95bc-8516357fabae-000000@eu-central-1.amazonses.com> References: <20261109204518.220617-1-andrey.drobyshev@virtuozzo.com> <20261109204518.220617-3-andrey.drobyshev@virtuozzo.com> <010701a1c219e9a1-c180c4e9-e74d-4b09-95bc-8516357fabae-000000@eu-central-1.amazonses.com> Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: VIVP296CA0101.AUTP296.PROD.OUTLOOK.COM (2603:10a6:800:355::18) To VI0PR08MB10656.eurprd08.prod.outlook.com (2603:10a6:800:20a::12) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: VI0PR08MB10656:EE_|DU0PR08MB9608:EE_ X-MS-Office365-Filtering-Correlation-Id: 4bb57225-2e22-4952-af0b-08df3ea57637 X-LD-Processed: 0bc7f26d-0264-416e-a6fc-8352af79c58f,ExtAddr List-Id: svt-core@virtuozzo.com X-MS-Exchange-Transport-Rules-Loop: 1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014|52116014|23010399003|10067099003|38350700014|5023799004|4143699003|56012099006|261021123027099003|261021121427099003|261026121527099003|261009223027099003|261102123227099003|18002099003|22082099003|5113699003; X-Microsoft-Antispam-Message-Info: ckREIfEuwc2cpbxZVauqSGMAuDKI/Zx2RE3dzo0+1oglr9BRTkRWMsmANQJW/rytGNJVUSw5NiiW72057WqvxmE256/FSuCpOjEAQzeRnZpwOcwLpuJ4uNpxii8Z0xq86QvoRPyiMHWyMzzntkMGGAj7FPrYpGGuqHHTBcNBM+1DlraowSd+TTwku+O4KLz2w9FRXXLVAo1AYS/+qr09HET4wT1lGc57VRFmFNbECvngZiLQzF++BxTTs/cfoj/xK5DBg+H4SeVBInDAPeSo/WB23UbHcfh4g8CdNQaqRD9cSsNsyCA1KNslItTKHWuqKbrBmM9eepLjg+OsdO9I8m0NY1fE3z6Qu10ZurVo/G963yYubW1E04y5yVu5hzY+kzR9wg+jnZCSYyKXXWvOzu4c6kMhCnpgqhXSH/iiv1815Yj+QJybR1uGaPsWsa5Cr7XVyKlumel+MDEuQChSI8QYCOVgtFkFwCPIHWeREMR+yTGSXjKeV2MnSsIAS7THfBXEQA4TBJU5wKgfnCeQhJgQSyNVMbX/oY2b6BMadLXlAoeFm/B3+U6wdm2ZlvJu0GvjsuETPscsGRmqT0pjcjdGjJQqeknrq6GxVGLj62ZFktVIeG/eIsAhgofiYFXeuGPVpK8t9ES4JUxNs53Z6daaHMqpjJvp6UfVxgG02gDj3cDR6ft2CnOBZFXUjVE+BCJTMxPsUDDRm1qsePw5AK8PvOSsbw5d0l3wqDG3nbo= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI0PR08MB10656.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014)(52116014)(23010399003)(10067099003)(38350700014)(5023799004)(4143699003)(56012099006)(261021123027099003)(261021121427099003)(261026121527099003)(261009223027099003)(261102123227099003)(18002099003)(22082099003)(5113699003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Y1Cxk/L4oxQahloA5E5FsCdTZMFfw1OyCtmfHIbfwV+yElUzy3SPpSqDzDy+?= =?us-ascii?Q?aGNZ64jD7QZeU/s1DablmJHghceLsaf2DLCzkBv0a/dVlb2De9L7bRRN1y6J?= =?us-ascii?Q?qTtFolA02X5PnM55p2JyT3G9UNT6XOdWVzu7CaiMwU+WCY9aFGPcNZ+6y7Yb?= =?us-ascii?Q?xrRb86dEJgrTapWg33822rlAYD2nZAI7ySzP45yrxmV8J3MX24BRfB0/8moe?= =?us-ascii?Q?XMxjCD6wV2JzFj2XdlTrgUlJvZKSiXGa/zf1LkRDtIX2OtGBTGwwrm8SyYH1?= =?us-ascii?Q?75HUIhr841MJAAqHk+83YJXZ/BwOQ8CNak5LY89Bp6zl58EXHtJgRuoSDX0g?= =?us-ascii?Q?1cr1LAcNuecB9lKVE7UxdfE2FaSWCUXL4QZC6LZQsLFnhj+FgPtT3ljgPCcu?= =?us-ascii?Q?nJW7qJaUDJmuYbzxGbF4f62Q/tNj57N5g5G/DbnCXEjGT886vSNrydxWNuRZ?= =?us-ascii?Q?9Kg3xeFyEBaYP9G02DCvNa3CRFLiGClJTq4hPvO1K59aK2dg6krvv/HZcsMM?= =?us-ascii?Q?rBOcJrMXn+ImqC4WFXaoArb8Fl+oOz8iysOExwvgjbG8DQztTpLbmbnAu/5j?= =?us-ascii?Q?8cP3mPNVhOklb5u3H1TbSWzzpBaj/OT1o4TN4o3QleVfImqTpzf8ACC8rkUR?= =?us-ascii?Q?5UELPWOu7lIUExAB1DTooWFWgON9KZPOnoLoaAz2lxpLGNsoU+hwj/7VFgk6?= =?us-ascii?Q?PJ9lcewq7OrtntRNwr4RCUcqYTs9VKLWxxeBg93V8M3PgkxIskKLn0LeHJYN?= =?us-ascii?Q?d/srMF1aKkOiUmu8OKzYDepbc2jd74AoMqrP0Iv3cMBOm0V+LLdVMMJCNDui?= =?us-ascii?Q?OV7K3kCTVByexkPKssnjGLd6WG1KUyQ4LlqAB5uzC4z1jzDFGVu+VdruAWrJ?= =?us-ascii?Q?/cWYlz9afD5vN1giA6vKhkgSiULy7UOrcoTpvb6MkJEtQqk+0oNH9HaKLOus?= =?us-ascii?Q?jYAdWS0mm8+8qkRBFc6dlP4cdkF3WqsXoco+bjWziNrwBq4t9ydEbRRQ3nW8?= =?us-ascii?Q?m2/O4srSLzp37xetoS0HVcdbFfNdU2JspEne7IHJhLnpQ8r3Rn3TOdYTTfBb?= =?us-ascii?Q?igWsszHgEZWhPEzjOSOH1jX61owzzHstPrSQ2mMU0ofQ5ZAsAR/8Q+T40431?= =?us-ascii?Q?b7joJ5wFz7Hnpi6qzIsFXLYFTNxeUH6FHcL8WWZhgUbUJ3IF/fIskLChWhkR?= =?us-ascii?Q?l5qDFB/wF44argtWY6Tl3D51Va2iR6iVdMvikzYItqiRUqZ7JAJzf+2QxUme?= =?us-ascii?Q?cH/SNbp+fjGQ2iLxKvFAUslBU7HNCrHxDytRtmgYU7XQ9TSgSXiTM4+YpZ0l?= =?us-ascii?Q?PPXET5L/zlJxwoxKuR8en2MoPVSMzYj2qcUXVDnK02glUCYLpdFBZhGzwbha?= =?us-ascii?Q?FqXvN3/E8zaY2qaXELL2q0m09XiFh+y2xKxY5nVg1B6RQtMJGLhB5x028cjr?= =?us-ascii?Q?73oT4Npwbbo+2fJWgp6gganggPEBQwOZDYcmJ3WhpAbdgljKS3s8hy3xVIvm?= =?us-ascii?Q?Jw5u1W1UAmS52fjtMg1EiQo3k9uPtZvPUIGcTGa764+6nyEdYTVvvUjfAGwH?= =?us-ascii?Q?OnvjtuUIw7YCcURwkBxUEnMVRszBoZz/aAjnufG0KTy0aigI4oUZtGS9JSjw?= =?us-ascii?Q?zNuFTB/fjDO8wgOlQeutfUQaHBbsXvhEmDWcoSwNjJldi8ICCptJQJA6Yp7Q?= =?us-ascii?Q?LlKQ9yMQ+fnmNgdlqfgsn/9iT5beSaKVl0JuTJhnGOHasi+glSccNQfRTMuf?= =?us-ascii?Q?qWtGCZS9hv5n+7vPSByIIBGq2ElLqg8=3D?= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4bb57225-2e22-4952-af0b-08df3ea57637 X-MS-Exchange-CrossTenant-AuthSource: VI0PR08MB10656.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Nov 2026 21:07:28.4790 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: O7INuaascJG05mpkUDveXCP40iwHu2ln+ydBhRZu2TGVaaCUMnB2rGt4tay9mjtuVXsqA6d1qNfcR/VbcdV6qPv32CCyE5Ep+SPEnJe265o= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DU0PR08MB9608 On 11/9/26 22:57, no-reply@sashiko.virtuozzo.com wrote: > [Severity: Low] > The predicate tests only !running, state == RUN_STATE_SHUTDOWN and > qemu_shutdown_action_is_pause(). It does not look at what caused the stop. > [...] > Under -no-shutdown, these host-originated stops would also skip the device > STOP, even though the guest driver is not necessarily gone. Yes, and that is fine. Every stop with the "pause" action ends the same way: the VM can only be reset, quit or migrated, and each of those resets or stops the device. Until then the vCPUs are stopped and nothing new is asked of the device. Whether the guest driver is still loaded makes no difference to that, so the cause is not needed. > [Severity: Medium] > This early return leaves the device in RUNNING or RUNNING_P2P while the VM > sits in RUN_STATE_SHUTDOWN. The commit message and the new comment name only > reset, quit and migration as the follow-up paths that stop the device. > > What about device removal? > [...] > Before this patch the device was always in STOP by the time it was > unrealized. Can the device now be closed while still in RUNNING or > RUNNING_P2P if it is unplugged after a guest shutdown, with no reset, quit or > save in between? A hot-unplug needs the guest to eject the device, through the attention button on PCIe or the ACPI notification on legacy PCI. The guest is halted, so the request stays pending until the reset and cont, and then the device is handled as on any running VM. The one shortcut that skips the guest, a slot the guest had already powered off, means the guest had already released that device. And that shortcut closes a running device on any running VM today, that is how every VFIO hot-unplug works. So nothing new is reachable here. > [Severity: Low] > This skip covers only the final STOP. vfio_vmstate_change_prepare() is not > gated by the new check: > [...] > RUNNING_P2P is a migration-flow state. Could a vendor manager that treats > migration-flow transitions as the start of a migration still be confused by > it? > > The commit message says the P2P round is deliberate, but it does not say > whether the NVIDIA vGPU manager tolerates RUNNING_P2P. It also does not say > whether the testing was done on a device with P2P support. It does. The NVIDIA vGPU VF driver takes RUNNING to RUNNING_P2P and back without telling its manager, that transition is a no-op in its state table. All testing was done on that device, which supports P2P, so the P2P round ran at every guest shutdown: 16 soft reboots followed by a live migration, and four runs of the vGPU test suite with 29 migrations each, all passed. I will say so in the commit message.