From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from AM0PR02CU008.outbound.protection.outlook.com (mail-westeuropeazon11023138.outbound.protection.outlook.com [52.101.72.138]) by lore.virtuozzo.com (Postfix) with ESMTPS id 9F9AE8017D for ; Mon, 9 Nov 2026 16:54:51 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=BxqEQZxASNxfg97lWyh8oqg8ALg9ClJCLaFAyoPUL5JKlWlJ7fEbwF/B1ZpDHZmWIv23FTuz0l531CXh8jyCZ5P8nT94V8azkHk7lFTbt8WmYuIqeNQr/jvvpupiPCAwcvFYt6CfBFzSkwCT8xn+H62/u3Q3qQo5vimniJzjKCrnKeh2yLAxyI0abutPG91/cTJimbFVWwlbdUaCcqd1CB2he6Ut91AS4P49xpyo1HGL4vk+S/zwswhLqLFYCPsgFSuaCrGGL5CqPz702KDPL1f7zFkkS5F/8ojOWpzhrrcLG/NO797IBJ6BZtIZgl3zrzuymHZ6pz4W0xfezvZuBQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=DMiVoj7D8On6EIB1HP3SlQ00pwQafWd7kladmzmQApc=; b=JekBTx5uQcU2R9tMcAsq0g/6hENjydjdlduqIzdkyrt0ahNyLUx5q0Xdo/JZJPNk1Zgy5S0ZgHsZyWLcHQ/sxvmUvumF9LLenhq4yHM8Fu2KHHmGiS7aYxpCn3rb5FFWU7ra8FPCgd1CB+IEqPjiid40y7ZqWn5fCxs+ntD+IF/qmikhp48d0JodCYZ+AP/Dmr9d0VnL2rHh7yIJKV9o6K6KpWpruqi0TBFYtjiQXVHXrho7paZMD7Qjp87awBRasEYW276KbfRuitsa81K8r8RSWDjHjWpcCxRod0NVgzeNJ6lZ/NMoMx15OOUR+be9NmRbsX1nLyj97Uek1HJa7A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DMiVoj7D8On6EIB1HP3SlQ00pwQafWd7kladmzmQApc=; b=zDtSth1xdPqTvedihRxQBW45mkp8AjuXKgK0Go0Ei0RYmQUWMK632WCAbvpNWOeeAc0W/TZwoNvF2eADcTX16q2KG5l+mKG0ecq2lr45DH/Y3QiO9t6qQTAaDaVP3KydPuOnSXwpSijlzfYwD0LFH6T8o0iC1zPDgFnTghTaTSyeGqwYr6WlbJt7+mrIve0cNVslJEiyRdJ62sKJVqFmxdbUdCPtVFzO+LUGlSmrb5SGruw5+vcLLgDztdi+6ZCtvC7DmZyqTZeYhfW/EQjxlzK2LMhpQ2mZRAAUv20aATNYHqVZonE8W+HS3ppJ+7SFrINeYgETwG4ReDNGoMaA8w== Authentication-Results: mx.microsoft.com 1; dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=virtuozzo.com; Received: from VI0PR08MB10656.eurprd08.prod.outlook.com (2603:10a6:800:20a::12) by GVXPR08MB11423.eurprd08.prod.outlook.com (2603:10a6:150:2e2::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.592.12; Mon, 9 Nov 2026 16:54:45 +0000 Received: from VI0PR08MB10656.eurprd08.prod.outlook.com ([fe80::4e37:b189:ddcd:3dd8]) by VI0PR08MB10656.eurprd08.prod.outlook.com ([fe80::4e37:b189:ddcd:3dd8%4]) with mapi id 15.21.0592.010; Mon, 9 Nov 2026 16:54:44 +0000 From: Andrey Drobyshev To: svt-core@virtuozzo.com Cc: andrey.drobyshev@virtuozzo.com, den@openvz.org Subject: [PATCH hci-dev v4 2/2]vfio/migration: don't stop the device on guest shutdown #VSTOR-153976 Date: Mon, 9 Nov 2026 18:54:35 +0200 Message-ID: <20261109165435.175715-3-andrey.drobyshev@virtuozzo.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20261109165435.175715-1-andrey.drobyshev@virtuozzo.com> References: <20261109165435.175715-1-andrey.drobyshev@virtuozzo.com> Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: VIZP296CA0002.AUTP296.PROD.OUTLOOK.COM (2603:10a6:800:2a1::6) To VI0PR08MB10656.eurprd08.prod.outlook.com (2603:10a6:800:20a::12) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: VI0PR08MB10656:EE_|GVXPR08MB11423:EE_ X-MS-Office365-Filtering-Correlation-Id: 137407e5-08b7-4f14-25f1-08df3e8226ce X-LD-Processed: 0bc7f26d-0264-416e-a6fc-8352af79c58f,ExtAddr List-Id: svt-core@virtuozzo.com X-MS-Exchange-Transport-Rules-Loop: 1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|23010399003|52116014|376014|1800799024|366016|38350700014|5023799004|261021121427099003|261026140227099003|56012099006|261022105027099003|261009223027099003|261026121527099003|261028144727099003|261102123227099003|10067099003|22082099003|18002099003|5113699003; X-Microsoft-Antispam-Message-Info: ZPeUyEA4YVzDoNnbxJ4E0v80klld2N6sBsxSE33r222ooYhncEv0fkrqaykkVQw0P84MJre+HKRtYVr6vGLtTjrAtN6KJQVaJsB+dtfY97aAAo7hVs+apw0K++TQlQ83cfQ+bHmtl+mxWVeC11OHimCTLVENnkj6kPra39le87bMV2vtZo3LeG+Rt3CMPuYHDq2QDyBafLa7XiPBGoRbJCKxbaJUJKo27H0RojV7ib+Hmm6WLhGatW1LfY79Pp3OpHgoE+slLHh1eIi2VDW0nVVLW4M2ngPocdSb2OP9OdUZ6C/cPT1WDsUVsn93cEQ3c3ksJaxl6c2toGIXdGdx9zN4fqln9bR1SoyXx3DhjBMZQlF6Xj3J/fctM7Ieg1Po1ihU95o80dFPwm40hiHgQQb03U3T8KTS3RNE07QLPUl8jWseJxmXNKFTYzKxgjS+e8PoR64Gd7odkjmW2wq68bZ8o14S9m062xG7yp7fnNk+SQkYKxW5tArvlGG0aEW9UNYT/sySYm3onpdfnjrEslrkEBaYSypRPjS6z3e+gq/skb5qCfwXEPqBpU/IToCg/o7gOx0k6y+rDXYvPrgMYnF52UZU75jRQ7da6oYnRxGCQJS8ADkwdF56pvSu+QPo+JgtK7CYFXw006F2e67Whb3g/GG1d34PTXgxmSfJMC3e+RqDqhjUjqLz9L7Ck0MOr7l7rsvWnQuaIrhbB/ZecGYgTUX7e3vzFZA+PHhJ7Ww= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI0PR08MB10656.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(23010399003)(52116014)(376014)(1800799024)(366016)(38350700014)(5023799004)(261021121427099003)(261026140227099003)(56012099006)(261022105027099003)(261009223027099003)(261026121527099003)(261028144727099003)(261102123227099003)(10067099003)(22082099003)(18002099003)(5113699003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?lr/5VGI8+8mQLAp0VG2Vp5BJERQlgrUR2rpZjVmldIqossUcaddpJvvaKEpv?= =?us-ascii?Q?Gf5gFhPbuqPguoGgdVmRrdtzENJM35AUP7s/xcxB3A9/iEE4vQNctoLbH9AS?= =?us-ascii?Q?9ggrUTyusAOEvAPb2exOP5KF9xkmWLTMpIw6RNeXEghh9n5WX6wenr20KD+W?= =?us-ascii?Q?D8IObt3tlbbSQuezYjIek+j7LoHGWtuEuPcVfOjF+VDcY+nBjMKziYB1AOuI?= =?us-ascii?Q?a+WiHDurMkMFGhC2PVLRa75DpKzzENui3rKV0p59ciWn5BOT9Ww84mp7v6tE?= =?us-ascii?Q?TR92JH3/pJfrn4JyZ32KsKO+SXN2Lm04/38m+ym0RuYvV+oug5TkZkboVwOU?= =?us-ascii?Q?qySN2wMrPUvoHsD+MA7ghW9pzoJIh8Ettof7hqCYmMLXMBIF0/yBb+ATh2P1?= =?us-ascii?Q?sKEIF29/R3zzjsONrKaWGweKvlTEhqET1f2530QWxlgHWqYbYXcmYaYo6+EW?= =?us-ascii?Q?92gf4SaC7Balookr3fbm5ucab6YBWSXLXwiyXRWUoTY1wZfZapmXZtI3Z4bH?= =?us-ascii?Q?KXX0gKt1P1t5Rdhn0w0+uMAA5eQRduMmyPwRnQaVPBbPrV41eXqDFF9BtYjg?= =?us-ascii?Q?U7IpjohEGJX4sN9f+ZIEswCnehhJOQbHDPGYq8erruobDEqwPV0pAjhCqkY0?= =?us-ascii?Q?yCHhslGNSm47FJDS6kPXwTbOMMx0QVAuo2S8yBgV8Hq2vAVMN0wSNmclkTtP?= =?us-ascii?Q?Z28AbSodrUDAkcve7mGdA1QzbQGGe1gDs89Imm4KSFa/HUDlFNrQqjJYNHfp?= =?us-ascii?Q?hCH/bLqgECdBQQ2+nSOoyQH1dROqZEqEb0i4yQFgArFEMaGBPU201oM38grd?= =?us-ascii?Q?eXAdO2R0EhvBri5GEE0pZ5vskhdlmBKt8/n5giHEF9Us/dQim9YAgyC01Tnj?= =?us-ascii?Q?frQpKRR/AUuq/538MOc1o87eWWGzP7w4upHIE9FtGRvNT9HdUiJZTq4CIgn+?= =?us-ascii?Q?0E69jKGrkKsG5nEeVSlRfcPo05c+VJbXFVBi/NcQvcB8E4F5HNTL5ZoFVgM8?= =?us-ascii?Q?0Vr4Kt2Sc9Ga/+Yw7H1QyQnfA6pJMA/aLbZWFG1ts8YH+YqfZ24H4k013duf?= =?us-ascii?Q?XQXfP9tm5+hWwfQxjyLyR0AsrayoDW2raT89xepFVucvSS2gJ2wG4+e9BF0D?= =?us-ascii?Q?GrAWU8I5l6uhKk2o+jxKHhHvFeJoBVJ7NJQwO8Fg8kt3hR0TUXGEqh+1lfoc?= =?us-ascii?Q?HPCHGBYFZ9cofBswcqbWDEQvqiK2CGMosYKNPhipu5etIwu/BdBVfjN/CNOU?= =?us-ascii?Q?XReyOdIyk5M+R2dF1idCxsPNJ5QmrY4TiqX/AnvVOlAs8LQ3jgNt/PuA97Nw?= =?us-ascii?Q?XF7YrPO2htHG7678Y006wXx/hWxse1J5TtrFNmqRjhRZkow2d8i+Odk8qeKa?= =?us-ascii?Q?I4wayHhvwkD1KY4snAqlo6oU2aBnmemVuTm4Jq8h+r3IhworEosvMnkUZn8a?= =?us-ascii?Q?pITD2g+dw4JxhlTvTxcQih1m14zly5XQbpI70Yi8S6HyumGPmdM3n2U7oLZk?= =?us-ascii?Q?v2pgcecLnRrbJxgSUYJVBVT9dn0kTgXGjUt0COiZUY4FFjjDLEQTPws3Bp+W?= =?us-ascii?Q?zpcmDnzY5CNX6ORMssTaqpfS0vMfKuasAlVt/2WN65AfMMb+W/lYnaEba65I?= =?us-ascii?Q?Cz0yTdQv8h9IXsk5Q+k2ZsF3dM4Z5e6ey1UwU4XGfKDudDhFkfhJng+BtHAF?= =?us-ascii?Q?hFJB0AwhGSPWb93fjOYzQp4n+gEVDKnsXavWrQVivNFyFbGk3ODijd8K//xQ?= =?us-ascii?Q?PVuq8jYzysQVnKg6QyxxCxxVjegtbA0=3D?= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 137407e5-08b7-4f14-25f1-08df3e8226ce X-MS-Exchange-CrossTenant-AuthSource: VI0PR08MB10656.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Nov 2026 16:54:42.8857 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: qOmrbYD/heILwVsZj5pvR8cFt+QMJZZKjwc1Q5Ps1OJK9xjN1FnAY5cYWjiMFFAxEmS+XybwEflm12glnuR9xRpn6BOb3Q8Jhzm9exrkp9Q= X-MS-Exchange-Transport-CrossTenantHeadersStamped: GVXPR08MB11423 When the guest shuts down and QEMU runs with -no-shutdown, the VM stays stopped and can only be reset or quit. A libvirt soft reboot is exactly that: guest shutdown, system_reset, cont. Today vfio_vmstate_change() treats this stop like any other and moves the device to STOP, then cont brings it back to RUNNING. NVIDIA's vGPU manager (595.71.03) takes that STOP for the start of a migration, finds no guest driver, breaks its dirty page tracking and never repairs it when the driver loads again. Every live migration of the VM after a soft reboot then fails: "Error saving page in pipelined mode". Skip the STOP in this case. The VM stop still runs its P2P round, so the device only stops its peer to peer DMA. The vCPUs are stopped and nothing more is asked of the device until the reset, quit or migration that must follow, and each of those resets or stops it: the reset by itself, the two others with the previous patch. A device in the middle of a migration is stopped as before, the migration needs it stopped. The case is told apart by the shutdown action: only "pause" (-no-shutdown) keeps the VM alive. A quit of a running VM, a signal and a plain guest shutdown use "poweroff" and still stop the device as before. With this patch applied, soft reboot + subsequent live migration pass successfully. Signed-off-by: Andrey Drobyshev --- hw/vfio/migration.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c index 064da62ddc2..34aaa35a2cf 100644 --- a/hw/vfio/migration.c +++ b/hw/vfio/migration.c @@ -415,6 +415,22 @@ static bool vfio_precopy_supported(VFIODevice *vbasedev) return migration->mig_flags & VFIO_MIGRATION_PRE_COPY; } +/* + * The guest has shut down and QEMU keeps the VM stopped (-no-shutdown). Only + * a reset, a quit or a migration can follow. The device is not stopped + * here, it only stops its P2P DMA like on any VM stop: the reset resets it, + * the two other paths stop it with vfio_migration_stop_after_shutdown(). + * A device in the middle of a migration is stopped as before. + * + * The same run state with the "poweroff" action is a quit of a running VM, + * which stops the device as usual. + */ +static bool vfio_vm_stopped_by_guest_shutdown(bool running, RunState state) +{ + return !running && state == RUN_STATE_SHUTDOWN && + qemu_shutdown_action_is_pause(); +} + /* * Stop a device still running while the VM is stopped after a guest * shutdown, before it is closed or saved. Such a device comes from the @@ -964,6 +980,11 @@ static void vfio_vmstate_change(void *opaque, bool running, RunState state) return; } + if (vfio_vm_stopped_by_guest_shutdown(running, state) && + !vfio_device_state_is_precopy(vbasedev)) { + return; + } + if (running) { new_state = VFIO_DEVICE_STATE_RUNNING; } else { -- 2.47.1