From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <010701a19bf84920-b5393f2e-53dd-435f-b32a-680d3ff0886b-000000@eu-central-1.amazonses.com> Received: from AM0PR83CU005.outbound.protection.outlook.com (mail-westeuropeazon11020079.outbound.protection.outlook.com [52.101.69.79]) by lore.virtuozzo.com (Postfix) with ESMTPS id E22568017D for ; Mon, 2 Nov 2026 11:15:06 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Qc0vcqP4L0Af2Ig+02HhGu9kRBh670YUbe5wDtCRpujKKGLMZS1ZuBzbk49G/szdh5UQLGLSW5QN83XYP45rRstg/YhU4X3wM2RXG+XsqMKbf6iX5Xs/aAO9zxOkKCuWImBPCo5Ts7/X3ugoY7Q7uL7G+pHx3mqr3BzIAtiKmSDQ1TTGr3JobD1SLYfF6aXP6KPyOT6XmzaqY2Lwu8C9QlsLRnzdjxCSFNxoQHDy6zGG+mGU7KPnxx84Q8/9Jc+YWaDgYbfL9mp+SygDT0ivX+mrYnYGdWnz6EdkUu28EJ7XcCYvFXx9aUtT4eOR9DMTzlIFvXH/74s7ykL/7ZHuXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0l5V1XEIq6gQdR5eRY6ikpGcjjQ8zuogvedYwazierM=; b=dWhHzgVe1LVh2TrY767YEZ2Siia2QUhro20oE+kdRqNVWDX78YLqwYhHIubXOftgj22nFoj1MeMnu8z6EW/eDKCk50RDlbTxWhL7oDgUjeme0ahGfR9XtSGSWM8wFyGRJr2j1vdRmZdIt7o/Vx4Hf/CBIpKRNdBP1Bz5LR5+24PUuxsvX8RY45/U9y/lkRhuHlBroddHrbiwgIRwHVyFbACdIOHk0aB9RjA36QnDVWWVAdvSeU0kudCUFM5/BjESL/S0UOmXXKyxsnRd/2/j66aGOUmyRJRZIhsXmNLy1olXgap1lCIR5fekiEhigkwhlL5sITlY5y+VshPRzjndZw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 69.169.224.6) smtp.rcpttodomain=virtuozzo.com smtp.mailfrom=eu-central-1.amazonses.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=sashiko.virtuozzo.com; dkim=pass (signature was verified) header.d=sashiko.virtuozzo.com; dkim=pass (signature was verified) header.d=amazonses.com; arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.onmicrosoft.com; s=selector2-virtuozzo-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0l5V1XEIq6gQdR5eRY6ikpGcjjQ8zuogvedYwazierM=; b=R2E7j5aZl3wFerk7oiXnbFc7RPKISibQ7TppPTqWAimEBARI7uyZS4Tay1u/OvCx2df5pNNgZNEZwyib2Q3KXxiP9ntQkyltbpiJ7bpMNKqSCiYgO9qgLI2mN3j2vpqetLTyQUiSVjzXUr3TUnvdXwzSs1a3LzxW76+pjUFs+o4= Received: from PR1P264CA0009.FRAP264.PROD.OUTLOOK.COM (2603:10a6:102:19e::14) by AM7PR08MB5430.eurprd08.prod.outlook.com (2603:10a6:20b:106::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.569.12; Mon, 2 Nov 2026 11:15:03 +0000 Received: from MA3PEPF00000408.eurprd04.prod.outlook.com (2603:10a6:102:19e:cafe::9) by PR1P264CA0009.outlook.office365.com (2603:10a6:102:19e::14) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.569.12 via Frontend Transport; Mon, 2 Nov 2026 11:15:03 +0000 Authentication-Results: mx.microsoft.com 1; spf=pass (sender IP is 69.169.224.6) smtp.mailfrom=eu-central-1.amazonses.com; dkim=pass (signature was verified) header.d=sashiko.virtuozzo.com;dkim=pass (signature was verified) header.d=amazonses.com;dmarc=pass action=none header.from=sashiko.virtuozzo.com; Received-SPF: Pass (protection.outlook.com: domain of eu-central-1.amazonses.com designates 69.169.224.6 as permitted sender) receiver=protection.outlook.com; client-ip=69.169.224.6; helo=b224-6.smtp-out.eu-central-1.amazonses.com; pr=C Received: from b224-6.smtp-out.eu-central-1.amazonses.com (69.169.224.6) by MA3PEPF00000408.mail.protection.outlook.com (10.167.241.153) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.592.5 via Frontend Transport; Mon, 2 Nov 2026 11:15:03 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=23yjbh4k34hwzfubhuiim7m4inbd3ick; d=sashiko.virtuozzo.com; t=1793618102; h=Message-ID:From:Subject:To:In-Reply-To:References:Content-Type:Content-Transfer-Encoding:Date; bh=BosWfv4b8Om3qDOi14wGDmKDkZzqIMhv3MRRZTH7jlc=; b=BXsC2yhGO8cqV4Z+quAaGUgIOTnPBofC3EKfufQOmlFGF+c5y7d5NjeoFbjJKz8D J0V7a7Y0u5kD/PgaWo1Ry/eHlCfDZTtFKpB4H7JN4ykwLFUU2E16DfbieZlrPLiJd4b 5kI1el5fLwFv46+lonHdTdVBjeRt8nUlG8a7Wdp9rRouTbzL6mUGA0rBc03EQvRhHKY 2j0glxrRlGi+QR/U8DAB/RBCFTljJWYceN2+fQPft6Q8gKPsSyBtqBCchZgfYbmlyIH Egh7FdcTOIQnTtzttJCucFZ7sdgREuGsTnciAZsvOl9tWB0AqIHnPdxYLLPj96V9sEw x241/l3kcA== DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=qftdzk2dqsatjnlrq4r5brjbihpfcrsh; d=amazonses.com; t=1793618102; h=Message-ID:From:Subject:To:In-Reply-To:References:Content-Type:Content-Transfer-Encoding:Date:Feedback-ID; bh=BosWfv4b8Om3qDOi14wGDmKDkZzqIMhv3MRRZTH7jlc=; b=TO8y9wRWPdPv61BiTkGbZBro3FB9WNyh56OkAJIrEnpfWLgLyCr78BqS4LwXkOo6 V1k/txdDRo4f1bR3c6MvaExHvza2pr8Z/X18EOF06fR/kee0ns+nwZVWpV01BbbwFDa 7WwaZw2L+uxwQXr4W/VRh7xbe9WFouaAkztP1R0M= Message-ID: <010701a19bf84920-b5393f2e-53dd-435f-b32a-680d3ff0886b-000000@eu-central-1.amazonses.com> From: no-reply@sashiko.virtuozzo.com Subject: Re: [PATCH QEMU HCI-8.1 v6 34/34] VZ: vfio/migration: don't wait for stop-copy-only device state #VSTOR-141821 To: svt-core@virtuozzo.com, Andrey Drobyshev In-Reply-To: <20261102-vfio_v1-v6-34-9bc7cabc850d@virtuozzo.com> References: <20261102-vfio_v1-v6-0-9bc7cabc850d@virtuozzo.com> <20261102-vfio_v1-v6-34-9bc7cabc850d@virtuozzo.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Date: Mon, 2 Nov 2026 11:15:02 +0000 Feedback-ID: ::1.eu-central-1.2wRQJBpg9g1fGRYexx3dUuYGS25Bux+2TEOx72/m6U4=:AmazonSES X-SES-Outgoing: 2026.11.02-69.169.224.6 X-EOPAttributedMessage: 0 X-EOPTenantAttributedMessage: 0bc7f26d-0264-416e-a6fc-8352af79c58f:0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MA3PEPF00000408:EE_|AM7PR08MB5430:EE_ MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: d0565108-2ac4-4c30-4132-08df38d28ab9 X-LD-Processed: 0bc7f26d-0264-416e-a6fc-8352af79c58f,ExtAddr X-MS-Exchange-AtpMessageProperties: SA List-Id: svt-core@virtuozzo.com X-MS-Exchange-Transport-Rules-Loop: 1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|704161411799003|32142699015|61400799027|48200799018|376014|9140799003|5023799004|4143699003|56012099006|5063699009|261009223027099003|261028144727099003|261021123027099003|261026121527099003|261021121427099003|261026140227099003|261022105027099003|10067099003|16102099003|55112099003|18002099003|22082099003|4113699003|5113699003|13003099007; X-Microsoft-Antispam-Message-Info: 25Dt1QNO7Ev7WI4O4jrqBrw6YQ4HEqOeE/vsAdfiwW7MLXuXkAughEQAdRMEVpVIm/SsY2Z/MFQOA3Uc+W2DOi0QvZON6/0yGuSVqvnvv9JW244p0RyfXniIuta30jzM/C9WTaiyD1rAm+5ARTDrw5khTdf8HgZvF4/KWGppcZFW3FUAhyQxW+WVIwDby7Emci1cxBaabB6zLy7wOjuE4WIjbB8fnNYPq+eOBBbdWjOzByYcj0hSoWDk162fX6Ww4QveTIBjJANf4EmHr5YI93Cs6AYLbgCiPX55HTZvx1sBh0PE4ph4p3NQ3oTFclRPjTqK7pDX9XDHvoDzmHI0kMWddq07JbVec8r0iyI75MqQxYf6K6TCSscGGvwSCRrLGitpflTxGEFdcYtC0ak4mq7FF1hRrbRkwdvPpdTLi3FgAECuwAohkf4NB8UN6XYBlAzNSjYX5cVvmhZiU/S2ub9/FlKcOQU4Mqaa5fifmetSlTZknW4fuQ0ZIS0+Z1McEC5n38jNMLLIH5oO9oxe6wGJG59aZwk3JuSV6wwhW4clxMAqbYMaNGChg+NBRta3JY/GSP02axaq74qazkJz9Nh5Oh+XnOrM+pTiTZ8b2VG6XHctzhSwNOlhMwWDC0kcGGHHroC+cuHhU6E3plELnE5P7yT9rf/jCPiv035kKFEVAjbXK7Ff5moAGlbFOGj53hq95VyESFf5EE6WJQJ47bZRPWfG5c9/jSFOpIkevxQyuudTuTccKiQJhILmpZb6V5DNrLcMhhCT9+ky3oe6754KvQ3plO8U/F6/D3lJ7JZHNvxvoGzjHLw4jKSyoTfONgZ/ed1BmL7bo+dPG3it7UeTmVOPhgBNYdQut3HCM9k= X-Forefront-Antispam-Report: CIP:69.169.224.6;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:b224-6.smtp-out.eu-central-1.amazonses.com;PTR:b224-6.smtp-out.eu-central-1.amazonses.com;CAT:NONE;SFS:(13230040)(704161411799003)(32142699015)(61400799027)(48200799018)(376014)(9140799003)(5023799004)(4143699003)(56012099006)(5063699009)(261009223027099003)(261028144727099003)(261021123027099003)(261026121527099003)(261021121427099003)(261026140227099003)(261022105027099003)(10067099003)(16102099003)(55112099003)(18002099003)(22082099003)(4113699003)(5113699003)(13003099007);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: PBNjYdnovFR+brk/TM/RYNRgomhqUnnzIhcO+SQkUt0gAWmjONr7N5ZDG2ngq2h0Es4Vl48V2NUoJjPzbNSqPDL/GhIhxUJSbg00ykn2EcZpaoXBQXLRSsai5kTaBOQzk/mk5bU0JegkcEv/jV30R9S5A36XZPG4NG9wSiZ0JUhSkfjroncoXVVXsQ8b6HcNZjIj2OSoq3XhVkICpRZWkjU3mR+rLPDgM8W2uXWzb4QKoj2lxOCc+vrur8o0ObrD68qTcTAgZkUHJV6Rar+V99vhxTzVhLJSZV4onv/IWuMZZ3/v+wyOgFvs7mstpPKE+6IOOU6vaOcYSKQsdTvDgAFqW0P92KOTZCB8KLROYwO5kH/RthuAMvJI5wtnWkiZdbKjUfh78RERPfmKDOBe6eNkQ/gYYZm7g5T1bDElcOP1ZIS8mVodWkF+rB05Wjeo X-ExternalRecipientOutboundConnectors: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Nov 2026 11:15:03.0672 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d0565108-2ac4-4c30-4132-08df38d28ab9 X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-AuthSource: MA3PEPF00000408.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: Internet X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM7PR08MB5430 Thank you for your contribution! Sashiko AI review found 2 potential issue(= s) to consider: - [Medium] vfio: stop-copy size dropped from pending data, bypassing the do= wntime limit - [Medium] vfio/migration: stop-copy size dropped for devices that never en= ter PRE_COPY -- commit e935c30e55885d702d35a8da4365fcb9090511cd Author: Andrey Drobyshev VZ: vfio/migration: don't wait for stop-copy-only device state #VSTOR-141821 This patch changes vfio_state_pending_exact() so that the device's stop-copy size is added to must_precopy only when a query failed or pre-copy data (initial or dirty) is still pending. Once pre-copy has drained, the stop-copy size no longer holds up the switchover decision. > diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c > index 04d0de7cef0a..eff1172569d3 100644 > --- a/hw/vfio/migration.c > +++ b/hw/vfio/migration.c [ ... ] > @@ -569,16 +569,24 @@ static void vfio_state_pending_exact(void *opaque, = uint64_t *must_precopy, > VFIODevice *vbasedev =3D opaque; > VFIOMigration *migration =3D vbasedev->migration; > uint64_t stop_copy_size =3D VFIO_MIG_STOP_COPY_SIZE; > + int ret; > =20 > /* > * If getting pending migration size fails, VFIO_MIG_STOP_COPY_SIZE = is > * reported so downtime limit won't be violated. > */ [Severity: Medium] Is this comment still accurate? On the success path the stop-copy size is now deliberately left out of must_precopy, so the downtime limit is no longer protected for it. Only the error fallback still reports VFIO_MIG_STOP_COPY_SIZE. > - vfio_query_stop_copy_size(vbasedev, &stop_copy_size); > - *must_precopy +=3D stop_copy_size; > + ret =3D vfio_query_stop_copy_size(vbasedev, &stop_copy_size); > =20 > if (vfio_device_state_is_precopy(vbasedev)) { > - vfio_query_precopy_size(migration); > + ret |=3D vfio_query_precopy_size(migration); > + } > + > + /* > + * Once there's nothing left to pre-copy, the stop-copy state can't > + * shrink by iterating: don't let it hold up the switchover. > + */ > + if (ret || migration->precopy_init_size || migration->precopy_dirty_= size) { > + *must_precopy +=3D stop_copy_size; > } [Severity: Medium] The commit message says the stop-copy size is reported "only while the device still has something to pre-copy". Can this condition tell a drained PRE_COPY device apart from a device that never pre-copies? If the device lacks VFIO_MIGRATION_PRE_COPY, vfio_save_setup() never moves it to PRE_COPY. In that case vfio_device_state_is_precopy() is false and vfio_query_precopy_size() is never called: if (vfio_device_state_is_precopy(vbasedev)) { ret |=3D vfio_query_precopy_size(migration); } precopy_init_size and precopy_dirty_size then stay at 0. They are zeroed in vfio_save_cleanup() and are only decremented elsewhere. If vfio_query_stop_copy_size() succeeds, ret is 0, so this is the result: if (ret || migration->precopy_init_size || migration->precopy_dirty_siz= e) The condition is false and stop_copy_size is not added to *must_precopy. vfio_state_pending_estimate() also returns early when !vfio_device_state_is_precopy(), so nothing else accounts for the device's stop-copy data. Can migration_iteration_run() then reach this check with the device blob excluded? complete_ready =3D can_switchover && (pending_size <=3D s->threshold_si= ze); If so, the full stop-copy state would be sent after the guest is stopped, outside the configured downtime limit. Before the patch, the downtime limit applied to such devices. Should the omission be gated on vfio_device_state_is_precopy() (or vfio_precopy_supported()) together with successful queries? [Severity: Medium] Even for a PRE_COPY device, can the stop-copy size be dropped too early? vfio_update_estimated_pending_data() zeroes both counters whenever vfio_save_block() returns 0 bytes: migration->precopy_init_size =3D 0; migration->precopy_dirty_size =3D 0; vfio_query_precopy_size() can also report 0/0 transiently. Either case satisfies the "drained" condition, and nothing re-checks the stop-copy size afterwards. The change is also not opt-in. There is no property or capability gate, so every VFIO migration user now gets this behaviour. Was Nova's stepwise downtime-limit raise meant to be bypassed for all devices, and not only for the vGPU case described in the commit message? > =20 > trace_vfio_state_pending_exact(vbasedev->name, *must_precopy, *can_p= ostcopy, > stop_copy_size, migration->precopy_in= it_size, > migration->precopy_dirty_size); --=20 Sashiko AI review =C2=B7 https://sashiko.vzint.dev/#/patchset/20261102-vfio= _v1-v6-0-9bc7cabc850d@virtuozzo.com?part=3D34