From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <010701a1c1013ebe-14da1a48-f2c5-4131-b0d5-95329b962aa2-000000@eu-central-1.amazonses.com> Received: from GVXPR05CU001.outbound.protection.outlook.com (mail-swedencentralazon11023139.outbound.protection.outlook.com [52.101.83.139]) by lore.virtuozzo.com (Postfix) with ESMTPS id 4D1C081377 for ; Mon, 9 Nov 2026 15:50:50 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=yXn/2Qc+6oFnRHP9jp4dS71wp0ai6eO/bnckxeMfPV14zstvggdKLrvnm0Gzxbp3vP74vhOMnGCOgzHw2V9Xlqd8iavttgo511cc1VBBCWoPJ8ju4U9BmX0TQ1jidI1BE4/HQZLo9isV5VaJW/IaAhFI6nVksvvOy20jS3eV29CLdoyPVpHEM7VmdfrNckO/RwY3OZ0djPX6ADvpB0l+4OgbegXkG15nWZ9e0xt8NwReVYCAipXVvGGK1cp7/79W5/XmHGOUlKoidB1nRk5SV62EfW5SAaksg41xyWyeuoSPio+5HIUyOtc9ucyhXWYFm+2trNeLwtewZ28it1Ux3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xuwyNAgv03xV4OE07ZJPyRACXCB1US/Gr/JA610eN8Q=; b=am3m3UIlM8AJpuNUhD0yD33kZifRNu51xzrsXEjhPIIfTk3D81N2wC+9tok0f63dlGSD+YJR2yNFi0k25LFTkPtOWuH4OWtiLa21y4CLRhmdzxAxYFspP226Z6ZCNZR9nit0TxRYXUYnnhubr2Xmp8nJiiqe79DS0zcs76t0scFpm8cHZPkSqJ2nIYbNdADVij2smTF69HYmXrzjGlIANROkZIZO0XMAPu+U9eljQoonOaUZ2uVA7HqAHuUlztHHqlGk95UcrOHLdOy+2/ExbhS6CCjM5jXXNz/FXimkp5R7UnzOnEyGtegkKNMnYMq5VElneFEaBri5RVh3N9qxJA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 69.169.224.2) smtp.rcpttodomain=virtuozzo.com smtp.mailfrom=eu-central-1.amazonses.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=sashiko.virtuozzo.com; dkim=pass (signature was verified) header.d=sashiko.virtuozzo.com; dkim=pass (signature was verified) header.d=amazonses.com; arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.onmicrosoft.com; s=selector2-virtuozzo-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xuwyNAgv03xV4OE07ZJPyRACXCB1US/Gr/JA610eN8Q=; b=XzxH1H/2ejqUC+gcrCK1jJVBWYLrTfIC2xPfKcIXZ2hAhoIgcRy81W3wSqXTRXRb+GkILXEKX0FmWP5Br9Wo2LBiKQaLG/XJrlZhvtpgMuCtb2HCxxZPgVSrwfFl0NX+tHFtnshhi4mAGMhvTsNpSE5V+ISiu0BiZZxWz8LP/0k= Received: from DUZPR01CA0088.eurprd01.prod.exchangelabs.com (2603:10a6:10:46a::14) by FRZPR08MB10952.eurprd08.prod.outlook.com (2603:10a6:d10:137::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.592.9; Mon, 9 Nov 2026 15:50:47 +0000 Received: from MAD0EPF000008AB.eurprd04.prod.outlook.com (2603:10a6:10:46a:cafe::ad) by DUZPR01CA0088.outlook.office365.com (2603:10a6:10:46a::14) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.592.13 via Frontend Transport; Mon, 9 Nov 2026 15:50:47 +0000 Authentication-Results: mx.microsoft.com 1; spf=pass (sender IP is 69.169.224.2) smtp.mailfrom=eu-central-1.amazonses.com; dkim=pass (signature was verified) header.d=sashiko.virtuozzo.com;dkim=pass (signature was verified) header.d=amazonses.com;dmarc=pass action=none header.from=sashiko.virtuozzo.com; Received-SPF: Pass (protection.outlook.com: domain of eu-central-1.amazonses.com designates 69.169.224.2 as permitted sender) receiver=protection.outlook.com; client-ip=69.169.224.2; helo=b224-2.smtp-out.eu-central-1.amazonses.com; pr=C Received: from b224-2.smtp-out.eu-central-1.amazonses.com (69.169.224.2) by MAD0EPF000008AB.mail.protection.outlook.com (10.167.241.183) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.615.4 via Frontend Transport; Mon, 9 Nov 2026 15:50:47 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=23yjbh4k34hwzfubhuiim7m4inbd3ick; d=sashiko.virtuozzo.com; t=1794239446; h=Message-ID:From:Subject:To:Cc:In-Reply-To:References:Content-Type:Content-Transfer-Encoding:Date; bh=kVEGnyZNsHftvRJC2ii5k7l6IIIiYHC+UnAC0ptjS44=; b=VRVmGA5d5HUxI1e7LvU7g1n0rfluEBAJIgNDyc5lf9iLrh9D3svOvjT3Ccd3iDkR g+makXTnkNoLwSTRfhnUOqhmq241juUtcyj8lJjppVEIST3U8bfVEansehXRkTSEj1F HDzmZkYOZtr8Q+jBTBvHYFfrtIX2ICLi6ue8wJcZ2+SUbKtOnKW7Jkpn9PTxn95vcCC P/wwLfU3boxw4NIKto2npgTUhU2A0LjEdqw7AF5Op7d0ltsA4+PLKr5TCdb8LDITSkD yYmAj4FLoCilRHIDWa5RXTCQzs3oKY9euEW8QYPxbG40Buz/N7AQYYSTMeYJTlNTY7R E9f8TX7wxQ== DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/simple; s=qftdzk2dqsatjnlrq4r5brjbihpfcrsh; d=amazonses.com; t=1794239446; h=Message-ID:From:Subject:To:Cc:In-Reply-To:References:Content-Type:Content-Transfer-Encoding:Date:Feedback-ID; bh=kVEGnyZNsHftvRJC2ii5k7l6IIIiYHC+UnAC0ptjS44=; b=eDLFCIOo5vUPglrzfCsKBXfjMrsrcCWh4Rcag1fYtd05TQ4U1nzMf0lY1EQyZ94W tvQbn5XNZVFHUpQbmpdVidTLl0ZLJ2QPGtihoQA9X5DHvuNR69rMzewn+l64hOID5wr j19xXmLf0TStnVJ0hsGOFlSGUXGlI6c6CGxz09hE= Message-ID: <010701a1c1013ebe-14da1a48-f2c5-4131-b0d5-95329b962aa2-000000@eu-central-1.amazonses.com> From: no-reply@sashiko.virtuozzo.com Subject: Re: [PATCH hci-dev v3 1/2]vfio/migration: stop a device left running after a guest shutdown #VSTOR-153976 To: Andrey Drobyshev , svt-core@virtuozzo.com Cc: den@openvz.org, andrey.drobyshev@virtuozzo.com In-Reply-To: <20261109154346.153153-2-andrey.drobyshev@virtuozzo.com> References: <20261109154346.153153-1-andrey.drobyshev@virtuozzo.com> <20261109154346.153153-2-andrey.drobyshev@virtuozzo.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Date: Mon, 9 Nov 2026 15:50:46 +0000 Feedback-ID: ::1.eu-central-1.2wRQJBpg9g1fGRYexx3dUuYGS25Bux+2TEOx72/m6U4=:AmazonSES X-SES-Outgoing: 2026.11.09-69.169.224.2 X-EOPAttributedMessage: 0 X-EOPTenantAttributedMessage: 0bc7f26d-0264-416e-a6fc-8352af79c58f:0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MAD0EPF000008AB:EE_|FRZPR08MB10952:EE_ MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 248c12aa-c248-480a-6414-08df3e7938bb X-LD-Processed: 0bc7f26d-0264-416e-a6fc-8352af79c58f,ExtAddr X-MS-Exchange-AtpMessageProperties: SA List-Id: svt-core@virtuozzo.com X-MS-Exchange-Transport-Rules-Loop: 1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|32142699015|48200799018|61400799027|9140799003|376014|5113699003|55112099003|4113699003|10067099003|16102099003|22082099003|18002099003|56012099006|5063699009|4143699003|261028144727099003|5023799004|261102123227099003|261021121427099003|261026140227099003|261009223027099003|261026121527099003|261022105027099003|261022075727099003; X-Microsoft-Antispam-Message-Info: w8A7MoSwP3oYBNoMDkle06xd9rcijJiBFIx3i32g+MAa5AHhZ9UKie819myHU5/ISRUG8fyCG87njdz8cPxcWDfXk3LQ+J7bGu7W9DAXZI/Ae1Yc581/7P1TdwSe2dZlNPXsad16wFR5RxvbD54GoKI4MlA/iJ+xawa2ry46HaJaFUB4tyrF6KR5JFIwebXyPdMJ9vdJr4+Z1a+iOxBCLqjQh+MnL85InPC/eWhykFXielKvdM1hBgApRwg5SBUQ0hxK7w/QLKyRkaJ9F5yYmQc8YJVDKoH9hogEwdnka0EaVImr8ld4fTZfL6o1TRCZkd7sYirPxCN+nRXT48xo/VRMJPFOTqzNrsSUSgv/wht60Sv5eLGBzK+uM2tIMkVxIGgioHQQ15+8jTqjivlLy9+NRXh0TKg1c/CMJt1VzcWKqtRbVX0zdeQlUZTy2HGU+yDVzn2IOvuEH2Qn+YOlkp/Wg+YKGIFqOaR/F4a2Qb+Nd/cef9ZrEuplIvI2+BEjMyqLTH2/ETVvBelAIvlodQyj8cfbxnPJrJEqR/XvSrxjboMhI8Ai0vNQOPTZi2dsFLpQkKTrhgEM5E1v8QCDfVf5fTU/1INzPZjwBMaFlC2zNdLoS5IBBGAbMXSD4u1bz/bp5DKzlT83cS21y1wQ0qgJiV4Hk7/BAZuOemRB8lgCIXRnKu+4klQNSz2XfHDS5RHgV+erozH6kGCbOOVd8/jOrxwvhZzGoN6z6vfSC0XMlGs3VFnnhcjuDGjtk+dhpIvciwpRHKrxAWw3Df/66vUEV0twFbU3hQGmeszZmDIHHjLRmsevcsMewbVZvsGtekpg0Y7aTadIEv6JB0P6aCOdswQl456YMWKGZAJK7F1V8bSgX0D2Gc1sRx1whv2O X-Forefront-Antispam-Report: CIP:69.169.224.2;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:b224-2.smtp-out.eu-central-1.amazonses.com;PTR:b224-2.smtp-out.eu-central-1.amazonses.com;CAT:NONE;SFS:(13230040)(32142699015)(48200799018)(61400799027)(9140799003)(376014)(5113699003)(55112099003)(4113699003)(10067099003)(16102099003)(22082099003)(18002099003)(56012099006)(5063699009)(4143699003)(261028144727099003)(5023799004)(261102123227099003)(261021121427099003)(261026140227099003)(261009223027099003)(261026121527099003)(261022105027099003)(261022075727099003);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: HNZY/7nE16Nw/9wrVsfnC40D8+TWnw8cU4+lLnhSAftUhoCPE7n/CmuEgRMw4iMvFqJ5Eb53aAL/dxQkWXTmgzETwIaEK6HMCHZwF4RzADA8KRLmLspXnEr41gfZ70KmCMYeYA2/usuyQPZJJEY6byRVUSogyIJbZQceZH9j+MPBE84hbz2frpqN1gwQ2Ny+8Zv7pfbiCkVHQUX3FuhQoNku40vUn8pBmemVT1hOeYwxzH3s7JxVsCDarcx2nmuqFGz3zuGTxjhHNMZ4Kkg7U/kPNPCvguLoZ4c0ebPJPy2IHEnC2diDPgyF8A/V+iJEsO/1b5DTW6haYz+fpzKfw+HxgI6Iz1JkIGHRfOcAGXG6Kqu4atlc5eaL8ekrYHiAbapkq/KQTpFota2EcYFOfTaV9A0VubLMOzdfoLRthmGCKy87NGnZYGeT+teaIVoG X-ExternalRecipientOutboundConnectors: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Nov 2026 15:50:47.2684 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 248c12aa-c248-480a-6414-08df3e7938bb X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-AuthSource: MAD0EPF000008AB.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: Internet X-MS-Exchange-Transport-CrossTenantHeadersStamped: FRZPR08MB10952 Thank you for your contribution! Sashiko AI review found 2 potential issue(= s) to consider: - [Low] vfio: comment claims nothing else leaves a running device in SHUTDO= WN - [Low] vfio/migration: RUNNING to STOP skips the P2P quiesce step for P2P-= capable devices -- commit b98ac2b60638ee9b579c31103b9e4ae12b82d728 Author: Andrey Drobyshev vfio/migration: stop a device left running after a guest shutdown This commit adds vfio_migration_stop_after_shutdown(). It moves a still-RUNNING VFIO device to STOP when the VM is in RUN_STATE_SHUTDOWN. It is called from a new shutdown notifier when QEMU quits, and from vfio_save_setup(). The commit message says both calls are no-ops today. > diff --git a/hw/vfio/migration.c b/hw/vfio/migration.c > index 7e7fb496bbecb..bfafc14e5b942 100644 > --- a/hw/vfio/migration.c > +++ b/hw/vfio/migration.c [ ... ] > @@ -415,6 +415,25 @@ static bool vfio_precopy_supported(VFIODevice *vbase= dev) > return migration->mig_flags & VFIO_MIGRATION_PRE_COPY; > } > =20 > +/* > + * Stop a device still running while the VM is stopped after a guest > + * shutdown. Nothing else leaves a running device in the SHUTDOWN run st= ate, > + * so this is a no-op for every other VM. > + */ [Severity: Low] Is the statement "Nothing else leaves a running device in the SHUTDOWN run state" accurate? vfio_migration_state_notifier() restores the device to RUNNING on MIG_EVENT_PRECOPY_FAILED, without looking at the run state: if (e->type =3D=3D MIG_EVENT_PRECOPY_FAILED) { ret =3D vfio_migration_set_state_or_reset(vbasedev, VFIO_DEVICE_STATE_RUNNING, &local_err); Consider a -no-shutdown VM sitting in RUN_STATE_SHUTDOWN: vfio_save_setup() vfio_migration_stop_after_shutdown() /* device -> STOP */ migration fails or is cancelled vfio_migration_state_notifier(MIG_EVENT_PRECOPY_FAILED) vfio_migration_set_state_or_reset(..., VFIO_DEVICE_STATE_RUNNING) The VM stays in SHUTDOWN and no VM state change is sent, so the device ends up RUNNING in a stopped VM. This is a second way to get that state, besides the guest shutdown itself. The quit notifier and the next vfio_save_setup() stop the device again, so nothing breaks functionally. Could the comment, and the commit message wording that both paths are no-ops, be made more precise? The same applies to a migration destination that ends up in SHUTDOWN, where the device keeps its init-time RUNNING state. > +static int vfio_migration_stop_after_shutdown(VFIODevice *vbasedev, > + Error **errp) > +{ > + VFIOMigration *migration =3D vbasedev->migration; > + > + if (!runstate_check(RUN_STATE_SHUTDOWN) || > + migration->device_state !=3D VFIO_DEVICE_STATE_RUNNING) { > + return 0; > + } > + > + return vfio_migration_set_state(vbasedev, VFIO_DEVICE_STATE_STOP, > + VFIO_DEVICE_STATE_RUNNING, errp); > +} [Severity: Low] Can this skip the P2P quiesce step for P2P-capable devices? This moves a single device straight from RUNNING to STOP. Other VM-stop paths go through vfio_vmstate_change_prepare(), installed in vfio_migration_init() when VFIO_MIGRATION_P2P is set: prepare_cb =3D migration->mig_flags & VFIO_MIGRATION_P2P ? vfio_vmstate_change_prepare : NULL; That function first moves all devices to RUNNING_P2P or PRE_COPY_P2P. Only then does vfio_vmstate_change() move each device to STOP. With the next patch, a guest shutdown no longer runs vfio_vmstate_change_prepare() or vfio_vmstate_change(), because they return early via vfio_vm_stopped_by_guest_shutdown(). The devices stay RUNNING. The only remaining stops are this helper, called from the quit notifier and from vfio_save_setup(). Each stops one device at a time, while other VFIO devices may still be RUNNING and able to DMA to it. This is speculative. It needs several P2P-capable devices and a guest that left a device doing peer DMA after shutdown. Is the bypass intended to be safe in that case? The commit message and comments don't mention P2P. [ ... ] --=20 Sashiko AI review =C2=B7 https://sashiko.vzint.dev/#/patchset/2026110915434= 6.153153-1-andrey.drobyshev@virtuozzo.com?part=3D1