From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from PA4PR04CU001.outbound.protection.outlook.com (mail-francecentralazon11023082.outbound.protection.outlook.com [40.107.162.82]) by lore.virtuozzo.com (Postfix) with ESMTPS id 592BB802BE for ; Fri, 4 Sep 2026 19:05:02 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=hw1a9OMpG90np97+C3cpd+x5764afJaGJj+9yfJKgjjPQqE28vECtJkMjQiaZShU9eYUb26AiGOSGWDSuEmYp37gXTPbozQQDiQZtQrCnvILQds1WS1RHvHYfAkgqPprXzzOdKaXU2gdc3y9Vjg9e3H1NU9hm/V4Zjqq1lailheeJfEx/6HXxED9v+s3wLuGuhRiOGu9xw4kEa678G+pOHRHIwLowgKYwz3VLbqwy+4eZzjfGU/BTMzGNQI5Bpk96kckpfrgA+iSM+JEjTr51AEs4VM9nZhibRJ3V39McbYPI8rTGETdhTKIP/x3gkVD87dhtY8RAptkUUApUJhKcw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nMxb1rHeKZJmC8UIDKPPLMaqaCNGBC/aSkkEl4W1zpI=; b=qQAf2OLaWeoqOLBDtL2VmilNW2IXHnlM+m0QTzOAQmkviVfPJH3Rsc2Bf9Rj0RDbEawVc4K2vqCqWABKf8njhQMrJSKLkKXClYVUkEAd/VSTC8jFr9pnNyEJeGr6XJav6RA/DSm9CjJeH5wB4vsxPBZzOu0AJlca93hx0EG+sGPPUpupIK4Sj5YsSjadf1k9VWUZtZZ8QQi2Mfx1Q0K8OmW3Psy08LRmVbX26yamjwq169gn+bRMUer/XXTxEFiJZy0m+ufCfoQP51D7iL+b6N24jkjYFahHKO/mpi7cF9QG/+IRGSh5JeEa2OTulsd8TB78ivIpx1E7/e/rixSh2Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nMxb1rHeKZJmC8UIDKPPLMaqaCNGBC/aSkkEl4W1zpI=; b=QSRGnJJ+2W2PTNn/H+kwWOVhs7g4kffEW2y+ZkfsLnIBCrmo7TgN0y7JFwKGKPmNdo13jsXvO2YL/HhZ29PsRURsLJ8+bNnUvgxMUmBKvlll6hQfAPTi7AWInvMVR2JajSp9fwRJx5UZKaYsvwHbZckM30TC67hCA/VFtVfEhdepsaTOOg3+bQHRsaP3q+467IqC0zg3NIsrZmAm92Ghu2OL52OTUo2Ay+Cp280AcYgwuq5nYs+PCHg+5mMyUdqr+nb9yzEV89Ikm/GouCV+sY73nczGD44pG+rJF9SSxxn9vyLf10iPlbHd2VjnOMMbU1KyIAaWHde5JA2LD1RjSQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=virtuozzo.com; Received: from DBBPR08MB10650.eurprd08.prod.outlook.com (2603:10a6:10:52d::7) by DB9PR08MB9827.eurprd08.prod.outlook.com (2603:10a6:10:460::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.382.10; Fri, 4 Sep 2026 19:04:57 +0000 Received: from DBBPR08MB10650.eurprd08.prod.outlook.com ([fe80::1b2a:832c:d136:a37c]) by DBBPR08MB10650.eurprd08.prod.outlook.com ([fe80::1b2a:832c:d136:a37c%5]) with mapi id 15.21.0382.007; Fri, 4 Sep 2026 19:04:57 +0000 From: Andrey Drobyshev To: svt-core@virtuozzo.com Cc: andrey.drobyshev@virtuozzo.com, den@openvz.org Subject: [QEMU HCI-8.0 PATCH v2 06/13] usb-host: drain in-flight URBs across CPR #VSTOR-137800 Date: Fri, 4 Sep 2026 22:04:36 +0300 Message-ID: <20260904190443.795902-7-andrey.drobyshev@virtuozzo.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20260904190443.795902-1-andrey.drobyshev@virtuozzo.com> References: <20260904190443.795902-1-andrey.drobyshev@virtuozzo.com> Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: VI1PR07CA0215.eurprd07.prod.outlook.com (2603:10a6:802:58::18) To DBBPR08MB10650.eurprd08.prod.outlook.com (2603:10a6:10:52d::7) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DBBPR08MB10650:EE_|DB9PR08MB9827:EE_ X-MS-Office365-Filtering-Correlation-Id: e2cadcb7-e439-4847-fc5a-08df0ab76944 List-Id: svt-core@virtuozzo.com X-LD-Processed: 0bc7f26d-0264-416e-a6fc-8352af79c58f,ExtAddr X-MS-Exchange-Transport-Rules-Loop: 1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|52116014|1800799024|23010399003|366016|38350700014|10067099003|6133799003|22082099003|5113699003|18002099003|56012099006; X-Microsoft-Antispam-Message-Info: 1DUKUXl2LI/u1SfLC+eErXG4dOiR/plqGViQZO2G0hzL+Qstc9WIWvjnhHmcrP9scQZK4oPCszuJD1yJjWVOxVccmcvo2p2Xt5VfJ6UcuqNUS8xIXv7AQ7379IojQO0iLhnvtkDTDN1oPYxwvf+cnqzRExZl6jlZSrk0c3ComT+Jqex+xzL3gZ1ZX6bJLN3tTsy65ixnMoiNUjD5iFZGv9I8NPb8v55W/n6qePV/liOalGvevEt2FIoOVUSY1G0L6cOnTVAMV9VJLrwZ5gTqnS/mXgtSjMN0cPE7UGSFsuCtOMsbtCxYcAqNC2oGHthHKI2/f/mchOgbJv7y9EZxwwx0tCLfRBUuC9rJyg9WX7M3Iv6y/ukm2eYjb/Kqml/KFlIhZyqtTzXsz5Z8lGpYJC0O+2hhquuAS3mm94kEmgKYmt6PAM4vltcd8Xv8TknZSQFy90sZpjiE7d/vlggBCDQnEFOfsbItfx6s4STzPtKl7gVbtgjWgqd4/osy0yPBIC54/GhtA37/pcm1HLFFz9RiRouLNfNQUYhRuPuGbQ4I88QHIB3i1thr/QsgnaJXTStZ+MANkddX0EVrhMKmE/Q/kKTWB9QrLxz20/gfFngvFMl9ZbNG/4G3G94jHO5v7sKc7JwH2L6UQ+9cVExA6QLPCW8nKhGP81XxTKTk8Nh+QXslxR5GOhk3yL3RQumqfuJ2lLXM5vmk+q62eNvYhVFr225ib2+J8trLaA26ioA= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DBBPR08MB10650.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(52116014)(1800799024)(23010399003)(366016)(38350700014)(10067099003)(6133799003)(22082099003)(5113699003)(18002099003)(56012099006);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?gLcIPEkmvJz+OjbPNvZStKM5lSeHdaXeapUufF4NF8cB0PBFGeIanjZJdOVY?= =?us-ascii?Q?OLlzQRyDp2KJmu8KIisSiWIvFLUNzpP9FzG84CkJhpAdP3kcqKy19vN1O5ak?= =?us-ascii?Q?0nYg8lJ6hflkEQ68OFsR8bxhXzIIAndXVCZKwYBwW3H/j8WbXgNoRDm+0gJ3?= =?us-ascii?Q?MJULEk8jrLlNTfYoDOeIvj3rGsurmd1gi+HfmymS4mCx3aBbyGeUBDlzjeyE?= =?us-ascii?Q?rS5rHtqLnl8ckzEY0vqTEVYFv5cR5WoTzKD8ot33K09HxPpKN2YQ+tFUbeif?= =?us-ascii?Q?h1Z+I5Dxi2k9JcfJN2AElOjDz2koSrmeMnJb5tXkEa8izqWowV9T5V4bxvKl?= =?us-ascii?Q?xNg6OdPTrefagjmy/eUmsM8S9/B/nHU4YE8j5qDYRC62rY1YwlIjb5KUMGss?= =?us-ascii?Q?JignEL8AJX97Hqdd4Y7GLgBh/TdpNz5MEUeh8VPrL0ZTBALEWtwi4BQmbHQG?= =?us-ascii?Q?oKRXwrLz0LOI9SMW+dSOC3IPky3o/TTG1c7r7pLnQlLyy6LU3pGaJnuOBIG5?= =?us-ascii?Q?N9UbHa9GA1lO8qTrdZSB4iVpEIls9bmZGIVTz7wAIL4IlViWQpJQLKryXEKD?= =?us-ascii?Q?wshDm/SiZTRkwA6Offl3WKMzrrl0m97KjpcMToNPir7lFA69SolLEkESPd4o?= =?us-ascii?Q?S+78Gnmq3rtohDRRF9DgGEGO5aCFGuFNqU1gsVwiIwj4Woo7N7Q/SX4KG1Ut?= =?us-ascii?Q?YwcVpZs/FJ0rr7NfMt4WkIAi3C6UdlEmFD6y2PhIaWsz2MdjUsZcB8d0K4Ei?= =?us-ascii?Q?JckpuLIAAa4sH8aeMSJXZz1fRacBEWw9O5ETGT36I+j6Ia9DFJ9ytgPNX2Fs?= =?us-ascii?Q?WAaeTnjYcAdeIbe4s/cuZw3lhhJNda5cJ6QC0oxAU6HpSD8bc4ilej/Xlv39?= =?us-ascii?Q?qv7S/coeIMb9U4VjtXfi/znqLKtZ7qmnl65x+7NdiFeknREDRO8JsEm8Ny6z?= =?us-ascii?Q?E7EGYUBRRQxVgsqxBBzjV6gPS/t1YU5NgCE2vge0633k6pBXqoa4PTgG2/uc?= =?us-ascii?Q?Ne1MVp18ySODOXemGqUZQdII/CjbCb4fgHj0xB49glylroTJ4M9p16RX56ik?= =?us-ascii?Q?th7vdlD29eDFrEYwvTHW4Cu+91GR+SO4ijFOpY4VzH6ezXsNrnqCJhYJ1cDL?= =?us-ascii?Q?SULStlYn71oGLIuJ+K7Sa2iVepXCcc12pFIBvQ1DSkKviNnadDcWZDN1HOPq?= =?us-ascii?Q?3eKgwZs7wWBZdYfPfUl6OBDPTGjA9uw2hLP+DWnh1R23edexjqviRMlLOU97?= =?us-ascii?Q?DLSlAcgx1l4FTttwATR40ctYzfHwkg+Oo4gNKj19QCd/xUFcQ46gnk9qOUd9?= =?us-ascii?Q?GG3Zjun8Q1NyU1pw1LKXzeyfbSOlrE7JHXx2ubCW1BMnVCxf/umTNZ7vKZ1z?= =?us-ascii?Q?nRxKrBXQGIgt0pik3GKc+UZ0/BsY++XEgDHksaIzFXsqTa3kdLAwvgSTRUJ1?= =?us-ascii?Q?yPTr56m/bjPDVNOLIZCrVwYC58oYdVhAUh7Xf/ny3/ocMn5D48ixg3h+FbVR?= =?us-ascii?Q?N3128svxX0Y9vVPEGRFGDgAb0swvKjXPEEBCUHrIdHx+jpeJwbdiRhCbSNYA?= =?us-ascii?Q?Bi0EDdj1maGovyL2ogvupA0W4Ix7TbAZA2ZypJFVbHdl2Qlm/Cdm1tLrBFUv?= =?us-ascii?Q?W6UnHNheif9CpZ1h9OINr1A3SyzU/iCo5QnIK8lIt9gzTDjRgg/YyHk9QhHL?= =?us-ascii?Q?GLm7KYe617WeJ8+XQvgYt81dAcEsup5ONZ9hhjhfrTd3Pkoxj7VEV1jOvoE9?= =?us-ascii?Q?PvZpiCqHnq7NH5SyRbUWDdD1lfPQ5RA=3D?= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: e2cadcb7-e439-4847-fc5a-08df0ab76944 X-MS-Exchange-CrossTenant-AuthSource: DBBPR08MB10650.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Sep 2026 19:04:57.2860 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 5uqdR8iqrwlPPTmDPt+2sbxjvu3uAoQXmV9p7QrhbmWAJJqWuStzDdAtGNHWjB0qsscUxA3CaDsToaiDW+XpN8/8Wc7HXClJOFKx84Hoeos= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB9827 URBs submitted by the old QEMU must not outlive it across the switchover: the kernel would complete them into buffers of an address space which is gone after exec, and the new libusb context would reap URB pointers it never submitted. So, in .pre_save() for the CPR modes, cancel and reap every pending transfer, without completing its packet. The packets stay ASYNC and their TDs remain on the transfer rings, so the target re-executes them when usb_xhci_post_load() kicks the endpoints - the same in-flight replay as on regular live migration. A partially executed transfer is simply re-run from the start of the TD, which is safe for mass storage. Isochronous URBs are not on the request list but on their own rings, and freeing them while in-flight would leave them queued on the preserved FD. Cancel them so they are unlinked and reaped: an iso URB in flight when the guest stops may never complete on its own. The target reconstructs the iso stream from the guest's transfer ring. Signed-off-by: Andrey Drobyshev --- hw/usb/host-libusb.c | 79 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 79 insertions(+) diff --git a/hw/usb/host-libusb.c b/hw/usb/host-libusb.c index d695ba3170f..c8a893073a6 100644 --- a/hw/usb/host-libusb.c +++ b/hw/usb/host-libusb.c @@ -48,6 +48,7 @@ #include "qapi/error.h" #include "migration/blocker.h" #include "migration/cpr.h" +#include "migration/misc.h" #include "migration/vmstate.h" #include "monitor/monitor.h" #include "qemu/error-report.h" @@ -1145,6 +1146,68 @@ static void usb_host_abort_xfers(USBHostDevice *s) usb_host_reap_xfers(s); } +static bool usb_host_iso_inflight(USBHostDevice *s) +{ + USBHostIsoRing *ring; + + QTAILQ_FOREACH(ring, &s->isorings, next) { + if (!QTAILQ_EMPTY(&ring->inflight)) { + return true; + } + } + return false; +} + +/* + * Quiesce the device for a CPR switchover. Cancel and reap all URBs, + * so that no URB submitted by this process can outlive it. + */ +static int usb_host_cpr_drain_xfers(USBHostDevice *s) +{ + USBHostRequest *r, *rtmp; + USBHostIsoRing *ring; + USBHostIsoXfer *xfer; + int limit; + + QTAILQ_FOREACH_SAFE(r, &s->requests, next, rtmp) { + if (r->p) { + /* Clear r->p so the reap callback early-outs on it: the + * callback must still run (it frees the request), but it + * must not complete the packet. */ + r->p = NULL; + } + libusb_cancel_transfer(r->xfer); + } + + usb_host_reap_xfers(s); + + /* + * Iso URBs are on the rings, not s->requests. Cancel them: one in + * flight when the guest stops may never complete on its own, and none + * may be left on the preserved fd. The target replays from the ring. + */ + QTAILQ_FOREACH(ring, &s->isorings, next) { + QTAILQ_FOREACH(xfer, &ring->inflight, next) { + libusb_cancel_transfer(xfer->xfer); + } + } + + /* + * Cap the reap at 2x a full ring: + * iso_urb_count URBs x iso_urb_frames packets each + */ + limit = 2 * s->iso_urb_count * s->iso_urb_frames; + while (usb_host_iso_inflight(s)) { + struct timeval tv = { .tv_usec = 1000 }; + libusb_handle_events_timeout(ctx, &tv); + if (limit-- == 0) { + return -1; + } + } + usb_host_iso_free_all(s); + return 0; +} + static int usb_host_close(USBHostDevice *s) { USBDevice *udev = USB_DEVICE(s); @@ -1821,10 +1884,26 @@ static int usb_host_post_load(void *opaque, int version_id) return 0; } +static int usb_host_pre_save(void *opaque) +{ + USBHostDevice *s = opaque; + MigMode mode = migrate_mode(); + + /* URBs submitted by this process must not outlive CPR migration */ + if ((mode == MIG_MODE_CPR_EXEC || mode == MIG_MODE_CPR_TRANSFER) && + s->dh) { + if (usb_host_cpr_drain_xfers(s) < 0) { + return -1; + } + } + return 0; +} + static const VMStateDescription vmstate_usb_host = { .name = "usb-host", .version_id = 1, .minimum_version_id = 1, + .pre_save = usb_host_pre_save, .post_load = usb_host_post_load, .fields = (const VMStateField[]) { VMSTATE_USB_DEVICE(parent_obj, USBHostDevice), -- 2.47.1