From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from GVXPR05CU001.outbound.protection.outlook.com (mail-swedencentralazon11023082.outbound.protection.outlook.com [52.101.83.82]) by lore.virtuozzo.com (Postfix) with ESMTPS id CC8F4803C3 for ; Fri, 4 Sep 2026 19:05:04 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=cbO1IUMRq/ujv5tPfDqaEw0EWxWR/dt/QT5NOE0G+wSaUM3fQ/e7jM7P5fg3zq5YkUYkbjBjNq45i5FEkqEWvSJXtCDDhP0Cq9FecL0OeWRbPWvVG4jqJ/GlI5xc7FKVGs6EYsDPqQEBeQy950i2yG9CqVIMqJUd6EoZOKtafcycZlmdzsJQ/WlNgTYmVQD9x5RE3rL4WNRrlFI4f4XsuRJiQv+m6P/qI03OBo/mpvw5RO17Fs3B5lGMmMHdZPwKIuk5RZj09Jj0M5jKZ2zjZrWoPm5p/LQDW2xQwD7j6596znZVkzPg/XgRsm+4poquJqklDTiq+MJ6cBxWAht39A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=WvToAsBfOI7txbran5hBvRc86Q7KuNP6Ol5y8BU11do=; b=XXOTiZ2Q8V//DEwgjRFg9c2rwPqi1IFWRGiy2QDojB/zuZHAbwg1UdWbWO0iKmrTRhr1jfS+2HWM2dNS8FZjiEiMoCpu6dBwB0qTD/ww/+Gpkb+30T68p3WWUvheS0kn6kpUIVoFnHJREBUQGGje1IpsLcOPU5iEMg2/kBYLb4dMKph/COZphyRZWczL+yZaa2GjN/NcIi1h5zSdHLZmjUQVZ7BcX9UmaxCs+3ItwiYAbaM6VfsSi6tT/Mc/Z4uTZjLTH3wSuVv8wSNhDI9ewlImYf8Yf7ZyMQbNUE58Fvhl/qG0W64WWWNkYBjBa0hlDdocna4Wj0LFdQY9ntHu0A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=virtuozzo.com; dmarc=pass action=none header.from=virtuozzo.com; dkim=pass header.d=virtuozzo.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=virtuozzo.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WvToAsBfOI7txbran5hBvRc86Q7KuNP6Ol5y8BU11do=; b=FNGhstwruOnWA9hZD0jGpaD86nfzX0Pe2/sNaODANeXpY+DP+6/0myRbPi+o0629XhQxAxiUjkJbp1tjScd7JBkf83cHg+PgRC0j7Q6UP+HB/+ck8PWnktsyYDqr9AG7lx4SS8pSAkAIHV2LGjifDdtfznt86iBzrQhohiMl2d9vjTLvLqFOq9qkSgtCg4YX3LfnyccAeTvTQcSirilLbl7l/G/BPFWkSAvYl2ABkKMk0wdziUlXE6BTbYkRbdJtN1HRHGgjIqPLsWAONICBil6EpJRAdibaImv1gBz6wmxMyXbQw+4bwzkUCPT0qPq1SrgRqK/JwDBeN3LQEzAI5w== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=virtuozzo.com; Received: from DBBPR08MB10650.eurprd08.prod.outlook.com (2603:10a6:10:52d::7) by DB9PR08MB9827.eurprd08.prod.outlook.com (2603:10a6:10:460::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.382.10; Fri, 4 Sep 2026 19:05:01 +0000 Received: from DBBPR08MB10650.eurprd08.prod.outlook.com ([fe80::1b2a:832c:d136:a37c]) by DBBPR08MB10650.eurprd08.prod.outlook.com ([fe80::1b2a:832c:d136:a37c%5]) with mapi id 15.21.0382.007; Fri, 4 Sep 2026 19:05:01 +0000 From: Andrey Drobyshev To: svt-core@virtuozzo.com Cc: andrey.drobyshev@virtuozzo.com, den@openvz.org Subject: [QEMU HCI-8.0 PATCH v2 09/13] usb-host: hand off the device across cpr-transfer #VSTOR-137800 Date: Fri, 4 Sep 2026 22:04:39 +0300 Message-ID: <20260904190443.795902-10-andrey.drobyshev@virtuozzo.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20260904190443.795902-1-andrey.drobyshev@virtuozzo.com> References: <20260904190443.795902-1-andrey.drobyshev@virtuozzo.com> Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: VI1PR07CA0215.eurprd07.prod.outlook.com (2603:10a6:802:58::18) To DBBPR08MB10650.eurprd08.prod.outlook.com (2603:10a6:10:52d::7) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DBBPR08MB10650:EE_|DB9PR08MB9827:EE_ X-MS-Office365-Filtering-Correlation-Id: 52da8e4b-1d14-4208-47ab-08df0ab76a89 List-Id: svt-core@virtuozzo.com X-LD-Processed: 0bc7f26d-0264-416e-a6fc-8352af79c58f,ExtAddr X-MS-Exchange-Transport-Rules-Loop: 1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|52116014|1800799024|23010399003|366016|38350700014|10067099003|22082099003|5113699003|18002099003|56012099006; X-Microsoft-Antispam-Message-Info: 5vQUe1UL+gevkmQJ09O4dL2gzWarEEQM9x3UFkdlD7Nx/bOAuDKnO+8FseIEpiBZ/Jc4OmxVrWGQ7mY9iKLpFLhvDX0xiMPUtJ7ccSeR3OD248bj6Vb3rv+VAUIKvVbzcf9W81v7v4xcvfzAzmd0Pitb4EfZ5rYpYt7zbEKvCj+SWXN3lV9bt2ZGE/l0HorDNy5OcNCMd8r4upaUYbTg+ShXKKPAbooLeFMODP9NW/BnNzuhpb1j1Gc95WK63y0SI3XYvj0KlKtPAyvaoeYXZHCEAgj1s94BlMzxl365Jw5MHFcQytAZfQ3XJamAHBIIvSHSbreR1vQy6mTU1EPOPzT8lfoS5URdvDAxHOts0JMbSqSjYVK/5p3iq0kPltOB//bpYqW7Zpb+wSWrg6Vm4iCZYIniMlE1S+O6gRKlQV/gyByr4Pvz2aN8oBt/F8WxlwU5g+gwOrxVY8u9ScLvmg1xs/hHlkavxBFIah83kQixGCfTBfN1fHGwbyLbeWyjhnOyWM1KH+dB2KvO3XJksBPKnujMtDz9UYZdyNhk+agZRdJetEIMDAwPA+mahBP7ytqhNjtzbVtdQdepgHr+d+FcNWDV0RdeN39teYofQ34uXJAULcD6jwVfkc7Zf5tEhvRKvQFkEyB1hLWmWSV0E8aELeKGkq41wSduaXktCKQQOv+v3iPSG3mh+PbwMQ72/PExGXyjU2h0Q3+Oyel72Oi2bNFBCp69rbzpwEG+fPw= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DBBPR08MB10650.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(52116014)(1800799024)(23010399003)(366016)(38350700014)(10067099003)(22082099003)(5113699003)(18002099003)(56012099006);DIR:OUT;SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?/WkyvGsNag6a8mySboX+e6cYs9lqm08lBffnI9uBJAgB9B8A14ZSRxNeMpqd?= =?us-ascii?Q?5K0PI+5qHOgJVOsWn5wKqRcRE/qqiqP4fLoEG7hQOP4rA+PcdRqsv5HTeqi5?= =?us-ascii?Q?2aQJnnCzK2NvJh7JdhD01v7lAWzLlKBsYSQy7qTUoa5Zl3ZMN4/BaBVp6Tnp?= =?us-ascii?Q?ydydIYoTPlE1RZMnmgvTAhwwHwGh95HGNW+dO0I2dTEhpqyPxtKACPn7kSz9?= =?us-ascii?Q?PLDNNvHuI2mdaLjxadSw19z9fBP5t0FEcdESFfrbYJlGCIro3wz7fyHjJel4?= =?us-ascii?Q?Pw73+YzJlvCdXMXJxQSMm1MZj984nnngHimF9dCcFJOuXFhISq6Vie5aYgO3?= =?us-ascii?Q?f10AovDhUARcBFVvUqGAkpZHXRLFRq0zBx3S/YQXnCOUzMfx1VNDvmFJ2kGv?= =?us-ascii?Q?x0NGFpdyLWDpKBe+sFObhYqYCAmvrDicn2rRoTzXjQLvLooNsrMywEw0xnO1?= =?us-ascii?Q?HDuu6jZS07qI63/eom8o//9ITYslEaWuMxJI2Bs1sDbMTJ7SygLqxwc4MOTc?= =?us-ascii?Q?PJiZiwW96ez9wiMiA2qpE4+aggaSpLk2+c0X7FB7btGwrcVvCxz0adxSxPZ7?= =?us-ascii?Q?7mlmfYV4v9o3f1FNGxNzSC3+I25QeXA59OSqYD+WKCjXh+zNJX5J+wjjbuWj?= =?us-ascii?Q?n6uSRF59fSxMgkGK9EBnp0uVWqVxfLZv8Yqu1oq2Sqb39ahViC2hcIWZ2eoV?= =?us-ascii?Q?T55lwx1zID2ormyeEVPpyXeru7PQAu2W4ldSIuohz5a9tqBBcX8oUywBvZ+o?= =?us-ascii?Q?YZo033nTC786/T6NQbddrNJFLiI+f2fFfngeOPLZR0MXE2rjodNUzsxHa7U4?= =?us-ascii?Q?B160D9c8NlU1YTFbnzDlNNi7087xillKD8tA5lR24jxjkIJ6XzvyPROacPxy?= =?us-ascii?Q?XFoeB/JGCjA/A4XUUrho06AhM7xmq5lY6mXR/VB94EreUvLQ33O35FBxO2fN?= =?us-ascii?Q?VbR9SBVMf47CRRcmUeHNetGw/InaCs/orc9lXpW//3cv90Tt/eDhO/KZq8JN?= =?us-ascii?Q?/AB3zG+NqT4ZwezqIoQDF+xsgQ69HbKAUF9Cwpq35bwiQZZL9EU+5cRIQzFY?= =?us-ascii?Q?LAvWgbQCCK5kxFwzxzldiZVsJKMn1izQGoF7UCst36PGdlvySWRdZJtVKyfe?= =?us-ascii?Q?mGEXsNTu9WjxZS2s+FoJ81siVg+GH5cKo7gH3QHGUDbmwatqawsQNp9N3jv0?= =?us-ascii?Q?ExcJk1Et9gD54XjNtMQ9/857tV7YHKuAH64dqknIE4nJAQUv2unMSfmaOGmq?= =?us-ascii?Q?+bAPMQhufvt9jBLPjHoxFl5LU6XarQtLKIRnTvDVcmEklXlYOTmmiNIFPe9G?= =?us-ascii?Q?SQKHkQjiCLsdsQ/ZL98xaM6X18nNRE1bM+mHPcz0UcujtVRkvPerXHTswKla?= =?us-ascii?Q?LCeMdDbe6V3RZ3bq1AgLkr7LLtuqC2thBLRUBhy2tdOn3Gyh+ACIA0sdAjxT?= =?us-ascii?Q?/NwXgols0fl+5TR7rKfEgH5NSP9j5b/3Sx79Rsg1U2eJj4RML8CuBmZBmyTl?= =?us-ascii?Q?7iFWZNJA4QEHo6Zo4MH4SQSqyMfb1rIcUW3ovw4TLb2UXRanG3Yi5j9Ybfm9?= =?us-ascii?Q?HnSsUoEyq7hLki3xsu5IwaJk25oq5P1sPhWqMCuH7v2/+R+uxlgMxd4ow0oJ?= =?us-ascii?Q?9uiNzPm992PSZtAP13kJ7kG63tE/Pl8a9J9d8qNTYYHWx12+0flw1+QamsbI?= =?us-ascii?Q?FpTg/ZCCneGx43o+nDhj7otFibSeDJdVFwK9Nzem7E7nastVDKQlQmCTjKfH?= =?us-ascii?Q?+hImI6rWFDNsjKh17C+JQSWW+XRS6ME=3D?= X-OriginatorOrg: virtuozzo.com X-MS-Exchange-CrossTenant-Network-Message-Id: 52da8e4b-1d14-4208-47ab-08df0ab76a89 X-MS-Exchange-CrossTenant-AuthSource: DBBPR08MB10650.eurprd08.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Sep 2026 19:04:59.4518 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 0bc7f26d-0264-416e-a6fc-8352af79c58f X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: qBr/KlFUyYkBlzNRdlPIhUyopGekPzCn4mNvCU6LhQg5tVGYPSDJFkS+y3V+5CQR+wChE0/EQtVA7WmCPcL/7PZRJK6F2gAe3+mamqfvd3w= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR08MB9827 Unlike cpr-exec, cpr-transfer keeps the source QEMU alive: it hands the usbfs FD to the new QEMU over the migration channel and only exits once the target is up. While the handoff is in flight both processes hold the same FD, and only one may reap URB events on it - if both do, one reaps the other's in-flight completions and wedges an isochronous transfer that cannot be retried. So reaping is handed off along with the FD: * The target doesn't reap until it takes over: it stops events right after opening the FD in .realize() and restarts in .post_load(). * The source stops reaping in .pre_save() and, since it stays alive, restarts from the MIG_EVENT_PRECOPY_FAILED notifier on failure. * On success the source's .exit() leaves the device untouched - no reset, interface release or host-driver rebind - and just closes its own copy of the FD. Signed-off-by: Andrey Drobyshev --- hw/usb/host-libusb.c | 78 +++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 74 insertions(+), 4 deletions(-) diff --git a/hw/usb/host-libusb.c b/hw/usb/host-libusb.c index 3c760e9f8f1..59f2b151cd2 100644 --- a/hw/usb/host-libusb.c +++ b/hw/usb/host-libusb.c @@ -110,6 +110,7 @@ struct USBHostDevice { NotifierWithReturn cpr_notifier; GSList *cpr_inflight; bool cpr_drained; + bool cpr_handed_off; libusb_device *dev; libusb_device_handle *dh; struct libusb_device_descriptor ddesc; @@ -1275,10 +1276,17 @@ static void usb_host_exit_notifier(struct Notifier *n, void *data) USBHostDevice *s = container_of(n, USBHostDevice, exit); if (s->dh) { - usb_host_abort_xfers(s); - usb_host_release_interfaces(s); - libusb_reset_device(s->dh); - usb_host_attach_kernel(s); + /* + * Handed to new QEMU across cpr-transfer: do not reset, release + * interfaces or rebind the host driver underneath it; just close + * our handle. + */ + if (!s->cpr_handed_off) { + usb_host_abort_xfers(s); + usb_host_release_interfaces(s); + libusb_reset_device(s->dh); + usb_host_attach_kernel(s); + } libusb_close(s->dh); } } @@ -1310,6 +1318,9 @@ static void usb_host_handle_data(USBDevice *udev, USBPacket *p); * from the packet alone; complete them as errors and let the guest * driver retry. */ +static void usb_host_cpr_stop_events(void); +static void usb_host_cpr_restart_events(void); + static int usb_host_cpr_notifier(NotifierWithReturn *notifier, MigrationEvent *e, Error **errp) { @@ -1322,6 +1333,11 @@ static int usb_host_cpr_notifier(NotifierWithReturn *notifier, return 0; } + if (s->cpr_handed_off) { + usb_host_cpr_restart_events(); + s->cpr_handed_off = false; + } + for (it = s->cpr_inflight; it; it = it->next) { p = it->data; /* Replay from the start of the TD */ @@ -1395,6 +1411,14 @@ static void usb_host_realize(USBDevice *udev, Error **errp) error_setg(errp, "failed to open host usb device %s", s->hostdevice); return; } + if (cpr_is_incoming()) { + /* + * The source still owns the shared fd until it hands off. Don't + * reap URBs on it yet, or we would steal the source's in-flight + * completions; .post_load() restarts events once we take over. + */ + usb_host_cpr_stop_events(); + } } else #endif if (s->match.addr && s->match.bus_num && @@ -1943,6 +1967,7 @@ static int usb_host_post_load(void *opaque, int version_id) * Skip it for CPR. */ if (cpr_is_incoming()) { + usb_host_cpr_restart_events(); return 0; } @@ -1955,6 +1980,47 @@ static int usb_host_post_load(void *opaque, int version_id) return 0; } +#ifndef CONFIG_WIN32 + +/* + * cpr-transfer: the usbfs fd is shared with the new QEMU via SCM_RIGHTS. + * Stop this (source) process from reaping URBs on it, so the new QEMU can + * drive the device without both processes racing on the same fd. Restart + * on migration failure, when the source resumes. + */ +static void usb_host_cpr_stop_events(void) +{ + const struct libusb_pollfd **poll = libusb_get_pollfds(ctx); + + libusb_set_pollfd_notifiers(ctx, NULL, NULL, NULL); + if (poll) { + for (int i = 0; poll[i] != NULL; i++) { + usb_host_del_fd(poll[i]->fd, ctx); + } + free(poll); + } +} + +static void usb_host_cpr_restart_events(void) +{ + const struct libusb_pollfd **poll = libusb_get_pollfds(ctx); + + libusb_set_pollfd_notifiers(ctx, usb_host_add_fd, usb_host_del_fd, ctx); + if (poll) { + for (int i = 0; poll[i] != NULL; i++) { + usb_host_add_fd(poll[i]->fd, poll[i]->events, ctx); + } + free(poll); + } +} + +#else + +static void usb_host_cpr_stop_events(void) {} +static void usb_host_cpr_restart_events(void) {} + +#endif + static int usb_host_pre_save(void *opaque) { USBHostDevice *s = opaque; @@ -1966,6 +2032,10 @@ static int usb_host_pre_save(void *opaque) if (usb_host_cpr_drain_xfers(s) < 0) { return -1; } + if (mode == MIG_MODE_CPR_TRANSFER) { + usb_host_cpr_stop_events(); + s->cpr_handed_off = true; + } } return 0; } -- 2.47.1