From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.openvz.org (unknown [69.168.225.77]) by lore.virtuozzo.com (Postfix) with ESMTPS id 725178015D for ; Tue, 15 Sep 2026 08:17:43 +0000 (UTC) Received: from mail.openvz.org (localhost [127.0.0.1]) by mail.openvz.org (8.14.4/8.14.4) with ESMTP id 68F8GODn003103; Tue, 15 Sep 2026 11:16:25 +0300 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.openvz.org 68F8GODn003103 Authentication-Results: mail.openvz.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=virtuozzo.com header.i=@virtuozzo.com header.b="MEi9jzOS" Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by mail.openvz.org (8.14.4/8.14.4) with ESMTP id 68F8GNCr003099 (version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128 verify=FAIL) for ; Tue, 15 Sep 2026 11:16:23 +0300 DKIM-Filter: OpenDKIM Filter v2.11.0 mail.openvz.org 68F8GNCr003099 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-490a767c7dcso33361775e9.2 for ; Tue, 15 Sep 2026 01:16:23 -0700 (PDT) X-Gm-Message-State: AFuF++li0zssu7aIWfQ5sRAoXf7c991xixrfugnGTc+n9SMOnrdNtr1h Hf9dBuS8G7WIv6iGdMN9BdJP8a8n4qikMe4f+u+qmf5mQIW3zSyT8W6vEK47ArByBqHkQnX4RlC Go1XOH2gfe1ffQk1TxHJHhK2lQT9Kt6SBMybvUfh/OwX5DLlK4s497g== X-Gm-Gg: AYBFou2NCoHUSeo7p7CcW7BF0jVRJcYYX9oEce9ZFaCofuWqvHwGKcfwK+KoAb7CKwL jU/LetoPIqwyK3a6J5fEvPRUN6xpZGc+oEIj8MF418YMNNbRECaQY9W9uXCvXq5P5KgTYMAAvHe Nr/5Z37GqGiSXDL1X0VN7sAMG3axcd3znNkQjjZShgpnnG81LCXQV9vRIxIT3EUV8F19+nQkIqY 7/DwFZ5q0mida10aKvWmbPWG1uD1aQgZgmbR1pKYZPzqTXf1KNpLY+JDq4eZF4b1al2Ptf/mO1Q PehQnsZOhW9zkqjwB/o+5vSlSr+2luTuLCJ7T6VlbCt84CNIidomnAL04rJpcxOw1F5pOFjQ8iZ apnrMFMLmQH7FlIlR X-Received: by 2002:a05:600c:4e94:b0:49c:fc6c:bdfc with SMTP id 5b1f17b1804b1-49e7a66ae6bmr73893855e9.19.1789460182632; Tue, 15 Sep 2026 01:16:22 -0700 (PDT) X-Received: by 2002:a05:600c:4e94:b0:49c:fc6c:bdfc with SMTP id 5b1f17b1804b1-49e7a66ae6bmr73893415e9.19.1789460182085; Tue, 15 Sep 2026 01:16:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1789460182; cv=none; d=google.com; s=arc-20260327; b=fCDUr1U8AMxy4idXOTJ2MnNExsGaZhWtSk1W9P4IEEj+eacTWmekPEKkJQ/jNzEV/E F518JzI24OY/PR+80r5YvanXPa5spEcRjQFprJqc/xTPHECFt2cui/3q/N2xJbPTYLgR 0x1iXPbV5I9d5zF1+VfzKpC2hbQHp4GSf2AvEe7lWb12DkJNvHQcDWug0U8g+4YZM4Ql YQ+JIZyAxkbJEUwkhLgyX2GxiEmwpPBgryJ6AS4GfUjhs6av/4fQyGge6WNH5GJB04b7 Uy+aqS1Jfe+/e05yNQS1TaL6DVjEWStpSjQIQkk9PLj4Q/viAFkX2YJe0HfRW/4WeXlw Xu0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20260327; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:dkim-signature; bh=uyQIXjTPRhVdic9GHo6EYfs96HU92ColiQLz4lW2jPU=; fh=CyEa5k0km+zUBGOSzDGfbole2hTjTZicm1CHqnc8J2I=; b=ov//O0a2N9kfw4Ki4rldXquZ/AYqgPzBLrywIxtVXTxb7x9a9oLpMLcLTDqJdtWSkB sng9Am+4LJrIuYZQxKiJWH4djcZbW6bpTkTdL6SWHLIuNpKffSgTwu2FRCoX+j06zOH1 FoxzM538WS8B3+6moqq3xr0i1BbAfrle+IbcrIT+1NvHXLOxnrKiHW9IqSQ7LV9VrOtG TJ+vvLJ7OfCalQGatp6GXQ5iof5JKsu9sIthB5r75j9JXgI0ThonHs7XtzukFHPgrrke URzK6+lcqtsi31hbuPFVIzreF8u0yKWWbJvj0jEJ6mWfuvMoeSVmOgWgux2ftmMRe6Io bOJA==; dara=google.com ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@virtuozzo.com header.s=relay header.b=MEi9jzOS; spf=pass (google.com: domain of kui.liu@virtuozzo.com designates 130.117.225.111 as permitted sender) smtp.mailfrom=kui.liu@virtuozzo.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=virtuozzo.com Received: from relay.virtuozzo.com (relay.virtuozzo.com. [130.117.225.111]) by mx.google.com with ESMTPS id ffacd0b85a97d-486eb36cc32si22357396f8f.107.2026.09.15.01.16.21 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Sep 2026 01:16:21 -0700 (PDT) Received-SPF: pass (google.com: domain of kui.liu@virtuozzo.com designates 130.117.225.111 as permitted sender) client-ip=130.117.225.111; Authentication-Results: mx.google.com; dkim=pass header.i=@virtuozzo.com header.s=relay header.b=MEi9jzOS; spf=pass (google.com: domain of kui.liu@virtuozzo.com designates 130.117.225.111 as permitted sender) smtp.mailfrom=kui.liu@virtuozzo.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=virtuozzo.com DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=virtuozzo.com; s=relay; h=MIME-Version:Message-ID:Date:Subject:From: Content-Type; bh=uyQIXjTPRhVdic9GHo6EYfs96HU92ColiQLz4lW2jPU=; b=MEi9jzOS5nwY mjqj6ATnEVzwItX4Ys9tdgCQ6qSwn0UeRjkt0uecBwvTJav2PAn+Zjyfh92r5y0ZjL/4L+Qt12mWU 67nRXtv//oM+Ua4CU1sxYXw+ZLVezCuOss29Ph5FcXxlEDXiEPPTkucLa4iU5BcXrteFEwKzHoM8h aDTHFAvaepSaYXftJXHkDPgCqBzkFBeikL1KwfuYSuIdikWZ/tuRHYYxHLVgFO167UzrD5zdleufN NAKzatIHxheZChOHPqdo6iDiH9+LMASeDxwoCmnEphU8MC9FT77jJB5VKveht45ITRN/Yq+m+NE4s HsZm/taGtHLCtOg8s+LJ6Q==; Received: from ch-vpn.virtuozzo.com ([130.117.225.6] helo=localhost.localdomain) by relay.virtuozzo.com with esmtp (Exim 4.96) (envelope-from ) id 1x6OIF-0060ys-2D; Tue, 15 Sep 2026 10:16:11 +0200 From: Liu Kui To: devel@openvz.org Date: Tue, 15 Sep 2026 16:16:12 +0800 Message-ID: <20260915081612.60313-1-kui.liu@virtuozzo.com> X-Mailer: git-send-email 2.50.1 MIME-Version: 1.0 X-OZ-Fwd: true Cc: azaitsev@virtuozzo.com, Liu Kui Subject: [Devel] [PATCH VZ10 v2] fs/fuse kio: fix RPC response drop for large messages sent over RDMA X-BeenThere: devel@openvz.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: OpenVZ development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: devel-bounces@openvz.org Errors-To: devel-bounces@openvz.org When a large message is sent via RDMA READ, it is completed with rio_msg_sent() - which moves the request onto the RPC pending queue where the response lookup can find it - only after both the RDMA_READ_REQ TX completion and the peer's RDMA_READ_ACK have been observed. However these two events are independent and not ordered: the ACK, and even the response to the request, can arrive before the local TX completion is polled. When that happens, the response finds no matching request on the RPC pending queue and is dropped, stalling the request until the RPC timeout logic kicks in. Fix it by completing the message in rio_handle_rx_read_ack() as soon as the RDMA_READ_ACK arrives. Messages in a single RX queue are guaranteed to be delivered in order, so the ACK is always processed before the response, and the request is therefore on the RPC pending queue by the time its response arrives. This is also what the comment in rio_submit() has promised all along. Note that rio_handle_tx() must still complete tx->msg of a TX_SUBMIT_RDMA_READ_ACK tx: it is the incoming msg whose body has just been fetched via RDMA READ, and completing it there is what delivers it to the upper layer. Do it in the case itself with a plain msg->done() - rio_msg_sent() is not suitable for an incoming msg, which was never submitted for send (no PCS_MSG_BUSY to clear, nothing for pcs_msg_sent() to do). The sender side instead clears tx->msg once the msg is completed in rio_handle_rx_read_ack(), so nothing is completed twice. The tx buffer is still recycled only after both events have been seen. A TX send failure is handled by the connection abort logic as before. The same applies to an error status carried in the RDMA_READ_ACK: since the request msg has already been added to the rpc's pending queue, the rio abort triggers an rpc abort, which completes the msg with the same PCS_ERR_NET_ABORT error as before. https://virtuozzo.atlassian.net/browse/VSTOR-141673 Signed-off-by: Liu Kui --- fs/fuse/kio/pcs/pcs_rdma_io.c | 21 ++++++++++----------- 1 file changed, 10 insertions(+), 11 deletions(-) diff --git a/fs/fuse/kio/pcs/pcs_rdma_io.c b/fs/fuse/kio/pcs/pcs_rdma_io.c index 1c464e0e60f3..63fb1e7130db 100644 --- a/fs/fuse/kio/pcs/pcs_rdma_io.c +++ b/fs/fuse/kio/pcs/pcs_rdma_io.c @@ -797,7 +797,6 @@ static inline void rio_kick_write_queue(struct pcs_rdmaio *rio) static void rio_handle_tx(struct pcs_rdmaio *rio, struct rio_tx *tx, int ok) { - struct pcs_msg *msg = tx->msg; u64 xid = tx->xid; /* override remote success if we already aborted */ @@ -808,30 +807,28 @@ static void rio_handle_tx(struct pcs_rdmaio *rio, struct rio_tx *tx, int ok) rio_abort(rio, PCS_ERR_NET_ABORT); switch (tx->tx_state) { - case TX_SUBMIT_RDMA_READ_ACK: + case TX_SUBMIT_RDMA_READ_ACK: { + struct pcs_msg *msg = tx->msg; + rio_put_tx(rio->dev, tx); rio_submit(rio, NULL, SUBMIT_RDMA_READ_ACK, xid, !ok, false); rio->n_rdma_read_ongoing--; + if (!ok) + pcs_set_local_error(&msg->error, PCS_ERR_NET_ABORT); + msg->done(msg); break; + } case TX_WAIT_FOR_TX_COMPL: case TX_WAIT_FOR_READ_ACK: if (++tx->tx_state != TX_MSG_DONE) return; - rio_put_tx(rio->dev, tx); - break; + fallthrough; case TX_MSG_DONE: rio_put_tx(rio->dev, tx); break; default: BUG(); } - - if (msg) { - if (!ok) - pcs_set_local_error(&msg->error, PCS_ERR_NET_ABORT); - - rio_msg_sent(rio, NULL, msg, 1); - } } /* @@ -931,6 +928,8 @@ static int rio_handle_rx_read_ack(struct pcs_rdmaio *rio, list_for_each_entry(tx, &rio->active_txs, list) if (tx->xid == rack->xid) { + rio_msg_sent(rio, NULL, tx->msg, 1); + tx->msg = NULL; list_del(&tx->list); rio_handle_tx(rio, tx, !rack->status); return 0; -- 2.50.1 (Apple Git-155) _______________________________________________ Devel mailing list Devel@openvz.org https://lists.openvz.org/mailman/listinfo/devel