* [Devel] [PATCH hci-8.1 1/8] ms/mptcp: fix retransmission loop when csum is enabled
2026-11-06 3:02 [Devel] [PATCH hci-8.1 0/8] ms/mptcp: stability fixes Eva Kurchatova
@ 2026-11-06 3:02 ` Eva Kurchatova
2026-11-06 3:07 ` no-reply
2026-11-06 3:02 ` [Devel] [PATCH hci-8.1 2/8] ms/mptcp: move the retrans loop to a separate helper Eva Kurchatova
` (6 subsequent siblings)
7 siblings, 1 reply; 17+ messages in thread
From: Eva Kurchatova @ 2026-11-06 3:02 UTC (permalink / raw)
To: khorenko; +Cc: devel
From: Paolo Abeni <pabeni@redhat.com>
Sashiko noted that retransmission with csum enabled can actually
transmit new data, but currently the relevant code does not update
accordingly snd_nxt.
The may cause incoming ack drop and an endless retransmission loop.
Address the issue incrementing snd_nxt as needed.
Fixes: 4e14867d5e91 ("mptcp: tune re-injections for csum enabled mode")
Cc: stable@vger.kernel.org
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Reviewed-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
Link: https://patch.msgid.link/20260602-net-mptcp-misc-fixes-7-1-rc7-v2-2-856831229976@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
(cherry picked from commit d1918b36edcaed0ec4ef6888b2358c6b1ddcff47)
https://virtuozzo.atlassian.net/browse/VSTOR-153491
Feature: fix ms/net
Signed-off-by: Eva Kurchatova <eva.kurchatova@virtuozzo.com>
---
net/mptcp/protocol.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index e526648a583e..9f1cc648a272 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -2666,6 +2666,10 @@ static void __mptcp_retrans(struct sock *sk)
msk->bytes_retrans += len;
dfrag->already_sent = max(dfrag->already_sent, len);
+ /* With csum enabled retransmission can send new data. */
+ if (after64(dfrag->already_sent + dfrag->data_seq, msk->snd_nxt))
+ WRITE_ONCE(msk->snd_nxt, dfrag->already_sent + dfrag->data_seq);
+
reset_timer:
mptcp_check_and_set_pending(sk);
--
2.56.0
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 17+ messages in thread* Re: [Devel] [PATCH hci-8.1 1/8] ms/mptcp: fix retransmission loop when csum is enabled
2026-11-06 3:02 ` [Devel] [PATCH hci-8.1 1/8] ms/mptcp: fix retransmission loop when csum is enabled Eva Kurchatova
@ 2026-11-06 3:07 ` no-reply
0 siblings, 0 replies; 17+ messages in thread
From: no-reply @ 2026-11-06 3:07 UTC (permalink / raw)
To: Eva Kurchatova, khorenko; +Cc: devel
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] mptcp: __mptcp_retrans() advances snd_nxt without updating bytes_sent
--
commit 4d64a3ef806956ed9e7612432b9f24e9542dcd61
Author: Paolo Abeni <pabeni@redhat.com>
ms/mptcp: fix retransmission loop when csum is enabled
This commit raises msk->snd_nxt in __mptcp_retrans() when a retransmission
with csum enabled sends data past the old dfrag->already_sent. The goal is
to stop incoming ACKs from being dropped, which caused an endless
retransmission loop.
Link: https://patch.msgid.link/20260602-net-mptcp-misc-fixes-7-1-rc7-v2-2-856831229976@kernel.org
> diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
> --- a/net/mptcp/protocol.c
> +++ b/net/mptcp/protocol.c
[ ... ]
> @@ -2666,6 +2666,10 @@ static void __mptcp_retrans(struct sock *sk)
> msk->bytes_retrans += len;
> dfrag->already_sent = max(dfrag->already_sent, len);
>
> + /* With csum enabled retransmission can send new data. */
> + if (after64(dfrag->already_sent + dfrag->data_seq, msk->snd_nxt))
> + WRITE_ONCE(msk->snd_nxt, dfrag->already_sent + dfrag->data_seq);
> +
[Severity: Medium]
Should msk->bytes_sent be updated here as well?
When csum is enabled, __mptcp_retrans() sets info.limit to dfrag->data_len,
so the loop can send bytes beyond the old dfrag->already_sent. Those bytes
are new data.
This hunk advances msk->snd_nxt for them, but it does not do the matching
accounting that mptcp_update_post_push() does on the normal push path:
mptcp_update_post_push() {
...
msk->bytes_sent += snd_nxt_new - msk->snd_nxt;
WRITE_ONCE(msk->snd_nxt, snd_nxt_new);
...
}
As a result, msk->bytes_sent, which is reported through MPTCP_INFO,
under-reports the new data sent from __mptcp_retrans().
The earlier msk->bytes_retrans += len in the same function also counts those
new bytes as retransmitted, so bytes_retrans appears to over-report.
Is msk->snd_burst also expected to be reduced for this new data, as
mptcp_update_post_push() does on the normal push path?
> reset_timer:
> mptcp_check_and_set_pending(sk);
>
--
Sashiko AI review · https://sashiko.vzint.dev/#/patchset/20261106030306.16014-1-eva.kurchatova@virtuozzo.com?part=1
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 17+ messages in thread
* [Devel] [PATCH hci-8.1 2/8] ms/mptcp: move the retrans loop to a separate helper
2026-11-06 3:02 [Devel] [PATCH hci-8.1 0/8] ms/mptcp: stability fixes Eva Kurchatova
2026-11-06 3:02 ` [Devel] [PATCH hci-8.1 1/8] ms/mptcp: fix retransmission loop when csum is enabled Eva Kurchatova
@ 2026-11-06 3:02 ` Eva Kurchatova
2026-11-06 3:06 ` no-reply
2026-11-06 3:02 ` [Devel] [PATCH hci-8.1 3/8] ms/mptcp: move the stale logic out of retrans scheduler Eva Kurchatova
` (5 subsequent siblings)
7 siblings, 1 reply; 17+ messages in thread
From: Eva Kurchatova @ 2026-11-06 3:02 UTC (permalink / raw)
To: khorenko; +Cc: devel
From: Paolo Abeni <pabeni@redhat.com>
This is a cleanup in order to make the next patch simpler.
No functional change intended.
Tested-by: Gang Yan <yangang@kylinos.cn>
Tested-by: Geliang Tang <geliang@kernel.org>
Acked-by: Geliang Tang <geliang@kernel.org>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
Link: https://patch.msgid.link/20260807-net-next-mptcp-oooq-pruning-v3-1-dbc1eb853cc3@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
(cherry picked from commit 789e6a844b51bf6362a7e7f551553ba988f3dc92)
Rebase note: this tree increments icsk_retransmits without the
WRITE_ONCE() that mainline added later, and the moved code keeps the
plain increment.
https://virtuozzo.atlassian.net/browse/VSTOR-153491
Feature: fix ms/net
Signed-off-by: Eva Kurchatova <eva.kurchatova@virtuozzo.com>
---
net/mptcp/protocol.c | 72 ++++++++++++++++++++++++++------------------
1 file changed, 42 insertions(+), 30 deletions(-)
diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index 9f1cc648a272..8c17857bbbcf 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -2578,40 +2578,14 @@ static void mptcp_check_fastclose(struct mptcp_sock *msk)
sk_error_report(sk);
}
-static void __mptcp_retrans(struct sock *sk)
+/* Retransmit the specified data fragment on all the selected subflows. */
+static int __mptcp_push_retrans(struct sock *sk, struct mptcp_data_frag *dfrag)
{
struct mptcp_sendmsg_info info = { .data_lock_held = true, };
struct mptcp_sock *msk = mptcp_sk(sk);
struct mptcp_subflow_context *subflow;
- struct mptcp_data_frag *dfrag;
struct sock *ssk;
- int ret, err;
- u16 len = 0;
-
- mptcp_clean_una_wakeup(sk);
-
- /* first check ssk: need to kick "stale" logic */
- err = mptcp_sched_get_retrans(msk);
- dfrag = mptcp_rtx_head(sk);
- if (!dfrag) {
- if (mptcp_data_fin_enabled(msk)) {
- struct inet_connection_sock *icsk = inet_csk(sk);
-
- icsk->icsk_retransmits++;
- mptcp_set_datafin_timeout(sk);
- mptcp_send_ack(msk);
-
- goto reset_timer;
- }
-
- if (!mptcp_send_head(sk))
- goto clear_scheduled;
-
- goto reset_timer;
- }
-
- if (err)
- goto reset_timer;
+ int ret, len = 0;
mptcp_for_each_subflow(msk, subflow) {
if (READ_ONCE(subflow->scheduled)) {
@@ -2639,7 +2613,7 @@ static void __mptcp_retrans(struct sock *sk)
!msk->allow_subflows) {
spin_unlock_bh(&msk->fallback_lock);
release_sock(ssk);
- goto clear_scheduled;
+ return -1;
}
while (info.sent < info.limit) {
@@ -2662,6 +2636,44 @@ static void __mptcp_retrans(struct sock *sk)
release_sock(ssk);
}
}
+ return len;
+}
+
+static void __mptcp_retrans(struct sock *sk)
+{
+ struct mptcp_sock *msk = mptcp_sk(sk);
+ struct mptcp_subflow_context *subflow;
+ struct mptcp_data_frag *dfrag;
+ int err, len;
+
+ mptcp_clean_una_wakeup(sk);
+
+ /* first check ssk: need to kick "stale" logic */
+ err = mptcp_sched_get_retrans(msk);
+ dfrag = mptcp_rtx_head(sk);
+ if (!dfrag) {
+ if (mptcp_data_fin_enabled(msk)) {
+ struct inet_connection_sock *icsk = inet_csk(sk);
+
+ icsk->icsk_retransmits++;
+ mptcp_set_datafin_timeout(sk);
+ mptcp_send_ack(msk);
+
+ goto reset_timer;
+ }
+
+ if (!mptcp_send_head(sk))
+ goto clear_scheduled;
+
+ goto reset_timer;
+ }
+
+ if (err)
+ goto reset_timer;
+
+ len = __mptcp_push_retrans(sk, dfrag);
+ if (len < 0)
+ goto clear_scheduled;
msk->bytes_retrans += len;
dfrag->already_sent = max(dfrag->already_sent, len);
--
2.56.0
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 17+ messages in thread* [Devel] [PATCH hci-8.1 3/8] ms/mptcp: move the stale logic out of retrans scheduler
2026-11-06 3:02 [Devel] [PATCH hci-8.1 0/8] ms/mptcp: stability fixes Eva Kurchatova
2026-11-06 3:02 ` [Devel] [PATCH hci-8.1 1/8] ms/mptcp: fix retransmission loop when csum is enabled Eva Kurchatova
2026-11-06 3:02 ` [Devel] [PATCH hci-8.1 2/8] ms/mptcp: move the retrans loop to a separate helper Eva Kurchatova
@ 2026-11-06 3:02 ` Eva Kurchatova
2026-11-06 3:07 ` no-reply
2026-11-06 3:02 ` [Devel] [PATCH hci-8.1 4/8] ms/mptcp: let the retrans scheduler do its job Eva Kurchatova
` (4 subsequent siblings)
7 siblings, 1 reply; 17+ messages in thread
From: Eva Kurchatova @ 2026-11-06 3:02 UTC (permalink / raw)
To: khorenko; +Cc: devel
From: Paolo Abeni <pabeni@redhat.com>
This allow separating the stale logic invocation and the retrans
scheduler, and will simplify the next patch.
It's also a cleaner design as the retrans scheduler has currently
too many side effects. As a possible downside, the retrans work will
now traverse the subflows list additional times; that does not matter
much, as this is slowpath.
While at it, pick more accurate names for the involved helpers and
explicitly note that the per subflow stale data is under msk socket
lock protection.
The scheduler and the stale logic may observe different subflow
statues, as no subflow lock is acquired. This is intentional and not
harmful, worst case leading to slower retransmissions.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Reviewed-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
Link: https://patch.msgid.link/20260807-net-next-mptcp-oooq-pruning-v3-2-dbc1eb853cc3@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
(cherry picked from commit 6cafe51e0f98fe60a106783d30b2f4c4b6039f4c)
Rebase note: this tree has no "closing" subflow flag, so only the
"stale" bit leaves the bitfield, and __unused grows from 9 to 10.
https://virtuozzo.atlassian.net/browse/VSTOR-153491
Feature: fix ms/net
Signed-off-by: Eva Kurchatova <eva.kurchatova@virtuozzo.com>
---
net/mptcp/pm.c | 43 ++++++++++++++++++++++++++++---------------
net/mptcp/protocol.c | 4 ++--
net/mptcp/protocol.h | 11 +++++++----
3 files changed, 37 insertions(+), 21 deletions(-)
diff --git a/net/mptcp/pm.c b/net/mptcp/pm.c
index a330e74bf5c1..05775d0b6493 100644
--- a/net/mptcp/pm.c
+++ b/net/mptcp/pm.c
@@ -908,7 +908,8 @@ bool mptcp_pm_is_backup(struct mptcp_sock *msk, struct sock_common *skc)
return mptcp_pm_nl_is_backup(msk, &skc_local);
}
-static void mptcp_pm_subflows_chk_stale(const struct mptcp_sock *msk, struct sock *ssk)
+static void
+mptcp_pm_subflow_chk_stale(const struct mptcp_sock *msk, struct sock *ssk)
{
struct mptcp_subflow_context *iter, *subflow = mptcp_subflow_ctx(ssk);
struct sock *sk = (struct sock *)msk;
@@ -945,22 +946,34 @@ static void mptcp_pm_subflows_chk_stale(const struct mptcp_sock *msk, struct soc
}
}
-void mptcp_pm_subflow_chk_stale(const struct mptcp_sock *msk, struct sock *ssk)
+void mptcp_pm_chk_stale(const struct mptcp_sock *msk)
{
- struct mptcp_subflow_context *subflow = mptcp_subflow_ctx(ssk);
- u32 rcv_tstamp = READ_ONCE(tcp_sk(ssk)->rcv_tstamp);
-
- /* keep track of rtx periods with no progress */
- if (!subflow->stale_count) {
- subflow->stale_rcv_tstamp = rcv_tstamp;
- subflow->stale_count++;
- } else if (subflow->stale_rcv_tstamp == rcv_tstamp) {
- if (subflow->stale_count < U8_MAX)
+ struct mptcp_subflow_context *subflow;
+
+ mptcp_for_each_subflow(msk, subflow) {
+ struct sock *ssk = mptcp_subflow_tcp_sock(subflow);
+ u32 rcv_tstamp;
+
+ if (!__mptcp_subflow_active(subflow))
+ continue;
+
+ /* No data outstanding at TCP level? not stale */
+ if (tcp_rtx_and_write_queues_empty(ssk))
+ continue;
+
+ /* keep track of rtx periods with no progress */
+ rcv_tstamp = READ_ONCE(tcp_sk(ssk)->rcv_tstamp);
+ if (!subflow->stale_count) {
+ subflow->stale_rcv_tstamp = rcv_tstamp;
subflow->stale_count++;
- mptcp_pm_subflows_chk_stale(msk, ssk);
- } else {
- subflow->stale_count = 0;
- mptcp_subflow_set_active(subflow);
+ } else if (subflow->stale_rcv_tstamp == rcv_tstamp) {
+ if (subflow->stale_count < U8_MAX)
+ subflow->stale_count++;
+ mptcp_pm_subflow_chk_stale(msk, ssk);
+ } else {
+ subflow->stale_count = 0;
+ mptcp_subflow_set_active(subflow);
+ }
}
}
diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index 8c17857bbbcf..838b18bc19c9 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -2291,7 +2291,6 @@ struct sock *mptcp_subflow_get_retrans(struct mptcp_sock *msk)
/* still data outstanding at TCP level? skip this */
if (!tcp_rtx_and_write_queues_empty(ssk)) {
- mptcp_pm_subflow_chk_stale(msk, ssk);
min_stale_count = min_t(int, min_stale_count, subflow->stale_count);
continue;
}
@@ -2646,9 +2645,10 @@ static void __mptcp_retrans(struct sock *sk)
struct mptcp_data_frag *dfrag;
int err, len;
+ mptcp_pm_chk_stale(msk);
+
mptcp_clean_una_wakeup(sk);
- /* first check ssk: need to kick "stale" logic */
err = mptcp_sched_get_retrans(msk);
dfrag = mptcp_rtx_head(sk);
if (!dfrag) {
diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h
index 22804ae2e85a..efddbb1920e6 100644
--- a/net/mptcp/protocol.h
+++ b/net/mptcp/protocol.h
@@ -530,12 +530,11 @@ struct mptcp_subflow_context {
send_infinite_map : 1,
remote_key_valid : 1, /* received the peer key from */
disposable : 1, /* ctx can be free at ulp release time */
- stale : 1, /* unable to snd/rcv data, do not use for xmit */
valid_csum_seen : 1, /* at least one csum validated */
is_mptfo : 1, /* subflow is doing TFO */
close_event_done : 1, /* has done the post-closed part */
mpc_drop : 1, /* the MPC option has been dropped in a rtx */
- __unused : 9;
+ __unused : 10;
bool data_avail;
bool scheduled;
bool pm_listener; /* a listener managed by the kernel PM? */
@@ -553,7 +552,11 @@ struct mptcp_subflow_context {
u8 reset_seen:1;
u8 reset_transient:1;
u8 reset_reason:4;
- u8 stale_count;
+ u8 stale_count; /* Protected by the msk socket lock */
+ u8 stale; /* Protected by the msk socket lock,
+ * if set the subflow is unable to snd/rcv
+ * data, the schedule should skip it
+ */
u32 subflow_id;
@@ -1004,7 +1007,7 @@ int mptcp_pm_parse_entry(struct nlattr *attr, struct genl_info *info,
bool mptcp_pm_addr_families_match(const struct sock *sk,
const struct mptcp_addr_info *loc,
const struct mptcp_addr_info *rem);
-void mptcp_pm_subflow_chk_stale(const struct mptcp_sock *msk, struct sock *ssk);
+void mptcp_pm_chk_stale(const struct mptcp_sock *msk);
void mptcp_pm_new_connection(struct mptcp_sock *msk, const struct sock *ssk, int server_side);
void mptcp_pm_fully_established(struct mptcp_sock *msk, const struct sock *ssk);
bool mptcp_pm_allow_new_subflow(struct mptcp_sock *msk);
--
2.56.0
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 17+ messages in thread* [Devel] [PATCH hci-8.1 4/8] ms/mptcp: let the retrans scheduler do its job
2026-11-06 3:02 [Devel] [PATCH hci-8.1 0/8] ms/mptcp: stability fixes Eva Kurchatova
` (2 preceding siblings ...)
2026-11-06 3:02 ` [Devel] [PATCH hci-8.1 3/8] ms/mptcp: move the stale logic out of retrans scheduler Eva Kurchatova
@ 2026-11-06 3:02 ` Eva Kurchatova
2026-11-06 3:10 ` no-reply
2026-11-06 3:02 ` [Devel] [PATCH hci-8.1 5/8] ms/mptcp: explicitly drop over memory limits Eva Kurchatova
` (3 subsequent siblings)
7 siblings, 1 reply; 17+ messages in thread
From: Eva Kurchatova @ 2026-11-06 3:02 UTC (permalink / raw)
To: khorenko; +Cc: devel
From: Paolo Abeni <pabeni@redhat.com>
Currently the MPTCP core enforces that when MPTCP-level retrans timer
fires, at most a single dfrag is retransmitted. In some corner-cases, it
may be necessary to retransmit multiple dfrags, and the MPTCP socket
will need to wait multiple retrans timeout to accomplish that.
Remove the mentioned constraint, allowing to transmit multiple dfrags
per retrans period, as long as the scheduler keeps selecting subflows
for retransmissions and pending data is available in the rtx queue.
The default scheduler will transmit a dfrag per available subflow.
Tested-by: Gang Yan <yangang@kylinos.cn>
Tested-by: Geliang Tang <geliang@kernel.org>
Acked-by: Geliang Tang <geliang@kernel.org>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
Link: https://patch.msgid.link/20260807-net-next-mptcp-oooq-pruning-v3-3-dbc1eb853cc3@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
(cherry picked from commit 96d846e3e2a7ea01ff8a584a0b5e2a42fa9ccc1e)
https://virtuozzo.atlassian.net/browse/VSTOR-153491
Feature: fix ms/net
Signed-off-by: Eva Kurchatova <eva.kurchatova@virtuozzo.com>
---
net/mptcp/protocol.c | 106 ++++++++++++++++++++++++++++++-------------
1 file changed, 75 insertions(+), 31 deletions(-)
diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index 838b18bc19c9..2dcce47a1ef7 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -1000,13 +1000,6 @@ static void __mptcp_clean_una_wakeup(struct sock *sk)
mptcp_write_space(sk);
}
-static void mptcp_clean_una_wakeup(struct sock *sk)
-{
- mptcp_data_lock(sk);
- __mptcp_clean_una_wakeup(sk);
- mptcp_data_unlock(sk);
-}
-
static void mptcp_enter_memory_pressure(struct sock *sk)
{
struct mptcp_subflow_context *subflow;
@@ -2577,8 +2570,12 @@ static void mptcp_check_fastclose(struct mptcp_sock *msk)
sk_error_report(sk);
}
-/* Retransmit the specified data fragment on all the selected subflows. */
-static int __mptcp_push_retrans(struct sock *sk, struct mptcp_data_frag *dfrag)
+/*
+ * Retransmit the specified data fragment on all the selected subflows,
+ * starting from the specified sequence
+ */
+static int __mptcp_push_retrans(struct sock *sk, struct mptcp_data_frag *dfrag,
+ u64 sent_seq)
{
struct mptcp_sendmsg_info info = { .data_lock_held = true, };
struct mptcp_sock *msk = mptcp_sk(sk);
@@ -2588,6 +2585,7 @@ static int __mptcp_push_retrans(struct sock *sk, struct mptcp_data_frag *dfrag)
mptcp_for_each_subflow(msk, subflow) {
if (READ_ONCE(subflow->scheduled)) {
+ u16 offset = sent_seq - dfrag->data_seq;
u16 copied = 0;
mptcp_subflow_set_scheduled(subflow, false);
@@ -2597,7 +2595,7 @@ static int __mptcp_push_retrans(struct sock *sk, struct mptcp_data_frag *dfrag)
lock_sock(ssk);
/* limit retransmission to the bytes already sent on some subflows */
- info.sent = 0;
+ info.sent = offset;
info.limit = READ_ONCE(msk->csum_enabled) ? dfrag->data_len :
dfrag->already_sent;
@@ -2643,45 +2641,91 @@ static void __mptcp_retrans(struct sock *sk)
struct mptcp_sock *msk = mptcp_sk(sk);
struct mptcp_subflow_context *subflow;
struct mptcp_data_frag *dfrag;
+ u64 retrans_seq, sent_seq;
+ bool need_retrans;
int err, len;
mptcp_pm_chk_stale(msk);
- mptcp_clean_una_wakeup(sk);
-
- err = mptcp_sched_get_retrans(msk);
+ /* Get an updated and consistent rtx queue status. */
+ mptcp_data_lock(sk);
+ __mptcp_clean_una_wakeup(sk);
+ retrans_seq = msk->snd_una;
dfrag = mptcp_rtx_head(sk);
- if (!dfrag) {
+ need_retrans = !!dfrag;
+ mptcp_data_unlock(sk);
+
+ for (;;) {
+ bool already_acked;
+
+ err = mptcp_sched_get_retrans(msk);
+ if (err)
+ break;
+
+ /* `already_sent` can be 0 for `dfrag` belonging to the RTX
+ * queue due to __mptcp_retransmit_pending_data().
+ */
+ if (!dfrag || !dfrag->already_sent)
+ break;
+
+ /* Can fail only in case of fallback. */
+ len = __mptcp_push_retrans(sk, dfrag, retrans_seq);
+ if (len < 0)
+ goto clear_scheduled;
+
+ retrans_seq += len;
+ msk->bytes_retrans += len;
+ dfrag->already_sent = max_t(u16, dfrag->already_sent,
+ retrans_seq - dfrag->data_seq);
+
+ /* With csum enabled, retransmission can send new data. */
+ sent_seq = dfrag->already_sent + dfrag->data_seq;
+ if (after64(sent_seq, msk->snd_nxt))
+ WRITE_ONCE(msk->snd_nxt, sent_seq);
+
+ /* Attempt the next fragment only if the current one is
+ * completely retransmitted.
+ */
+ if (before64(retrans_seq, dfrag->data_seq + dfrag->data_len))
+ break;
+
+ dfrag = list_is_last(&dfrag->list, &msk->rtx_queue) ?
+ NULL : list_next_entry(dfrag, list);
+ if (!dfrag)
+ break;
+
+ /* Incoming acks can move snd_una after the current dfrag
+ * across loop iterations, if so start again from RTX head.
+ */
+ mptcp_data_lock(sk);
+ already_acked = !before64(msk->snd_una, dfrag->data_seq +
+ dfrag->already_sent);
+ if (already_acked) {
+ __mptcp_clean_una_wakeup(sk);
+ retrans_seq = msk->snd_una;
+ dfrag = mptcp_rtx_head(sk);
+ need_retrans = !!dfrag;
+ } else if (after64(msk->snd_una, retrans_seq)) {
+ retrans_seq = msk->snd_una;
+ }
+ mptcp_data_unlock(sk);
+ }
+
+ /* Attempt data-fin retransmission only when the RTX queue is empty. */
+ if (!need_retrans) {
if (mptcp_data_fin_enabled(msk)) {
struct inet_connection_sock *icsk = inet_csk(sk);
icsk->icsk_retransmits++;
mptcp_set_datafin_timeout(sk);
mptcp_send_ack(msk);
-
goto reset_timer;
}
if (!mptcp_send_head(sk))
goto clear_scheduled;
-
- goto reset_timer;
}
- if (err)
- goto reset_timer;
-
- len = __mptcp_push_retrans(sk, dfrag);
- if (len < 0)
- goto clear_scheduled;
-
- msk->bytes_retrans += len;
- dfrag->already_sent = max(dfrag->already_sent, len);
-
- /* With csum enabled retransmission can send new data. */
- if (after64(dfrag->already_sent + dfrag->data_seq, msk->snd_nxt))
- WRITE_ONCE(msk->snd_nxt, dfrag->already_sent + dfrag->data_seq);
-
reset_timer:
mptcp_check_and_set_pending(sk);
--
2.56.0
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 17+ messages in thread* [Devel] [PATCH hci-8.1 5/8] ms/mptcp: explicitly drop over memory limits
2026-11-06 3:02 [Devel] [PATCH hci-8.1 0/8] ms/mptcp: stability fixes Eva Kurchatova
` (3 preceding siblings ...)
2026-11-06 3:02 ` [Devel] [PATCH hci-8.1 4/8] ms/mptcp: let the retrans scheduler do its job Eva Kurchatova
@ 2026-11-06 3:02 ` Eva Kurchatova
2026-11-06 3:10 ` no-reply
2026-11-06 3:02 ` [Devel] [PATCH hci-8.1 6/8] ms/mptcp: implemented OoO queue pruning Eva Kurchatova
` (2 subsequent siblings)
7 siblings, 1 reply; 17+ messages in thread
From: Eva Kurchatova @ 2026-11-06 3:02 UTC (permalink / raw)
To: khorenko; +Cc: devel
From: Paolo Abeni <pabeni@redhat.com>
Currently the enforcement of the rcvbuf constraint is implemented
when moving the skbs into the msk receive or OoO queue, keeping the
incoming skbs in the subflow queue when over limits.
Under significant memory pressure the above can cause permanent data
transfer stalls, as the skb needed to make forward progress can be
stuck in a subflow queue.
Over memory limits, drop the incoming skb, relying on MPTCP-level
retransmissions.
Note that fallback socket must perform the limit before the skb reaches
the subflow-level queue, as dropping an in-sequence already acked skb
would break the stream.
This is not a complete fix for the stall issue, as the drop strategy
needs refinements that will come in the next patches.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Reviewed-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
Link: https://patch.msgid.link/20260807-net-next-mptcp-oooq-pruning-v3-4-dbc1eb853cc3@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
(cherry picked from commit e0e4d56b050597a690b4dd1c281af532469b0636)
Rebase note: this tree has no MPTCP-level backlog, its receive path
moves the skbs from the subflow queue straight to the msk. The drop goes
into __mptcp_move_skb(), and the loops that move the skbs stop over the
limits only for a fallback socket, which keeps them in the subflow queue
as before. The options.c check that drops them for a fallback socket
before that queue, and the BacklogDrop counter, are left out; RcvPruned
is already there.
https://virtuozzo.atlassian.net/browse/VSTOR-153491
Feature: fix ms/net
Signed-off-by: Eva Kurchatova <eva.kurchatova@virtuozzo.com>
---
net/mptcp/protocol.c | 20 +++++++++++++++-----
1 file changed, 15 insertions(+), 5 deletions(-)
diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index 2dcce47a1ef7..6e8779646d17 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -286,6 +286,15 @@ static bool __mptcp_move_skb(struct mptcp_sock *msk, struct sock *ssk,
skb_ext_reset(skb);
skb_orphan(skb);
+ /* Can't drop packets for fallback socket this late, or the stream
+ * will break.
+ */
+ if (unlikely(sk_rmem_alloc_get(sk) > READ_ONCE(sk->sk_rcvbuf)) &&
+ !__mptcp_check_fallback(msk)) {
+ MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_RCVPRUNED);
+ goto drop;
+ }
+
/* try to fetch required memory from subflow */
if (!sk_rmem_schedule(sk, skb, skb->truesize)) {
MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_RCVPRUNED);
@@ -599,7 +608,8 @@ static bool __mptcp_move_skbs_from_subflow(struct mptcp_sock *msk,
struct sk_buff *skb;
bool fin;
- if (sk_rmem_alloc_get(sk) > sk->sk_rcvbuf)
+ if (__mptcp_check_fallback(msk) &&
+ sk_rmem_alloc_get(sk) > sk->sk_rcvbuf)
break;
/* try to move as much data as available */
@@ -2078,11 +2088,11 @@ static bool __mptcp_move_skbs(struct sock *sk)
struct sock *ssk;
bool slowpath;
- /*
- * As an optimization avoid traversing the subflows list
- * and ev. acquiring the subflow socket lock before baling out
+ /* Over the limits a fallback socket leaves the data in the
+ * subflows, don't loop on them
*/
- if (sk_rmem_alloc_get(sk) > sk->sk_rcvbuf)
+ if (__mptcp_check_fallback(msk) &&
+ sk_rmem_alloc_get(sk) > sk->sk_rcvbuf)
break;
subflow = __mptcp_first_ready_from(msk, subflow);
--
2.56.0
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 17+ messages in thread* [Devel] [PATCH hci-8.1 6/8] ms/mptcp: implemented OoO queue pruning
2026-11-06 3:02 [Devel] [PATCH hci-8.1 0/8] ms/mptcp: stability fixes Eva Kurchatova
` (4 preceding siblings ...)
2026-11-06 3:02 ` [Devel] [PATCH hci-8.1 5/8] ms/mptcp: explicitly drop over memory limits Eva Kurchatova
@ 2026-11-06 3:02 ` Eva Kurchatova
2026-11-06 3:11 ` no-reply
2026-11-06 3:02 ` [Devel] [PATCH hci-8.1 7/8] ms/mptcp: being below memory limit is a likely() condition Eva Kurchatova
2026-11-06 3:02 ` [Devel] [PATCH hci-8.1 8/8] ms/mptcp: avoid pruning for OoW data Eva Kurchatova
7 siblings, 1 reply; 17+ messages in thread
From: Eva Kurchatova @ 2026-11-06 3:02 UTC (permalink / raw)
To: khorenko; +Cc: devel
From: Paolo Abeni <pabeni@redhat.com>
When moving incoming skbs in the msk receive queue and the latter
is above limits, prune it as needed quite alike what TCP is doing
at the subflow level. The main difference relies in the stop condition:
since MPTCP does not perform collapsing, it's better off dropping the
bare minimum to fit the (newer) incoming packet.
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Tested-by: Gang Yan <yangang@kylinos.cn>
Reviewed-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
Link: https://patch.msgid.link/20260807-net-next-mptcp-oooq-pruning-v3-7-dbc1eb853cc3@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
(cherry picked from commit e468d371180d3c5b3333660bd742103b88703adf)
Rebase note: in the older receive path of this tree the check goes
into __mptcp_move_skb() and mptcp_data_queue_ofo(), after the map_seq
of the skb is set, and replaces the drop of the previous patch, as in
mainline. The commits of the series between the two, about the backlog
this tree lacks and a cleanup, are not needed. OFOPruned is added after
the last counter of this tree.
https://virtuozzo.atlassian.net/browse/VSTOR-153491
Feature: fix ms/net
Signed-off-by: Eva Kurchatova <eva.kurchatova@virtuozzo.com>
---
net/mptcp/mib.c | 1 +
net/mptcp/mib.h | 1 +
net/mptcp/protocol.c | 79 +++++++++++++++++++++++++++++++++++++++-----
3 files changed, 72 insertions(+), 9 deletions(-)
diff --git a/net/mptcp/mib.c b/net/mptcp/mib.c
index 19eb9292bd60..fbe27ddea8e1 100644
--- a/net/mptcp/mib.c
+++ b/net/mptcp/mib.c
@@ -79,6 +79,7 @@ static const struct snmp_mib mptcp_snmp_list[] = {
SNMP_MIB_ITEM("RcvWndConflict", MPTCP_MIB_RCVWNDCONFLICT),
SNMP_MIB_ITEM("MPCurrEstab", MPTCP_MIB_CURRESTAB),
SNMP_MIB_ITEM("Blackhole", MPTCP_MIB_BLACKHOLE),
+ SNMP_MIB_ITEM("OFOPruned", MPTCP_MIB_OFOPRUNED),
SNMP_MIB_SENTINEL
};
diff --git a/net/mptcp/mib.h b/net/mptcp/mib.h
index 128282982843..f2b9934a6ecb 100644
--- a/net/mptcp/mib.h
+++ b/net/mptcp/mib.h
@@ -80,6 +80,7 @@ enum linux_mptcp_mib_field {
MPTCP_MIB_RCVWNDCONFLICT, /* Conflict with while updating msk rcv wnd */
MPTCP_MIB_CURRESTAB, /* Current established MPTCP connections */
MPTCP_MIB_BLACKHOLE, /* A blackhole has been detected */
+ MPTCP_MIB_OFOPRUNED, /* MPTCP-level OoO queue pruned */
__MPTCP_MIB_MAX
};
diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index 6e8779646d17..3b02a63ad1a9 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -157,6 +157,65 @@ static bool mptcp_ooo_try_coalesce(struct mptcp_sock *msk, struct sk_buff *to,
return mptcp_try_coalesce((struct sock *)msk, to, from);
}
+/* "Inspired" from the TCP version; main difference: stop as soon as the MPTCP
+ * socket is under memory limit.
+ */
+static void mptcp_prune_ofo_queue(struct sock *sk,
+ const struct sk_buff *in_skb)
+{
+ struct mptcp_sock *msk = mptcp_sk(sk);
+ struct rb_node *node, *prev;
+ bool pruned = false;
+ u64 mem;
+
+ if (RB_EMPTY_ROOT(&msk->out_of_order_queue))
+ return;
+
+ node = &msk->ooo_last_skb->rbnode;
+
+ do {
+ struct sk_buff *skb = rb_to_skb(node);
+
+ /* Stop pruning if the incoming skb would land in OoO tail. */
+ if (after64(MPTCP_SKB_CB(in_skb)->map_seq,
+ MPTCP_SKB_CB(skb)->map_seq))
+ break;
+
+ pruned = true;
+ prev = rb_prev(node);
+ rb_erase(node, &msk->out_of_order_queue);
+ mptcp_drop(sk, skb);
+ msk->ooo_last_skb = rb_to_skb(prev);
+
+ mem = (unsigned int)sk_rmem_alloc_get(sk);
+ if (mem <= sk->sk_rcvbuf)
+ break;
+
+ node = prev;
+ } while (node);
+
+ if (pruned)
+ MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_OFOPRUNED);
+}
+
+/* The stack can't drop packets for fallback socket at the msk level, or the
+ * stream will break.
+ */
+static bool mptcp_can_ingest(const struct sock *sk)
+{
+ return unlikely(sk_rmem_alloc_get(sk) <= READ_ONCE(sk->sk_rcvbuf)) ||
+ __mptcp_check_fallback(mptcp_sk(sk));
+}
+
+static bool mptcp_try_rmem_schedule(struct sock *sk, const struct sk_buff *skb)
+{
+ if (!mptcp_can_ingest(sk)) {
+ mptcp_prune_ofo_queue(sk, skb);
+ return mptcp_can_ingest(sk);
+ }
+ return true;
+}
+
/* "inspired" by tcp_data_queue_ofo(), main differences:
* - use mptcp seqs
* - don't cope with sacks
@@ -168,6 +227,12 @@ static void mptcp_data_queue_ofo(struct mptcp_sock *msk, struct sk_buff *skb)
u64 seq, end_seq, max_seq;
struct sk_buff *skb1;
+ if (!mptcp_try_rmem_schedule(sk, skb)) {
+ MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_RCVPRUNED);
+ mptcp_drop(sk, skb);
+ return;
+ }
+
seq = MPTCP_SKB_CB(skb)->map_seq;
end_seq = MPTCP_SKB_CB(skb)->end_seq;
max_seq = atomic64_read(&msk->rcv_wnd_sent);
@@ -286,15 +351,6 @@ static bool __mptcp_move_skb(struct mptcp_sock *msk, struct sock *ssk,
skb_ext_reset(skb);
skb_orphan(skb);
- /* Can't drop packets for fallback socket this late, or the stream
- * will break.
- */
- if (unlikely(sk_rmem_alloc_get(sk) > READ_ONCE(sk->sk_rcvbuf)) &&
- !__mptcp_check_fallback(msk)) {
- MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_RCVPRUNED);
- goto drop;
- }
-
/* try to fetch required memory from subflow */
if (!sk_rmem_schedule(sk, skb, skb->truesize)) {
MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_RCVPRUNED);
@@ -315,6 +371,11 @@ static bool __mptcp_move_skb(struct mptcp_sock *msk, struct sock *ssk,
if (MPTCP_SKB_CB(skb)->map_seq == msk->ack_seq) {
/* in sequence */
+ if (!mptcp_try_rmem_schedule(sk, skb)) {
+ MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_RCVPRUNED);
+ goto drop;
+ }
+
msk->bytes_received += copy_len;
WRITE_ONCE(msk->ack_seq, msk->ack_seq + copy_len);
tail = skb_peek_tail(&sk->sk_receive_queue);
--
2.56.0
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 17+ messages in thread* [Devel] [PATCH hci-8.1 7/8] ms/mptcp: being below memory limit is a likely() condition
2026-11-06 3:02 [Devel] [PATCH hci-8.1 0/8] ms/mptcp: stability fixes Eva Kurchatova
` (5 preceding siblings ...)
2026-11-06 3:02 ` [Devel] [PATCH hci-8.1 6/8] ms/mptcp: implemented OoO queue pruning Eva Kurchatova
@ 2026-11-06 3:02 ` Eva Kurchatova
2026-11-06 3:10 ` no-reply
2026-11-06 3:02 ` [Devel] [PATCH hci-8.1 8/8] ms/mptcp: avoid pruning for OoW data Eva Kurchatova
7 siblings, 1 reply; 17+ messages in thread
From: Eva Kurchatova @ 2026-11-06 3:02 UTC (permalink / raw)
To: khorenko; +Cc: devel
From: Paolo Abeni <pabeni@redhat.com>
The current compiler hint annotation is wrong, due to inverted
logic in the previous revision of the relevant code.
Fixes: e468d371180d ("mptcp: implemented OoO queue pruning")
Cc: stable@vger.kernel.org
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Reviewed-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
Link: https://patch.msgid.link/20260908-net-mptcp-misc-fixes-7-3-rc1-v2-14-df1de70348b6@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
(cherry picked from commit a4257a91af7a77a8347d33413ec9e54106f7ff48)
https://virtuozzo.atlassian.net/browse/VSTOR-153491
Feature: fix ms/net
Signed-off-by: Eva Kurchatova <eva.kurchatova@virtuozzo.com>
---
net/mptcp/protocol.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index 3b02a63ad1a9..b5aab32a970a 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -203,8 +203,8 @@ static void mptcp_prune_ofo_queue(struct sock *sk,
*/
static bool mptcp_can_ingest(const struct sock *sk)
{
- return unlikely(sk_rmem_alloc_get(sk) <= READ_ONCE(sk->sk_rcvbuf)) ||
- __mptcp_check_fallback(mptcp_sk(sk));
+ return likely(sk_rmem_alloc_get(sk) <= READ_ONCE(sk->sk_rcvbuf)) ||
+ __mptcp_check_fallback(mptcp_sk(sk));
}
static bool mptcp_try_rmem_schedule(struct sock *sk, const struct sk_buff *skb)
--
2.56.0
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 17+ messages in thread* [Devel] [PATCH hci-8.1 8/8] ms/mptcp: avoid pruning for OoW data
2026-11-06 3:02 [Devel] [PATCH hci-8.1 0/8] ms/mptcp: stability fixes Eva Kurchatova
` (6 preceding siblings ...)
2026-11-06 3:02 ` [Devel] [PATCH hci-8.1 7/8] ms/mptcp: being below memory limit is a likely() condition Eva Kurchatova
@ 2026-11-06 3:02 ` Eva Kurchatova
2026-11-06 3:11 ` no-reply
7 siblings, 1 reply; 17+ messages in thread
From: Eva Kurchatova @ 2026-11-06 3:02 UTC (permalink / raw)
To: khorenko; +Cc: devel
From: Paolo Abeni <pabeni@redhat.com>
Pruning is expansive and destructive, do it only when we expect
to accept the skb triggering the cleanup.
Fixes: e468d371180d ("mptcp: implemented OoO queue pruning")
Cc: stable@vger.kernel.org
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Reviewed-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
Link: https://patch.msgid.link/20260908-net-mptcp-misc-fixes-7-3-rc1-v2-15-df1de70348b6@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
(cherry picked from commit f01b8275745efe611284f6c3628099a81a421f0d)
https://virtuozzo.atlassian.net/browse/VSTOR-153491
Feature: fix ms/net
Signed-off-by: Eva Kurchatova <eva.kurchatova@virtuozzo.com>
---
net/mptcp/protocol.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c
index b5aab32a970a..26e0d2c5020b 100644
--- a/net/mptcp/protocol.c
+++ b/net/mptcp/protocol.c
@@ -227,12 +227,6 @@ static void mptcp_data_queue_ofo(struct mptcp_sock *msk, struct sk_buff *skb)
u64 seq, end_seq, max_seq;
struct sk_buff *skb1;
- if (!mptcp_try_rmem_schedule(sk, skb)) {
- MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_RCVPRUNED);
- mptcp_drop(sk, skb);
- return;
- }
-
seq = MPTCP_SKB_CB(skb)->map_seq;
end_seq = MPTCP_SKB_CB(skb)->end_seq;
max_seq = atomic64_read(&msk->rcv_wnd_sent);
@@ -249,6 +243,12 @@ static void mptcp_data_queue_ofo(struct mptcp_sock *msk, struct sk_buff *skb)
return;
}
+ if (!mptcp_try_rmem_schedule(sk, skb)) {
+ MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_RCVPRUNED);
+ mptcp_drop(sk, skb);
+ return;
+ }
+
p = &msk->out_of_order_queue.rb_node;
MPTCP_INC_STATS(sock_net(sk), MPTCP_MIB_OFOQUEUE);
if (RB_EMPTY_ROOT(&msk->out_of_order_queue)) {
--
2.56.0
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 17+ messages in thread