OpenVZ / Virtuozzo kernel development (devel@openvz.org)
 help / color / mirror / Atom feed
From: "Denis V. Lunev" <den@openvz.org>
To: khorenko@virtuozzo.com
Cc: devel@openvz.org
Subject: [Devel] [PATCH hci-8.1 v2 1/3] sched/stats: fix a hard lockup on task_lock in update_sched_lat()
Date: Mon, 16 Nov 2026 14:44:43 +0100	[thread overview]
Message-ID: <20261116134445.438480-2-den@openvz.org> (raw)
In-Reply-To: <20261116134445.438480-1-den@openvz.org>

From: Denis V. Lunev <den@openvz.org>

Commit ead7be4fd3ca ("ve: Switch from ->task_ve to *task_ve() helpers")
made update_sched_lat() look the ve up with get_task_ve(), which takes
task_lock. update_sched_lat() runs from __update_stats_wait_end() under
rq->__lock with interrupts off whenever kernel.sched_schedstats is on,
while task_lock is taken elsewhere with interrupts on. Any lock taken
in an interrupt and held across a wakeup then closes a cycle. wbt does
it on an I/O completion:

  CPU A: /proc reader     CPU B: NVMe irq         CPU C: scheduler
  -------------------     ---------------         ----------------
  getdents64(/proc/X/fd)
    task_lock(X)
                          nvme_irq()
                            wbt_done()
                              lock wq
                                                  __schedule()
                                                    rq_lock(C)
                                                    pick X
  nvme_irq()
    wbt_done()
      lock wq  -> spins
                              wake W, slept on C:
                                rq_lock(C) -> spins
                                                    set_next_entity(X)
                                                      update_sched_lat(X)
                                                        get_task_ve(X)
                                                          task_lock(X)
                                                            -> spins

wq is the wbt wait queue of the device and W a writer throttled on it.
W slept in iowait on C, so try_to_wake_up() locks C's rq to move the
iowait count off it before migrating W. Every other CPU completing I/O
on that device then spins on wq too, and the hard lockup detector
panics the node.

The put_ve() under rq->__lock is unsafe as well: dropping the last
reference queues the css release work, and that wakeup may need the rq
lock already held.

Read the pointer with task_ve() and take no reference, as the code did
before that commit. Whoever switches the ve of the task, the old one is
freed from css_free after an RCU grace period, and rq->__lock with
interrupts off is an RCU read-side critical section.

Fixes: ead7be4fd3ca ("ve: Switch from ->task_ve to *task_ve() helpers")

https://virtuozzo.atlassian.net/browse/VSTOR-155276

Feature: ve: ve generic structures
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
 kernel/sched/stats.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/kernel/sched/stats.c b/kernel/sched/stats.c
index 08ed7c8a9c4a..ede339b768cf 100644
--- a/kernel/sched/stats.c
+++ b/kernel/sched/stats.c
@@ -23,10 +23,8 @@ void __update_stats_wait_start(struct rq *rq, struct task_struct *p,
 static inline void update_sched_lat(struct task_struct *t, u64 delta, u64 now)
 {
 #ifdef CONFIG_VE
-	struct ve_struct *ve __free(put_ve) = get_task_ve(t);
-
 	KSTAT_LAT_PCPU_ADD(&kstat_glob.sched_lat, delta);
-	KSTAT_LAT_PCPU_ADD(&ve->sched_lat_ve, delta);
+	KSTAT_LAT_PCPU_ADD(&task_ve(t)->sched_lat_ve, delta);
 
 	t->alloc_lat[KSTAT_SCHED].totlat += delta;
 	t->alloc_lat[KSTAT_SCHED].count++;
-- 
2.55.0

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

  reply	other threads:[~2026-11-16 13:46 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-16 13:44 [Devel] [PATCH hci-8.1 v2 0/3] sched: fix hard lockups on task_lock taken under scheduler and diagnostic locks Denis V. Lunev
2026-11-16 13:44 ` Denis V. Lunev [this message]
2026-11-16 13:50   ` [Devel] [PATCH hci-8.1 v2 1/3] sched/stats: fix a hard lockup on task_lock in update_sched_lat() no-reply
2026-11-16 16:31   ` Konstantin Khorenko
2026-11-16 13:44 ` [Devel] [PATCH hci-8.1 v2 2/3] ve: free ve_name together with the ve, not at cgroup offline Denis V. Lunev
2026-11-16 13:49   ` no-reply
2026-11-16 16:31   ` Konstantin Khorenko
2026-11-16 13:44 ` [Devel] [PATCH hci-8.1 v2 3/3] sched: drop task_lock from sched_show_task() Denis V. Lunev
2026-11-16 13:52   ` no-reply
2026-11-16 16:31   ` Konstantin Khorenko
2026-11-16 16:03 ` [Devel] [PATCH hci-8.1 v2 0/3] sched: fix hard lockups on task_lock taken under scheduler and diagnostic locks Vasileios Almpanis
2026-11-16 16:31 ` Konstantin Khorenko
2026-11-16 16:31 ` Konstantin Khorenko

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261116134445.438480-2-den@openvz.org \
    --to=den@openvz.org \
    --cc=devel@openvz.org \
    --cc=khorenko@virtuozzo.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox