All Virtuozzo development lists (kernel + QEMU)
 help / color / mirror / Atom feed
From: "Denis V. Lunev" <den@openvz.org>
To: khorenko@virtuozzo.com
Cc: devel@openvz.org
Subject: [Devel] [PATCH hci-8.1 v2 1/3] sched/stats: fix a hard lockup on task_lock in update_sched_lat()
Date: Mon, 16 Nov 2026 14:44:43 +0100	[thread overview]
Message-ID: <20261116134445.438480-2-den@openvz.org> (raw)
In-Reply-To: <20261116134445.438480-1-den@openvz.org>

From: Denis V. Lunev <den@openvz.org>

Commit ead7be4fd3ca ("ve: Switch from ->task_ve to *task_ve() helpers")
made update_sched_lat() look the ve up with get_task_ve(), which takes
task_lock. update_sched_lat() runs from __update_stats_wait_end() under
rq->__lock with interrupts off whenever kernel.sched_schedstats is on,
while task_lock is taken elsewhere with interrupts on. Any lock taken
in an interrupt and held across a wakeup then closes a cycle. wbt does
it on an I/O completion:

  CPU A: /proc reader     CPU B: NVMe irq         CPU C: scheduler
  -------------------     ---------------         ----------------
  getdents64(/proc/X/fd)
    task_lock(X)
                          nvme_irq()
                            wbt_done()
                              lock wq
                                                  __schedule()
                                                    rq_lock(C)
                                                    pick X
  nvme_irq()
    wbt_done()
      lock wq  -> spins
                              wake W, slept on C:
                                rq_lock(C) -> spins
                                                    set_next_entity(X)
                                                      update_sched_lat(X)
                                                        get_task_ve(X)
                                                          task_lock(X)
                                                            -> spins

wq is the wbt wait queue of the device and W a writer throttled on it.
W slept in iowait on C, so try_to_wake_up() locks C's rq to move the
iowait count off it before migrating W. Every other CPU completing I/O
on that device then spins on wq too, and the hard lockup detector
panics the node.

The put_ve() under rq->__lock is unsafe as well: dropping the last
reference queues the css release work, and that wakeup may need the rq
lock already held.

Read the pointer with task_ve() and take no reference, as the code did
before that commit. Whoever switches the ve of the task, the old one is
freed from css_free after an RCU grace period, and rq->__lock with
interrupts off is an RCU read-side critical section.

Fixes: ead7be4fd3ca ("ve: Switch from ->task_ve to *task_ve() helpers")

https://virtuozzo.atlassian.net/browse/VSTOR-155276

Feature: ve: ve generic structures
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
 kernel/sched/stats.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/kernel/sched/stats.c b/kernel/sched/stats.c
index 08ed7c8a9c4a..ede339b768cf 100644
--- a/kernel/sched/stats.c
+++ b/kernel/sched/stats.c
@@ -23,10 +23,8 @@ void __update_stats_wait_start(struct rq *rq, struct task_struct *p,
 static inline void update_sched_lat(struct task_struct *t, u64 delta, u64 now)
 {
 #ifdef CONFIG_VE
-	struct ve_struct *ve __free(put_ve) = get_task_ve(t);
-
 	KSTAT_LAT_PCPU_ADD(&kstat_glob.sched_lat, delta);
-	KSTAT_LAT_PCPU_ADD(&ve->sched_lat_ve, delta);
+	KSTAT_LAT_PCPU_ADD(&task_ve(t)->sched_lat_ve, delta);
 
 	t->alloc_lat[KSTAT_SCHED].totlat += delta;
 	t->alloc_lat[KSTAT_SCHED].count++;
-- 
2.55.0

_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel

  reply	other threads:[~2026-11-16 13:46 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-16 13:44 [Devel] [PATCH hci-8.1 v2 0/3] sched: fix hard lockups on task_lock taken under scheduler and diagnostic locks Denis V. Lunev
2026-11-16 13:44 ` Denis V. Lunev [this message]
2026-11-16 13:50   ` [Devel] [PATCH hci-8.1 v2 1/3] sched/stats: fix a hard lockup on task_lock in update_sched_lat() no-reply
2026-11-16 16:31   ` Konstantin Khorenko
2026-11-16 13:44 ` [Devel] [PATCH hci-8.1 v2 2/3] ve: free ve_name together with the ve, not at cgroup offline Denis V. Lunev
2026-11-16 13:49   ` no-reply
2026-11-16 16:31   ` Konstantin Khorenko
2026-11-16 13:44 ` [Devel] [PATCH hci-8.1 v2 3/3] sched: drop task_lock from sched_show_task() Denis V. Lunev
2026-11-16 13:52   ` no-reply
2026-11-16 16:31   ` Konstantin Khorenko
2026-11-16 16:03 ` [Devel] [PATCH hci-8.1 v2 0/3] sched: fix hard lockups on task_lock taken under scheduler and diagnostic locks Vasileios Almpanis
2026-11-16 16:31 ` Konstantin Khorenko
2026-11-16 16:31 ` Konstantin Khorenko

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261116134445.438480-2-den@openvz.org \
    --to=den@openvz.org \
    --cc=devel@openvz.org \
    --cc=khorenko@virtuozzo.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.