* [Devel] [PATCH hci-8.1 v2 0/3] sched: fix hard lockups on task_lock taken under scheduler and diagnostic locks
@ 2026-11-16 13:44 Denis V. Lunev
2026-11-16 13:44 ` [Devel] [PATCH hci-8.1 v2 1/3] sched/stats: fix a hard lockup on task_lock in update_sched_lat() Denis V. Lunev
` (5 more replies)
0 siblings, 6 replies; 13+ messages in thread
From: Denis V. Lunev @ 2026-11-16 13:44 UTC (permalink / raw)
To: khorenko; +Cc: devel
With kernel.sched_schedstats on, update_sched_lat() takes task_lock
under the runqueue lock, and an hci-8.1 perf-storage node died of a hard
lockup from it. sched_show_task() takes the same lock under rnp->lock in
the RCU stall reports, from sysrq and the workqueue watchdog, and can
print a ve name already freed at cgroup offline.
https://virtuozzo.atlassian.net/browse/VSTOR-155276
https://virtuozzo.atlassian.net/browse/VSTOR-155341
Built; the lock cycle is confirmed in the vmcore, not run on a node.
v2: also fix sched_show_task() and free ve_name with the ve (Vasileios)
v1: http://lore.virtuozzo.com/kernel/20261116101516.154737-1-den@openvz.org/
Denis V. Lunev (3):
sched/stats: fix a hard lockup on task_lock in update_sched_lat()
ve: free ve_name together with the ve, not at cgroup offline
sched: drop task_lock from sched_show_task()
kernel/sched/core.c | 5 ++---
kernel/sched/stats.c | 4 +---
kernel/ve/ve.c | 12 +-----------
3 files changed, 4 insertions(+), 17 deletions(-)
--
2.55.0
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Devel] [PATCH hci-8.1 v2 1/3] sched/stats: fix a hard lockup on task_lock in update_sched_lat()
2026-11-16 13:44 [Devel] [PATCH hci-8.1 v2 0/3] sched: fix hard lockups on task_lock taken under scheduler and diagnostic locks Denis V. Lunev
@ 2026-11-16 13:44 ` Denis V. Lunev
2026-11-16 13:50 ` no-reply
2026-11-16 16:31 ` Konstantin Khorenko
2026-11-16 13:44 ` [Devel] [PATCH hci-8.1 v2 2/3] ve: free ve_name together with the ve, not at cgroup offline Denis V. Lunev
` (4 subsequent siblings)
5 siblings, 2 replies; 13+ messages in thread
From: Denis V. Lunev @ 2026-11-16 13:44 UTC (permalink / raw)
To: khorenko; +Cc: devel
From: Denis V. Lunev <den@openvz.org>
Commit ead7be4fd3ca ("ve: Switch from ->task_ve to *task_ve() helpers")
made update_sched_lat() look the ve up with get_task_ve(), which takes
task_lock. update_sched_lat() runs from __update_stats_wait_end() under
rq->__lock with interrupts off whenever kernel.sched_schedstats is on,
while task_lock is taken elsewhere with interrupts on. Any lock taken
in an interrupt and held across a wakeup then closes a cycle. wbt does
it on an I/O completion:
CPU A: /proc reader CPU B: NVMe irq CPU C: scheduler
------------------- --------------- ----------------
getdents64(/proc/X/fd)
task_lock(X)
nvme_irq()
wbt_done()
lock wq
__schedule()
rq_lock(C)
pick X
nvme_irq()
wbt_done()
lock wq -> spins
wake W, slept on C:
rq_lock(C) -> spins
set_next_entity(X)
update_sched_lat(X)
get_task_ve(X)
task_lock(X)
-> spins
wq is the wbt wait queue of the device and W a writer throttled on it.
W slept in iowait on C, so try_to_wake_up() locks C's rq to move the
iowait count off it before migrating W. Every other CPU completing I/O
on that device then spins on wq too, and the hard lockup detector
panics the node.
The put_ve() under rq->__lock is unsafe as well: dropping the last
reference queues the css release work, and that wakeup may need the rq
lock already held.
Read the pointer with task_ve() and take no reference, as the code did
before that commit. Whoever switches the ve of the task, the old one is
freed from css_free after an RCU grace period, and rq->__lock with
interrupts off is an RCU read-side critical section.
Fixes: ead7be4fd3ca ("ve: Switch from ->task_ve to *task_ve() helpers")
https://virtuozzo.atlassian.net/browse/VSTOR-155276
Feature: ve: ve generic structures
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
kernel/sched/stats.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/kernel/sched/stats.c b/kernel/sched/stats.c
index 08ed7c8a9c4a..ede339b768cf 100644
--- a/kernel/sched/stats.c
+++ b/kernel/sched/stats.c
@@ -23,10 +23,8 @@ void __update_stats_wait_start(struct rq *rq, struct task_struct *p,
static inline void update_sched_lat(struct task_struct *t, u64 delta, u64 now)
{
#ifdef CONFIG_VE
- struct ve_struct *ve __free(put_ve) = get_task_ve(t);
-
KSTAT_LAT_PCPU_ADD(&kstat_glob.sched_lat, delta);
- KSTAT_LAT_PCPU_ADD(&ve->sched_lat_ve, delta);
+ KSTAT_LAT_PCPU_ADD(&task_ve(t)->sched_lat_ve, delta);
t->alloc_lat[KSTAT_SCHED].totlat += delta;
t->alloc_lat[KSTAT_SCHED].count++;
--
2.55.0
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Devel] [PATCH hci-8.1 v2 2/3] ve: free ve_name together with the ve, not at cgroup offline
2026-11-16 13:44 [Devel] [PATCH hci-8.1 v2 0/3] sched: fix hard lockups on task_lock taken under scheduler and diagnostic locks Denis V. Lunev
2026-11-16 13:44 ` [Devel] [PATCH hci-8.1 v2 1/3] sched/stats: fix a hard lockup on task_lock in update_sched_lat() Denis V. Lunev
@ 2026-11-16 13:44 ` Denis V. Lunev
2026-11-16 13:49 ` no-reply
2026-11-16 16:31 ` Konstantin Khorenko
2026-11-16 13:44 ` [Devel] [PATCH hci-8.1 v2 3/3] sched: drop task_lock from sched_show_task() Denis V. Lunev
` (3 subsequent siblings)
5 siblings, 2 replies; 13+ messages in thread
From: Denis V. Lunev @ 2026-11-16 13:44 UTC (permalink / raw)
To: khorenko; +Cc: devel
From: Denis V. Lunev <den@openvz.org>
ve_offline() frees ve->ve_name as soon as the ve cgroup goes offline,
while the ve itself lives on as long as anything holds a reference, a
ve namespace for one. Tasks of that namespace still point at the ve,
and readers of another task's ve name, such as sched_show_task(),
dereference the string with nothing that keeps it from being freed
under them. A reference to the ve does not help either, as it does not
hold the cgroup online.
Free the name in ve_destroy() instead. That is css_free, which runs
after an RCU grace period, so the name is valid for as long as the ve
is and an RCU reader of task_ve() can print it. ve0 is never destroyed
and keeps its static name. ve_offline() is left empty and goes away.
https://virtuozzo.atlassian.net/browse/VSTOR-155341
Feature: ve: ve generic structures
Suggested-by: Vasileios Almpanis <vasileios.almpanis@virtuozzo.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
kernel/ve/ve.c | 12 +-----------
1 file changed, 1 insertion(+), 11 deletions(-)
diff --git a/kernel/ve/ve.c b/kernel/ve/ve.c
index 6cdfaaa4e703..cdfd03416b39 100644
--- a/kernel/ve/ve.c
+++ b/kernel/ve/ve.c
@@ -894,16 +894,6 @@ static int ve_online(struct cgroup_subsys_state *css)
return 0;
}
-static void ve_offline(struct cgroup_subsys_state *css)
-{
- struct ve_struct *ve = css_to_ve(css);
-
- if (!ve_is_super(ve)) {
- kfree(ve->ve_name);
- ve->ve_name = NULL;
- }
-}
-
static void ve_devmnt_free(struct ve_devmnt *devmnt)
{
if (!devmnt)
@@ -939,6 +929,7 @@ static void ve_destroy(struct cgroup_subsys_state *css)
ve_free_vdso(ve);
mntput(ve->devtmpfs_mnt);
free_percpu(ve->sched_lat_ve.cur);
+ kfree(ve->ve_name);
ve_set_state(ve, VE_STATE_DEAD);
kmem_cache_free(ve_cachep, ve);
}
@@ -1690,7 +1681,6 @@ static struct cftype ve_cftypes[] = {
struct cgroup_subsys ve_cgrp_subsys = {
.css_alloc = ve_create,
.css_online = ve_online,
- .css_offline = ve_offline,
.css_free = ve_destroy,
.legacy_cftypes = ve_cftypes,
.dfl_cftypes = ve_cftypes,
--
2.55.0
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 13+ messages in thread
* [Devel] [PATCH hci-8.1 v2 3/3] sched: drop task_lock from sched_show_task()
2026-11-16 13:44 [Devel] [PATCH hci-8.1 v2 0/3] sched: fix hard lockups on task_lock taken under scheduler and diagnostic locks Denis V. Lunev
2026-11-16 13:44 ` [Devel] [PATCH hci-8.1 v2 1/3] sched/stats: fix a hard lockup on task_lock in update_sched_lat() Denis V. Lunev
2026-11-16 13:44 ` [Devel] [PATCH hci-8.1 v2 2/3] ve: free ve_name together with the ve, not at cgroup offline Denis V. Lunev
@ 2026-11-16 13:44 ` Denis V. Lunev
2026-11-16 13:52 ` no-reply
2026-11-16 16:31 ` Konstantin Khorenko
2026-11-16 16:03 ` [Devel] [PATCH hci-8.1 v2 0/3] sched: fix hard lockups on task_lock taken under scheduler and diagnostic locks Vasileios Almpanis
` (2 subsequent siblings)
5 siblings, 2 replies; 13+ messages in thread
From: Denis V. Lunev @ 2026-11-16 13:44 UTC (permalink / raw)
To: khorenko; +Cc: devel
From: Denis V. Lunev <den@openvz.org>
Commit 35518dbbfd76 ("ve: Switch from task_ve_name() to ve_name()")
made sched_show_task() look the ve up with get_task_ve(), which takes
task_lock of the task being shown. sched_show_task() is a diagnostic
called with interrupts off under raw locks or from an interrupt, while
task_lock is taken with interrupts on, by a /proc/PID/fd read for one:
- the RCU stall reports show the tasks blocking the grace period
under rnp->lock, from the scheduler tick;
- dump_cpu_task() shows the current task of a stuck CPU from the
RCU stall and CSD lock reports; when that task is stuck holding
its own task_lock, the report spins forever instead of printing;
- the workqueue watchdog shows a hogging worker under pool->lock;
- sysrq-t runs show_state() from the keyboard or serial interrupt.
An interrupt or softirq on the CPU holding task_lock that needs
rnp->lock or pool->lock closes a lock cycle, and the node locks up
hard in the middle of reporting a stall.
Print the name under the RCU read lock the function already takes for
the parent pid, with task_ve() and no reference. The ve is freed from
css_free after a grace period, and so is its name since "ve: free
ve_name together with the ve, not at cgroup offline".
Fixes: 35518dbbfd76 ("ve: Switch from task_ve_name() to ve_name()")
https://virtuozzo.atlassian.net/browse/VSTOR-155341
Feature: ve: ve generic structures
Suggested-by: Vasileios Almpanis <vasileios.almpanis@virtuozzo.com>
Signed-off-by: Denis V. Lunev <den@openvz.org>
---
kernel/sched/core.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index 77587e9982b7..84d2a5c29c69 100644
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -7777,7 +7777,6 @@ EXPORT_SYMBOL(io_schedule);
void sched_show_task(struct task_struct *p)
{
- struct ve_struct *ve __free(put_ve) = get_task_ve(p);
unsigned long free;
int ppid;
@@ -7793,10 +7792,10 @@ void sched_show_task(struct task_struct *p)
rcu_read_lock();
if (pid_alive(p))
ppid = task_pid_nr(rcu_dereference(p->real_parent));
- rcu_read_unlock();
pr_cont(" stack:%-5lu pid:%-5d tgid:%-5d ppid:%-6d veid:%-4s task_flags:0x%04x flags:0x%08lx\n",
free, task_pid_nr(p), task_tgid_nr(p),
- ppid, ve_name(ve), p->flags, read_task_thread_flags(p));
+ ppid, ve_name(task_ve(p)), p->flags, read_task_thread_flags(p));
+ rcu_read_unlock();
print_worker_info(KERN_INFO, p);
print_stop_info(KERN_INFO, p);
--
2.55.0
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [Devel] [PATCH hci-8.1 v2 2/3] ve: free ve_name together with the ve, not at cgroup offline
2026-11-16 13:44 ` [Devel] [PATCH hci-8.1 v2 2/3] ve: free ve_name together with the ve, not at cgroup offline Denis V. Lunev
@ 2026-11-16 13:49 ` no-reply
2026-11-16 16:31 ` Konstantin Khorenko
1 sibling, 0 replies; 13+ messages in thread
From: no-reply @ 2026-11-16 13:49 UTC (permalink / raw)
To: Denis V. Lunev, khorenko; +Cc: devel
> From: Denis V. Lunev <den@openvz.org>
>
> ve_offline() frees ve->ve_name as soon as the ve cgroup goes offline,
> while the ve itself lives on as long as anything holds a reference, a
> ve namespace for one. Tasks of that namespace still point at the ve,
> and readers of another task's ve name, such as sched_show_task(),
> dereference the string with nothing that keeps it from being freed
> under them. A reference to the ve does not help either, as it does not
> hold the cgroup online.
>
> Free the name in ve_destroy() instead. That is css_free, which runs
> after an RCU grace period, so the name is valid for as long as the ve
> is and an RCU reader of task_ve() can print it. ve0 is never destroyed
> and keeps its static name. ve_offline() is left empty and goes away.
>
> https://virtuozzo.atlassian.net/browse/VSTOR-155341
>
> Feature: ve: ve generic structures
> Suggested-by: Vasileios Almpanis <vasileios.almpanis@virtuozzo.com>
> Signed-off-by: Denis V. Lunev <den@openvz.org>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.vzint.dev/#/patchset/20261116134445.438480-1-den@openvz.org?part=2
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [Devel] [PATCH hci-8.1 v2 1/3] sched/stats: fix a hard lockup on task_lock in update_sched_lat()
2026-11-16 13:44 ` [Devel] [PATCH hci-8.1 v2 1/3] sched/stats: fix a hard lockup on task_lock in update_sched_lat() Denis V. Lunev
@ 2026-11-16 13:50 ` no-reply
2026-11-16 16:31 ` Konstantin Khorenko
1 sibling, 0 replies; 13+ messages in thread
From: no-reply @ 2026-11-16 13:50 UTC (permalink / raw)
To: Denis V. Lunev, khorenko; +Cc: devel
> From: Denis V. Lunev <den@openvz.org>
>
> Commit ead7be4fd3ca ("ve: Switch from ->task_ve to *task_ve() helpers")
> made update_sched_lat() look the ve up with get_task_ve(), which takes
> task_lock. update_sched_lat() runs from __update_stats_wait_end() under
> rq->__lock with interrupts off whenever kernel.sched_schedstats is on,
> while task_lock is taken elsewhere with interrupts on. Any lock taken
> in an interrupt and held across a wakeup then closes a cycle. wbt does
> it on an I/O completion:
>
> CPU A: /proc reader CPU B: NVMe irq CPU C: scheduler
> ------------------- --------------- ----------------
> getdents64(/proc/X/fd)
> task_lock(X)
> nvme_irq()
> [ ... ]
>
> https://virtuozzo.atlassian.net/browse/VSTOR-155276
>
> Feature: ve: ve generic structures
> Signed-off-by: Denis V. Lunev <den@openvz.org>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.vzint.dev/#/patchset/20261116134445.438480-1-den@openvz.org?part=1
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [Devel] [PATCH hci-8.1 v2 3/3] sched: drop task_lock from sched_show_task()
2026-11-16 13:44 ` [Devel] [PATCH hci-8.1 v2 3/3] sched: drop task_lock from sched_show_task() Denis V. Lunev
@ 2026-11-16 13:52 ` no-reply
2026-11-16 16:31 ` Konstantin Khorenko
1 sibling, 0 replies; 13+ messages in thread
From: no-reply @ 2026-11-16 13:52 UTC (permalink / raw)
To: Denis V. Lunev, khorenko; +Cc: devel
> From: Denis V. Lunev <den@openvz.org>
>
> Commit 35518dbbfd76 ("ve: Switch from task_ve_name() to ve_name()")
> made sched_show_task() look the ve up with get_task_ve(), which takes
> task_lock of the task being shown. sched_show_task() is a diagnostic
> called with interrupts off under raw locks or from an interrupt, while
> task_lock is taken with interrupts on, by a /proc/PID/fd read for one:
>
> - the RCU stall reports show the tasks blocking the grace period
> under rnp->lock, from the scheduler tick;
> - dump_cpu_task() shows the current task of a stuck CPU from the
> RCU stall and CSD lock reports; when that task is stuck holding
> its own task_lock, the report spins forever instead of printing;
> - the workqueue watchdog shows a hogging worker under pool->lock;
> - sysrq-t runs show_state() from the keyboard or serial interrupt.
> [ ... ]
> https://virtuozzo.atlassian.net/browse/VSTOR-155341
>
> Feature: ve: ve generic structures
> Suggested-by: Vasileios Almpanis <vasileios.almpanis@virtuozzo.com>
> Signed-off-by: Denis V. Lunev <den@openvz.org>
Sashiko has reviewed this patch and found no issues.
--
Sashiko AI review · https://sashiko.vzint.dev/#/patchset/20261116134445.438480-1-den@openvz.org?part=3
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [Devel] [PATCH hci-8.1 v2 0/3] sched: fix hard lockups on task_lock taken under scheduler and diagnostic locks
2026-11-16 13:44 [Devel] [PATCH hci-8.1 v2 0/3] sched: fix hard lockups on task_lock taken under scheduler and diagnostic locks Denis V. Lunev
` (2 preceding siblings ...)
2026-11-16 13:44 ` [Devel] [PATCH hci-8.1 v2 3/3] sched: drop task_lock from sched_show_task() Denis V. Lunev
@ 2026-11-16 16:03 ` Vasileios Almpanis
2026-11-16 16:31 ` Konstantin Khorenko
2026-11-16 16:31 ` Konstantin Khorenko
5 siblings, 0 replies; 13+ messages in thread
From: Vasileios Almpanis @ 2026-11-16 16:03 UTC (permalink / raw)
To: Denis V. Lunev; +Cc: devel
On Mon, 16 Nov 2026 14:44:42 +0100, Denis V. Lunev <den@openvz.org> wrote:
> sched: fix hard lockups on task_lock taken under scheduler and diagnostic locks
>
> With kernel.sched_schedstats on, update_sched_lat() takes task_lock
> under the runqueue lock, and an hci-8.1 perf-storage node died of a hard
> lockup from it. sched_show_task() takes the same lock under rnp->lock in
> the RCU stall reports, from sysrq and the workqueue watchdog, and can
> print a ve name already freed at cgroup offline.
>
> [...]
Reviewed-by: Vasileios Almpanis <vasileios.almpanis@virtuozzo.com>
--
Vasileios Almpanis <vasileios.almpanis@virtuozzo.com>
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [Devel] [PATCH hci-8.1 v2 1/3] sched/stats: fix a hard lockup on task_lock in update_sched_lat()
2026-11-16 13:44 ` [Devel] [PATCH hci-8.1 v2 1/3] sched/stats: fix a hard lockup on task_lock in update_sched_lat() Denis V. Lunev
2026-11-16 13:50 ` no-reply
@ 2026-11-16 16:31 ` Konstantin Khorenko
1 sibling, 0 replies; 13+ messages in thread
From: Konstantin Khorenko @ 2026-11-16 16:31 UTC (permalink / raw)
To: Denis V. Lunev; +Cc: devel
Applying as is.
On Mon, 16 Nov 2026 14:44:43 +0100, Denis V. Lunev <den@openvz.org> wrote:
> - struct ve_struct *ve __free(put_ve) = get_task_ve(t);
> -
> KSTAT_LAT_PCPU_ADD(&kstat_glob.sched_lat, delta);
> - KSTAT_LAT_PCPU_ADD(&ve->sched_lat_ve, delta);
> + KSTAT_LAT_PCPU_ADD(&task_ve(t)->sched_lat_ve, delta);
Unchanged since v1.
--
Konstantin Khorenko,
Virtuozzo Linux Kernel Team
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [Devel] [PATCH hci-8.1 v2 0/3] sched: fix hard lockups on task_lock taken under scheduler and diagnostic locks
2026-11-16 13:44 [Devel] [PATCH hci-8.1 v2 0/3] sched: fix hard lockups on task_lock taken under scheduler and diagnostic locks Denis V. Lunev
` (3 preceding siblings ...)
2026-11-16 16:03 ` [Devel] [PATCH hci-8.1 v2 0/3] sched: fix hard lockups on task_lock taken under scheduler and diagnostic locks Vasileios Almpanis
@ 2026-11-16 16:31 ` Konstantin Khorenko
2026-11-16 16:31 ` Konstantin Khorenko
5 siblings, 0 replies; 13+ messages in thread
From: Konstantin Khorenko @ 2026-11-16 16:31 UTC (permalink / raw)
To: Denis V. Lunev; +Cc: devel
Applying with these fixes on my side:
- 2/3: added Fixes: fc7da91c1b73
- 3/3: dropped the dump_cpu_task() bullet and extended the paragraph
after the list to the report that interrupts the CPU holding
task_lock
On Mon, 16 Nov 2026 14:44:42 +0100, Denis V. Lunev <den@openvz.org> wrote:
> Denis V. Lunev (3):
> sched/stats: fix a hard lockup on task_lock in update_sched_lat()
> ve: free ve_name together with the ve, not at cgroup offline
> sched: drop task_lock from sched_show_task()
The reasons are in the replies to 2/3 and 3/3.
--
Konstantin Khorenko,
Virtuozzo Linux Kernel Team
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [Devel] [PATCH hci-8.1 v2 2/3] ve: free ve_name together with the ve, not at cgroup offline
2026-11-16 13:44 ` [Devel] [PATCH hci-8.1 v2 2/3] ve: free ve_name together with the ve, not at cgroup offline Denis V. Lunev
2026-11-16 13:49 ` no-reply
@ 2026-11-16 16:31 ` Konstantin Khorenko
1 sibling, 0 replies; 13+ messages in thread
From: Konstantin Khorenko @ 2026-11-16 16:31 UTC (permalink / raw)
To: Denis V. Lunev; +Cc: devel
Applying with these fixes on my side:
- added Fixes: fc7da91c1b73 ("ve: Add ve cgroup and ve_hook subsys")
On Mon, 16 Nov 2026 14:44:44 +0100, Denis V. Lunev <den@openvz.org> wrote:
> ve_offline() frees ve->ve_name as soon as the ve cgroup goes offline,
> while the ve itself lives on as long as anything holds a reference, a
> ve namespace for one.
The free at offline came with fc7da91c1b73, and it is a bug even
without 3/3: a zombie leaves its cgroup in cgroup_exit() but keeps
its ve until release_task(), so once the CT cgroup is removed,
/proc/<zombie>/status prints envID from freed memory.
--
Konstantin Khorenko,
Virtuozzo Linux Kernel Team
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [Devel] [PATCH hci-8.1 v2 3/3] sched: drop task_lock from sched_show_task()
2026-11-16 13:44 ` [Devel] [PATCH hci-8.1 v2 3/3] sched: drop task_lock from sched_show_task() Denis V. Lunev
2026-11-16 13:52 ` no-reply
@ 2026-11-16 16:31 ` Konstantin Khorenko
1 sibling, 0 replies; 13+ messages in thread
From: Konstantin Khorenko @ 2026-11-16 16:31 UTC (permalink / raw)
To: Denis V. Lunev; +Cc: devel
Applying with these fixes on my side:
- dropped the dump_cpu_task() bullet
- the paragraph after the list now also covers a report that
interrupts the CPU holding task_lock
On Mon, 16 Nov 2026 14:44:45 +0100, Denis V. Lunev <den@openvz.org> wrote:
> - dump_cpu_task() shows the current task of a stuck CPU from the
> RCU stall and CSD lock reports; when that task is stuck holding
> its own task_lock, the report spins forever instead of printing;
dump_cpu_task() calls sched_show_task() only when
trigger_single_cpu_backtrace() returns false, and on x86_64 and arm64
it always returns true. This report never gets there on our kernels.
> An interrupt or softirq on the CPU holding task_lock that needs
> rnp->lock or pool->lock closes a lock cycle, and the node locks up
> hard in the middle of reporting a stall.
sysrq-t takes neither lock: it locks up when it interrupts the CPU
that holds task_lock, and so does the RCU stall report from the tick.
The paragraph now reads:
A report run from an interrupt on the CPU holding task_lock spins on
it, and an interrupt or softirq there that needs rnp->lock or
pool->lock closes a lock cycle. Either way the node locks up hard in
the middle of reporting a stall.
--
Konstantin Khorenko,
Virtuozzo Linux Kernel Team
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [Devel] [PATCH hci-8.1 v2 0/3] sched: fix hard lockups on task_lock taken under scheduler and diagnostic locks
2026-11-16 13:44 [Devel] [PATCH hci-8.1 v2 0/3] sched: fix hard lockups on task_lock taken under scheduler and diagnostic locks Denis V. Lunev
` (4 preceding siblings ...)
2026-11-16 16:31 ` Konstantin Khorenko
@ 2026-11-16 16:31 ` Konstantin Khorenko
5 siblings, 0 replies; 13+ messages in thread
From: Konstantin Khorenko @ 2026-11-16 16:31 UTC (permalink / raw)
To: Denis V. Lunev; +Cc: devel
On Mon, 16 Nov 2026 14:44:42 +0100, Denis V. Lunev wrote:
> sched: fix hard lockups on task_lock taken under scheduler and diagnostic locks
>
> With kernel.sched_schedstats on, update_sched_lat() takes task_lock
> under the runqueue lock, and an hci-8.1 perf-storage node died of a hard
> lockup from it. sched_show_task() takes the same lock under rnp->lock in
> the RCU stall reports, from sysrq and the workqueue watchdog, and can
> print a ve name already freed at cgroup offline.
>
> [...]
Applied to branch-rh10-6.12.0-211.75.1.24.x.vz10-ovz, will be released in rh10-6.12.0-211.75.1.24.2.vz10, thanks!
[1/3] sched/stats: fix a hard lockup on task_lock in update_sched_lat()
https://bitbucket.org/virtuozzocore/vzkernel.vzs/commits/d30fdeac98c0
[2/3] ve: free ve_name together with the ve, not at cgroup offline
https://bitbucket.org/virtuozzocore/vzkernel.vzs/commits/628eb7c4c1ba
[3/3] sched: drop task_lock from sched_show_task()
https://bitbucket.org/virtuozzocore/vzkernel.vzs/commits/faac659ea3a6
--
Konstantin Khorenko,
Virtuozzo Linux Kernel Team
_______________________________________________
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2026-11-16 16:32 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-11-16 13:44 [Devel] [PATCH hci-8.1 v2 0/3] sched: fix hard lockups on task_lock taken under scheduler and diagnostic locks Denis V. Lunev
2026-11-16 13:44 ` [Devel] [PATCH hci-8.1 v2 1/3] sched/stats: fix a hard lockup on task_lock in update_sched_lat() Denis V. Lunev
2026-11-16 13:50 ` no-reply
2026-11-16 16:31 ` Konstantin Khorenko
2026-11-16 13:44 ` [Devel] [PATCH hci-8.1 v2 2/3] ve: free ve_name together with the ve, not at cgroup offline Denis V. Lunev
2026-11-16 13:49 ` no-reply
2026-11-16 16:31 ` Konstantin Khorenko
2026-11-16 13:44 ` [Devel] [PATCH hci-8.1 v2 3/3] sched: drop task_lock from sched_show_task() Denis V. Lunev
2026-11-16 13:52 ` no-reply
2026-11-16 16:31 ` Konstantin Khorenko
2026-11-16 16:03 ` [Devel] [PATCH hci-8.1 v2 0/3] sched: fix hard lockups on task_lock taken under scheduler and diagnostic locks Vasileios Almpanis
2026-11-16 16:31 ` Konstantin Khorenko
2026-11-16 16:31 ` Konstantin Khorenko
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox